[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:17:35.730201 14429 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.23.126:34187
I20260709 13:17:35.731786 14429 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:17:35.732875 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:35.749365 14436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:35.749424 14438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:35.751010 14429 server_base.cc:1061] running on GCE node
W20260709 13:17:35.749394 14435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:36.861663 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:36.861956 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:36.862155 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603056862132 us; error 0 us; skew 500 ppm
I20260709 13:17:36.862988 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:36.868474 14429 webserver.cc:533] Webserver started at http://127.14.23.126:46791/ using document root <none> and password file <none>
I20260709 13:17:36.869376 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:36.869614 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:36.870100 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:36.875921 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603055563165-14429-0/minicluster-data/master-0-root/instance:
uuid: "0c9c7bb62d7a4ab4947c8a1a6f8c8748"
format_stamp: "Formatted at 2026-07-09 13:17:36 on dist-test-slave-twc1"
I20260709 13:17:36.884105 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.009s	sys 0.000s
I20260709 13:17:36.889613 14444 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:36.890857 14429 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:17:36.891233 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603055563165-14429-0/minicluster-data/master-0-root
uuid: "0c9c7bb62d7a4ab4947c8a1a6f8c8748"
format_stamp: "Formatted at 2026-07-09 13:17:36 on dist-test-slave-twc1"
I20260709 13:17:36.891593 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603055563165-14429-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603055563165-14429-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603055563165-14429-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:36.974687 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:36.976337 14429 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:17:36.976814 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:37.048280 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.126:34187
I20260709 13:17:37.048345 14495 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.126:34187 every 8 connection(s)
I20260709 13:17:37.056094 14496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.078050 14496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748: Bootstrap starting.
I20260709 13:17:37.084057 14496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.086006 14496 log.cc:826] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748: Log is configured to *not* fsync() on all Append() calls
I20260709 13:17:37.090896 14496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748: No bootstrap required, opened a new log
I20260709 13:17:37.111723 14496 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9c7bb62d7a4ab4947c8a1a6f8c8748" member_type: VOTER }
I20260709 13:17:37.112285 14496 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.112534 14496 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c9c7bb62d7a4ab4947c8a1a6f8c8748, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.113447 14496 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9c7bb62d7a4ab4947c8a1a6f8c8748" member_type: VOTER }
I20260709 13:17:37.113983 14496 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:17:37.114289 14496 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:17:37.114637 14496 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:37.118845 14496 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9c7bb62d7a4ab4947c8a1a6f8c8748" member_type: VOTER }
I20260709 13:17:37.119694 14496 leader_election.cc:304] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c9c7bb62d7a4ab4947c8a1a6f8c8748; no voters: 
I20260709 13:17:37.121510 14496 leader_election.cc:290] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:17:37.121814 14499 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:37.124538 14499 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748 [term 1 LEADER]: Becoming Leader. State: Replica: 0c9c7bb62d7a4ab4947c8a1a6f8c8748, State: Running, Role: LEADER
I20260709 13:17:37.125483 14499 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9c7bb62d7a4ab4947c8a1a6f8c8748" member_type: VOTER }
I20260709 13:17:37.126693 14496 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:17:37.135953 14500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c9c7bb62d7a4ab4947c8a1a6f8c8748" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9c7bb62d7a4ab4947c8a1a6f8c8748" member_type: VOTER } }
I20260709 13:17:37.136291 14501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c9c7bb62d7a4ab4947c8a1a6f8c8748. Latest consensus state: current_term: 1 leader_uuid: "0c9c7bb62d7a4ab4947c8a1a6f8c8748" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9c7bb62d7a4ab4947c8a1a6f8c8748" member_type: VOTER } }
I20260709 13:17:37.136795 14501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748 [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:37.136799 14500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748 [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:37.140314 14508 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:17:37.153239 14508 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:17:37.155637 14429 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:17:37.171437 14508 catalog_manager.cc:1383] Generated new cluster ID: 92daaec18c3e46dcbf09d3536ab03df4
I20260709 13:17:37.171792 14508 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:17:37.199326 14508 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:17:37.200917 14508 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:17:37.222416 14508 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748: Generated new TSK 0
I20260709 13:17:37.223415 14508 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:17:37.286597 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:37.295197 14518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:37.296795 14519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:37.300264 14429 server_base.cc:1061] running on GCE node
W20260709 13:17:37.300292 14521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:37.301321 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:37.301535 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:37.301710 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603057301691 us; error 0 us; skew 500 ppm
I20260709 13:17:37.302343 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:37.306933 14429 webserver.cc:533] Webserver started at http://127.14.23.65:36261/ using document root <none> and password file <none>
I20260709 13:17:37.307663 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:37.307930 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:37.308233 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:37.309365 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603055563165-14429-0/minicluster-data/ts-0-root/instance:
uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
format_stamp: "Formatted at 2026-07-09 13:17:37 on dist-test-slave-twc1"
I20260709 13:17:37.315307 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.001s
I20260709 13:17:37.319865 14526 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:37.320691 14429 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:17:37.321010 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603055563165-14429-0/minicluster-data/ts-0-root
uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
format_stamp: "Formatted at 2026-07-09 13:17:37 on dist-test-slave-twc1"
I20260709 13:17:37.321362 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603055563165-14429-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603055563165-14429-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603055563165-14429-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:37.337477 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:37.338928 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:37.341632 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:37.346397 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:37.346638 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:37.346942 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:37.347101 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:37.444517 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.65:34713
I20260709 13:17:37.444608 14588 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.65:34713 every 8 connection(s)
I20260709 13:17:37.451680 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:37.463573 14593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:37.465303 14594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:37.469969 14596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:37.471385 14429 server_base.cc:1061] running on GCE node
I20260709 13:17:37.472466 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:37.472751 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:37.472955 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603057472933 us; error 0 us; skew 500 ppm
I20260709 13:17:37.473734 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:37.477046 14429 webserver.cc:533] Webserver started at http://127.14.23.66:43773/ using document root <none> and password file <none>
I20260709 13:17:37.477626 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:37.477885 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:37.478276 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:37.479632 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603055563165-14429-0/minicluster-data/ts-1-root/instance:
uuid: "d3cd29370cc54e1099c022714674d225"
format_stamp: "Formatted at 2026-07-09 13:17:37 on dist-test-slave-twc1"
I20260709 13:17:37.481490 14589 heartbeater.cc:344] Connected to a master server at 127.14.23.126:34187
I20260709 13:17:37.482026 14589 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:37.483290 14589 heartbeater.cc:507] Master 127.14.23.126:34187 requested a full tablet report, sending...
I20260709 13:17:37.487078 14461 ts_manager.cc:194] Registered new tserver with Master: 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713)
I20260709 13:17:37.487413 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.005s	sys 0.004s
I20260709 13:17:37.490258 14461 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48846
I20260709 13:17:37.492862 14601 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:37.493872 14429 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 13:17:37.494256 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603055563165-14429-0/minicluster-data/ts-1-root
uuid: "d3cd29370cc54e1099c022714674d225"
format_stamp: "Formatted at 2026-07-09 13:17:37 on dist-test-slave-twc1"
I20260709 13:17:37.494546 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603055563165-14429-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603055563165-14429-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603055563165-14429-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:37.514591 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:37.516001 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:37.518415 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:37.521281 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:37.521520 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:37.521790 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:37.521965 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:17:37.613018 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.66:40643
I20260709 13:17:37.613358 14664 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.66:40643 every 8 connection(s)
I20260709 13:17:37.620216 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:37.630162 14668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:37.633006 14669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:37.635746 14665 heartbeater.cc:344] Connected to a master server at 127.14.23.126:34187
I20260709 13:17:37.636232 14665 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:37.637163 14665 heartbeater.cc:507] Master 127.14.23.126:34187 requested a full tablet report, sending...
I20260709 13:17:37.640004 14461 ts_manager.cc:194] Registered new tserver with Master: d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643)
W20260709 13:17:37.642084 14671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:37.642275 14461 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48860
I20260709 13:17:37.643231 14429 server_base.cc:1061] running on GCE node
I20260709 13:17:37.644407 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:37.644670 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:37.644912 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603057644888 us; error 0 us; skew 500 ppm
I20260709 13:17:37.645476 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:37.648201 14429 webserver.cc:533] Webserver started at http://127.14.23.67:39303/ using document root <none> and password file <none>
I20260709 13:17:37.648841 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:37.649080 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:37.649389 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:37.650576 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603055563165-14429-0/minicluster-data/ts-2-root/instance:
uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
format_stamp: "Formatted at 2026-07-09 13:17:37 on dist-test-slave-twc1"
I20260709 13:17:37.655138 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 13:17:37.658586 14676 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:37.659379 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.002s
I20260709 13:17:37.659654 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603055563165-14429-0/minicluster-data/ts-2-root
uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
format_stamp: "Formatted at 2026-07-09 13:17:37 on dist-test-slave-twc1"
I20260709 13:17:37.659925 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603055563165-14429-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603055563165-14429-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603055563165-14429-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:37.683874 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:37.685151 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:37.686977 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:37.689847 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:37.690088 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:37.690341 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:37.690481 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:37.788658 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.67:42673
I20260709 13:17:37.788746 14738 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.67:42673 every 8 connection(s)
I20260709 13:17:37.805425 14739 heartbeater.cc:344] Connected to a master server at 127.14.23.126:34187
I20260709 13:17:37.805789 14739 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:37.806623 14739 heartbeater.cc:507] Master 127.14.23.126:34187 requested a full tablet report, sending...
I20260709 13:17:37.808838 14461 ts_manager.cc:194] Registered new tserver with Master: ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:37.809744 14429 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016453929s
I20260709 13:17:37.810504 14461 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48870
I20260709 13:17:37.863577 14461 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48884:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:37.867304 14461 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:37.976794 14704 tablet_service.cc:1511] Processing CreateTablet for tablet b395b43c234641fc985583e8ec908250 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:37.977582 14702 tablet_service.cc:1511] Processing CreateTablet for tablet a36368cb545c46bd929582b28a3fece5 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:37.985209 14702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a36368cb545c46bd929582b28a3fece5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.984912 14703 tablet_service.cc:1511] Processing CreateTablet for tablet 5c4c219a8c864700b9f22ff2a9f23e68 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:37.986222 14700 tablet_service.cc:1511] Processing CreateTablet for tablet accbd816047c4054a6f506623e82f905 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:37.988315 14701 tablet_service.cc:1511] Processing CreateTablet for tablet b42fa1c328df4d64818288ee87e30a46 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:37.990921 14703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c4c219a8c864700b9f22ff2a9f23e68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.991616 14701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b42fa1c328df4d64818288ee87e30a46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.994089 14699 tablet_service.cc:1511] Processing CreateTablet for tablet 5d1ea631a1bb4cf8a4f25e155142c0d3 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:37.995149 14700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet accbd816047c4054a6f506623e82f905. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.995852 14699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d1ea631a1bb4cf8a4f25e155142c0d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.996785 14704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b395b43c234641fc985583e8ec908250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.008283 14698 tablet_service.cc:1511] Processing CreateTablet for tablet 9bfd1e022dbc48a7805a15dbf9324642 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:38.019923 14698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bfd1e022dbc48a7805a15dbf9324642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.018867 14628 tablet_service.cc:1511] Processing CreateTablet for tablet 5c4c219a8c864700b9f22ff2a9f23e68 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:38.030146 14626 tablet_service.cc:1511] Processing CreateTablet for tablet b42fa1c328df4d64818288ee87e30a46 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:38.032506 14626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b42fa1c328df4d64818288ee87e30a46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.020954 14553 tablet_service.cc:1511] Processing CreateTablet for tablet 5c4c219a8c864700b9f22ff2a9f23e68 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:38.018276 14629 tablet_service.cc:1511] Processing CreateTablet for tablet b395b43c234641fc985583e8ec908250 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:38.037031 14553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c4c219a8c864700b9f22ff2a9f23e68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.040560 14554 tablet_service.cc:1511] Processing CreateTablet for tablet b395b43c234641fc985583e8ec908250 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:38.042013 14552 tablet_service.cc:1511] Processing CreateTablet for tablet a36368cb545c46bd929582b28a3fece5 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:38.043164 14554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b395b43c234641fc985583e8ec908250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.046888 14628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c4c219a8c864700b9f22ff2a9f23e68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.046749 14627 tablet_service.cc:1511] Processing CreateTablet for tablet a36368cb545c46bd929582b28a3fece5 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:38.051489 14625 tablet_service.cc:1511] Processing CreateTablet for tablet accbd816047c4054a6f506623e82f905 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:38.055213 14551 tablet_service.cc:1511] Processing CreateTablet for tablet b42fa1c328df4d64818288ee87e30a46 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:38.061578 14629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b395b43c234641fc985583e8ec908250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.057528 14550 tablet_service.cc:1511] Processing CreateTablet for tablet accbd816047c4054a6f506623e82f905 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:38.065701 14552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a36368cb545c46bd929582b28a3fece5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.068490 14624 tablet_service.cc:1511] Processing CreateTablet for tablet 5d1ea631a1bb4cf8a4f25e155142c0d3 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:38.073071 14551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b42fa1c328df4d64818288ee87e30a46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.089972 14549 tablet_service.cc:1511] Processing CreateTablet for tablet 5d1ea631a1bb4cf8a4f25e155142c0d3 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:38.092825 14627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a36368cb545c46bd929582b28a3fece5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.093060 14550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet accbd816047c4054a6f506623e82f905. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.103504 14547 tablet_service.cc:1511] Processing CreateTablet for tablet 2c99ec13aa4c4ff5af744e67c5164098 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:38.105785 14622 tablet_service.cc:1511] Processing CreateTablet for tablet 2c99ec13aa4c4ff5af744e67c5164098 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:38.108242 14623 tablet_service.cc:1511] Processing CreateTablet for tablet 9bfd1e022dbc48a7805a15dbf9324642 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:38.089648 14548 tablet_service.cc:1511] Processing CreateTablet for tablet 9bfd1e022dbc48a7805a15dbf9324642 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:38.115605 14547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c99ec13aa4c4ff5af744e67c5164098. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.115839 14624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d1ea631a1bb4cf8a4f25e155142c0d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.123230 14623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bfd1e022dbc48a7805a15dbf9324642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.126642 14548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bfd1e022dbc48a7805a15dbf9324642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.136176 14756 tablet_bootstrap.cc:492] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff: Bootstrap starting.
I20260709 13:17:38.140856 14549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d1ea631a1bb4cf8a4f25e155142c0d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.137072 14622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c99ec13aa4c4ff5af744e67c5164098. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:17:38.145478 14509 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:38.141675 14702 tablet_service.cc:1511] Processing CreateTablet for tablet 2c99ec13aa4c4ff5af744e67c5164098 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:38.149610 14702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c99ec13aa4c4ff5af744e67c5164098. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.153362 14546 tablet_service.cc:1511] Processing CreateTablet for tablet c246b0d9436a4b30819c7ce5f07fbd77 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:38.154551 14621 tablet_service.cc:1511] Processing CreateTablet for tablet c246b0d9436a4b30819c7ce5f07fbd77 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:38.155779 14546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c246b0d9436a4b30819c7ce5f07fbd77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.170011 14625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet accbd816047c4054a6f506623e82f905. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.185962 14621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c246b0d9436a4b30819c7ce5f07fbd77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.201253 14699 tablet_service.cc:1511] Processing CreateTablet for tablet c246b0d9436a4b30819c7ce5f07fbd77 (DEFAULT_TABLE table=test-workload [id=d4be0d1c9d32421a99b304a997221f5f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:38.203195 14699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c246b0d9436a4b30819c7ce5f07fbd77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.243093 14757 tablet_bootstrap.cc:492] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3: Bootstrap starting.
I20260709 13:17:38.263044 14758 tablet_bootstrap.cc:492] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225: Bootstrap starting.
I20260709 13:17:38.270818 14756 tablet_bootstrap.cc:654] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.301805 14757 tablet_bootstrap.cc:654] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.310456 14758 tablet_bootstrap.cc:654] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.311595 14757 tablet_bootstrap.cc:492] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3: No bootstrap required, opened a new log
I20260709 13:17:38.312136 14757 ts_tablet_manager.cc:1403] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3: Time spent bootstrapping tablet: real 0.073s	user 0.007s	sys 0.005s
I20260709 13:17:38.315263 14757 raft_consensus.cc:359] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } }
I20260709 13:17:38.316203 14757 raft_consensus.cc:385] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.318447 14757 raft_consensus.cc:740] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.319527 14757 consensus_queue.cc:260] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } }
I20260709 13:17:38.321780 14758 tablet_bootstrap.cc:492] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225: No bootstrap required, opened a new log
I20260709 13:17:38.322374 14758 ts_tablet_manager.cc:1403] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225: Time spent bootstrapping tablet: real 0.082s	user 0.006s	sys 0.009s
I20260709 13:17:38.322865 14756 tablet_bootstrap.cc:492] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff: No bootstrap required, opened a new log
I20260709 13:17:38.323441 14756 ts_tablet_manager.cc:1403] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff: Time spent bootstrapping tablet: real 0.193s	user 0.016s	sys 0.004s
I20260709 13:17:38.325611 14758 raft_consensus.cc:359] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.326522 14758 raft_consensus.cc:385] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.326891 14758 raft_consensus.cc:740] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3cd29370cc54e1099c022714674d225, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.326766 14756 raft_consensus.cc:359] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.328115 14756 raft_consensus.cc:385] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.328653 14756 raft_consensus.cc:740] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac3d4d81e96f412caaff09b3861ef2ff, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.328459 14758 consensus_queue.cc:260] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.330044 14589 heartbeater.cc:499] Master 127.14.23.126:34187 was elected leader, sending a full tablet report...
I20260709 13:17:38.330298 14757 ts_tablet_manager.cc:1434] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3: Time spent starting tablet: real 0.018s	user 0.002s	sys 0.014s
I20260709 13:17:38.329984 14756 consensus_queue.cc:260] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.332106 14757 tablet_bootstrap.cc:492] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3: Bootstrap starting.
I20260709 13:17:38.342873 14757 tablet_bootstrap.cc:654] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.347366 14665 heartbeater.cc:499] Master 127.14.23.126:34187 was elected leader, sending a full tablet report...
I20260709 13:17:38.347699 14758 ts_tablet_manager.cc:1434] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.014s
I20260709 13:17:38.348482 14756 ts_tablet_manager.cc:1434] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.014s
I20260709 13:17:38.348577 14758 tablet_bootstrap.cc:492] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225: Bootstrap starting.
I20260709 13:17:38.349756 14756 tablet_bootstrap.cc:492] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff: Bootstrap starting.
I20260709 13:17:38.351023 14739 heartbeater.cc:499] Master 127.14.23.126:34187 was elected leader, sending a full tablet report...
I20260709 13:17:38.355345 14758 tablet_bootstrap.cc:654] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.359395 14764 raft_consensus.cc:493] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.359938 14764 raft_consensus.cc:515] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.360898 14756 tablet_bootstrap.cc:654] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.363209 14764 leader_election.cc:290] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713), d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643)
W20260709 13:17:38.369330 14666 tablet.cc:2406] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:17:38.373512 14756 tablet_bootstrap.cc:492] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff: No bootstrap required, opened a new log
I20260709 13:17:38.374192 14756 ts_tablet_manager.cc:1403] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.007s
I20260709 13:17:38.377486 14756 raft_consensus.cc:359] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.378334 14756 raft_consensus.cc:385] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.378659 14756 raft_consensus.cc:740] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac3d4d81e96f412caaff09b3861ef2ff, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.379545 14756 consensus_queue.cc:260] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.381912 14756 ts_tablet_manager.cc:1434] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:17:38.383107 14756 tablet_bootstrap.cc:492] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff: Bootstrap starting.
I20260709 13:17:38.391063 14756 tablet_bootstrap.cc:654] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.389817 14758 tablet_bootstrap.cc:492] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225: No bootstrap required, opened a new log
I20260709 13:17:38.400173 14758 ts_tablet_manager.cc:1403] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225: Time spent bootstrapping tablet: real 0.052s	user 0.017s	sys 0.004s
I20260709 13:17:38.413447 14757 tablet_bootstrap.cc:492] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3: No bootstrap required, opened a new log
I20260709 13:17:38.413990 14757 ts_tablet_manager.cc:1403] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3: Time spent bootstrapping tablet: real 0.082s	user 0.039s	sys 0.022s
I20260709 13:17:38.416994 14757 raft_consensus.cc:359] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.419296 14758 raft_consensus.cc:359] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.420032 14758 raft_consensus.cc:385] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.417729 14757 raft_consensus.cc:385] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.420339 14758 raft_consensus.cc:740] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3cd29370cc54e1099c022714674d225, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.422863 14758 consensus_queue.cc:260] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.422279 14757 raft_consensus.cc:740] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.434191 14757 consensus_queue.cc:260] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.435245 14758 ts_tablet_manager.cc:1434] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.003s
I20260709 13:17:38.436851 14564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4c219a8c864700b9f22ff2a9f23e68" candidate_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" is_pre_election: true
I20260709 13:17:38.437052 14758 tablet_bootstrap.cc:492] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225: Bootstrap starting.
I20260709 13:17:38.439486 14756 tablet_bootstrap.cc:492] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff: No bootstrap required, opened a new log
W20260709 13:17:38.439399 14679 leader_election.cc:343] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:38.439996 14756 ts_tablet_manager.cc:1403] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff: Time spent bootstrapping tablet: real 0.057s	user 0.025s	sys 0.016s
I20260709 13:17:38.443025 14756 raft_consensus.cc:359] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } }
I20260709 13:17:38.443790 14756 raft_consensus.cc:385] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.444104 14756 raft_consensus.cc:740] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac3d4d81e96f412caaff09b3861ef2ff, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.438014 14757 ts_tablet_manager.cc:1434] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.001s
I20260709 13:17:38.444811 14756 consensus_queue.cc:260] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } }
I20260709 13:17:38.445473 14757 tablet_bootstrap.cc:492] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3: Bootstrap starting.
I20260709 13:17:38.447292 14756 ts_tablet_manager.cc:1434] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 13:17:38.448252 14756 tablet_bootstrap.cc:492] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff: Bootstrap starting.
I20260709 13:17:38.451926 14757 tablet_bootstrap.cc:654] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.454757 14756 tablet_bootstrap.cc:654] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.455034 14758 tablet_bootstrap.cc:654] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.465495 14758 tablet_bootstrap.cc:492] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225: No bootstrap required, opened a new log
I20260709 13:17:38.465636 14640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4c219a8c864700b9f22ff2a9f23e68" candidate_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3cd29370cc54e1099c022714674d225" is_pre_election: true
I20260709 13:17:38.466061 14758 ts_tablet_manager.cc:1403] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225: Time spent bootstrapping tablet: real 0.029s	user 0.006s	sys 0.007s
I20260709 13:17:38.467562 14757 tablet_bootstrap.cc:492] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3: No bootstrap required, opened a new log
W20260709 13:17:38.467528 14679 leader_election.cc:343] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:38.468148 14757 ts_tablet_manager.cc:1403] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.005s
I20260709 13:17:38.468096 14679 leader_election.cc:304] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac3d4d81e96f412caaff09b3861ef2ff; no voters: 8c779a0db43c47378f25bcb2bbeb54e3, d3cd29370cc54e1099c022714674d225
I20260709 13:17:38.469033 14764 raft_consensus.cc:2749] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:38.470150 14758 raft_consensus.cc:359] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } }
I20260709 13:17:38.470862 14758 raft_consensus.cc:385] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.471148 14758 raft_consensus.cc:740] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3cd29370cc54e1099c022714674d225, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.471242 14757 raft_consensus.cc:359] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.471957 14757 raft_consensus.cc:385] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.472234 14757 raft_consensus.cc:740] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.472970 14757 consensus_queue.cc:260] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.473721 14758 consensus_queue.cc:260] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } }
I20260709 13:17:38.475397 14757 ts_tablet_manager.cc:1434] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:17:38.476359 14757 tablet_bootstrap.cc:492] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3: Bootstrap starting.
I20260709 13:17:38.476482 14756 tablet_bootstrap.cc:492] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff: No bootstrap required, opened a new log
I20260709 13:17:38.476969 14756 ts_tablet_manager.cc:1403] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.004s
I20260709 13:17:38.479735 14756 raft_consensus.cc:359] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.480470 14756 raft_consensus.cc:385] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.480785 14756 raft_consensus.cc:740] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac3d4d81e96f412caaff09b3861ef2ff, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.483592 14757 tablet_bootstrap.cc:654] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.481644 14756 consensus_queue.cc:260] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.485939 14756 ts_tablet_manager.cc:1434] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:17:38.486963 14756 tablet_bootstrap.cc:492] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff: Bootstrap starting.
I20260709 13:17:38.483577 14758 ts_tablet_manager.cc:1434] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 13:17:38.492097 14758 tablet_bootstrap.cc:492] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225: Bootstrap starting.
I20260709 13:17:38.498373 14758 tablet_bootstrap.cc:654] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.500012 14757 tablet_bootstrap.cc:492] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3: No bootstrap required, opened a new log
I20260709 13:17:38.500512 14757 ts_tablet_manager.cc:1403] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.001s
I20260709 13:17:38.503662 14756 tablet_bootstrap.cc:654] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.503258 14757 raft_consensus.cc:359] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.504098 14757 raft_consensus.cc:385] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.504410 14757 raft_consensus.cc:740] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.505141 14757 consensus_queue.cc:260] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.510732 14757 ts_tablet_manager.cc:1434] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:17:38.511727 14757 tablet_bootstrap.cc:492] T c246b0d9436a4b30819c7ce5f07fbd77 P 8c779a0db43c47378f25bcb2bbeb54e3: Bootstrap starting.
I20260709 13:17:38.516340 14762 raft_consensus.cc:493] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.516841 14762 raft_consensus.cc:515] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.519616 14762 leader_election.cc:290] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:38.523311 14757 tablet_bootstrap.cc:654] T c246b0d9436a4b30819c7ce5f07fbd77 P 8c779a0db43c47378f25bcb2bbeb54e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.535575 14757 tablet_bootstrap.cc:492] T c246b0d9436a4b30819c7ce5f07fbd77 P 8c779a0db43c47378f25bcb2bbeb54e3: No bootstrap required, opened a new log
I20260709 13:17:38.536062 14757 ts_tablet_manager.cc:1403] T c246b0d9436a4b30819c7ce5f07fbd77 P 8c779a0db43c47378f25bcb2bbeb54e3: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.000s
I20260709 13:17:38.538965 14757 raft_consensus.cc:359] T c246b0d9436a4b30819c7ce5f07fbd77 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.539752 14757 raft_consensus.cc:385] T c246b0d9436a4b30819c7ce5f07fbd77 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.540069 14757 raft_consensus.cc:740] T c246b0d9436a4b30819c7ce5f07fbd77 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.540807 14757 consensus_queue.cc:260] T c246b0d9436a4b30819c7ce5f07fbd77 P 8c779a0db43c47378f25bcb2bbeb54e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.543493 14757 ts_tablet_manager.cc:1434] T c246b0d9436a4b30819c7ce5f07fbd77 P 8c779a0db43c47378f25bcb2bbeb54e3: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:17:38.544481 14757 tablet_bootstrap.cc:492] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3: Bootstrap starting.
I20260709 13:17:38.550979 14757 tablet_bootstrap.cc:654] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.586771 14757 tablet_bootstrap.cc:492] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3: No bootstrap required, opened a new log
I20260709 13:17:38.587107 14758 tablet_bootstrap.cc:492] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225: No bootstrap required, opened a new log
I20260709 13:17:38.587280 14757 ts_tablet_manager.cc:1403] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3: Time spent bootstrapping tablet: real 0.043s	user 0.025s	sys 0.008s
I20260709 13:17:38.587631 14758 ts_tablet_manager.cc:1403] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225: Time spent bootstrapping tablet: real 0.096s	user 0.028s	sys 0.021s
I20260709 13:17:38.590276 14757 raft_consensus.cc:359] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.590469 14758 raft_consensus.cc:359] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.591061 14757 raft_consensus.cc:385] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.591195 14758 raft_consensus.cc:385] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.591427 14757 raft_consensus.cc:740] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.592274 14757 consensus_queue.cc:260] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.593878 14756 tablet_bootstrap.cc:492] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff: No bootstrap required, opened a new log
I20260709 13:17:38.594376 14756 ts_tablet_manager.cc:1403] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff: Time spent bootstrapping tablet: real 0.108s	user 0.040s	sys 0.012s
I20260709 13:17:38.594884 14757 ts_tablet_manager.cc:1434] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:17:38.595888 14757 tablet_bootstrap.cc:492] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3: Bootstrap starting.
I20260709 13:17:38.596108 14758 raft_consensus.cc:740] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3cd29370cc54e1099c022714674d225, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.596897 14758 consensus_queue.cc:260] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.599361 14758 ts_tablet_manager.cc:1434] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:17:38.600315 14758 tablet_bootstrap.cc:492] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225: Bootstrap starting.
I20260709 13:17:38.603173 14757 tablet_bootstrap.cc:654] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.606642 14758 tablet_bootstrap.cc:654] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.611306 14756 raft_consensus.cc:359] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.612169 14756 raft_consensus.cc:385] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.612533 14756 raft_consensus.cc:740] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac3d4d81e96f412caaff09b3861ef2ff, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.613415 14756 consensus_queue.cc:260] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.616897 14756 ts_tablet_manager.cc:1434] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.003s
I20260709 13:17:38.618163 14756 tablet_bootstrap.cc:492] T 9bfd1e022dbc48a7805a15dbf9324642 P ac3d4d81e96f412caaff09b3861ef2ff: Bootstrap starting.
I20260709 13:17:38.619549 14640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42fa1c328df4d64818288ee87e30a46" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3cd29370cc54e1099c022714674d225" is_pre_election: true
I20260709 13:17:38.624935 14714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42fa1c328df4d64818288ee87e30a46" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" is_pre_election: true
W20260709 13:17:38.625483 14529 leader_election.cc:343] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:38.625833 14714 raft_consensus.cc:2468] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 0.
I20260709 13:17:38.628007 14529 leader_election.cc:304] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, ac3d4d81e96f412caaff09b3861ef2ff; no voters: d3cd29370cc54e1099c022714674d225
I20260709 13:17:38.629668 14762 raft_consensus.cc:2804] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:38.630316 14762 raft_consensus.cc:493] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.631142 14762 raft_consensus.cc:3060] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.634092 14756 tablet_bootstrap.cc:654] T 9bfd1e022dbc48a7805a15dbf9324642 P ac3d4d81e96f412caaff09b3861ef2ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.638140 14757 tablet_bootstrap.cc:492] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3: No bootstrap required, opened a new log
I20260709 13:17:38.638671 14757 ts_tablet_manager.cc:1403] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3: Time spent bootstrapping tablet: real 0.043s	user 0.018s	sys 0.016s
I20260709 13:17:38.639382 14758 tablet_bootstrap.cc:492] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225: No bootstrap required, opened a new log
I20260709 13:17:38.639766 14758 ts_tablet_manager.cc:1403] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225: Time spent bootstrapping tablet: real 0.040s	user 0.014s	sys 0.001s
I20260709 13:17:38.641716 14757 raft_consensus.cc:359] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.643005 14757 raft_consensus.cc:385] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.643254 14757 raft_consensus.cc:740] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.643366 14762 raft_consensus.cc:515] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.643954 14757 consensus_queue.cc:260] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.646629 14640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42fa1c328df4d64818288ee87e30a46" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3cd29370cc54e1099c022714674d225"
W20260709 13:17:38.648378 14529 leader_election.cc:343] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:38.649024 14764 raft_consensus.cc:493] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.650887 14785 raft_consensus.cc:493] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.651582 14764 raft_consensus.cc:515] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } }
I20260709 13:17:38.649211 14758 raft_consensus.cc:359] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.653183 14714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42fa1c328df4d64818288ee87e30a46" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
I20260709 13:17:38.651949 14762 leader_election.cc:290] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 election: Requested vote from peers d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:38.654677 14714 raft_consensus.cc:3060] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.652571 14758 raft_consensus.cc:385] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.651865 14785 raft_consensus.cc:515] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.655242 14640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b395b43c234641fc985583e8ec908250" candidate_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3cd29370cc54e1099c022714674d225" is_pre_election: true
I20260709 13:17:38.656303 14764 leader_election.cc:290] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643), 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713)
I20260709 13:17:38.656502 14640 raft_consensus.cc:2468] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac3d4d81e96f412caaff09b3861ef2ff in term 0.
I20260709 13:17:38.655233 14758 raft_consensus.cc:740] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3cd29370cc54e1099c022714674d225, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.658087 14679 leader_election.cc:304] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac3d4d81e96f412caaff09b3861ef2ff, d3cd29370cc54e1099c022714674d225; no voters: 
I20260709 13:17:38.662045 14564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b395b43c234641fc985583e8ec908250" candidate_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" is_pre_election: true
I20260709 13:17:38.664536 14757 ts_tablet_manager.cc:1434] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3: Time spent starting tablet: real 0.026s	user 0.016s	sys 0.003s
I20260709 13:17:38.663592 14764 raft_consensus.cc:2804] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:38.668262 14564 raft_consensus.cc:2468] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac3d4d81e96f412caaff09b3861ef2ff in term 0.
I20260709 13:17:38.668591 14764 raft_consensus.cc:493] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.669504 14785 leader_election.cc:290] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:38.670280 14764 raft_consensus.cc:3060] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.671051 14756 tablet_bootstrap.cc:492] T 9bfd1e022dbc48a7805a15dbf9324642 P ac3d4d81e96f412caaff09b3861ef2ff: No bootstrap required, opened a new log
I20260709 13:17:38.662523 14758 consensus_queue.cc:260] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.672413 14640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd1e022dbc48a7805a15dbf9324642" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3cd29370cc54e1099c022714674d225" is_pre_election: true
I20260709 13:17:38.672313 14713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd1e022dbc48a7805a15dbf9324642" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" is_pre_election: true
I20260709 13:17:38.673153 14640 raft_consensus.cc:2468] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 0.
W20260709 13:17:38.674083 14529 leader_election.cc:343] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:38.674726 14758 ts_tablet_manager.cc:1434] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225: Time spent starting tablet: real 0.035s	user 0.002s	sys 0.007s
I20260709 13:17:38.674993 14756 ts_tablet_manager.cc:1403] T 9bfd1e022dbc48a7805a15dbf9324642 P ac3d4d81e96f412caaff09b3861ef2ff: Time spent bootstrapping tablet: real 0.055s	user 0.010s	sys 0.010s
I20260709 13:17:38.674853 14529 leader_election.cc:304] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, d3cd29370cc54e1099c022714674d225; no voters: ac3d4d81e96f412caaff09b3861ef2ff
I20260709 13:17:38.676415 14714 raft_consensus.cc:2468] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 1.
I20260709 13:17:38.677778 14529 leader_election.cc:304] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, ac3d4d81e96f412caaff09b3861ef2ff; no voters: d3cd29370cc54e1099c022714674d225
I20260709 13:17:38.678208 14757 tablet_bootstrap.cc:492] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3: Bootstrap starting.
I20260709 13:17:38.679420 14785 raft_consensus.cc:2804] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:38.681941 14764 raft_consensus.cc:515] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } }
I20260709 13:17:38.683929 14764 leader_election.cc:290] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 1 election: Requested vote from peers d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643), 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713)
I20260709 13:17:38.680421 14758 tablet_bootstrap.cc:492] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225: Bootstrap starting.
I20260709 13:17:38.681493 14756 raft_consensus.cc:359] T 9bfd1e022dbc48a7805a15dbf9324642 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.685014 14756 raft_consensus.cc:385] T 9bfd1e022dbc48a7805a15dbf9324642 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.679543 14786 raft_consensus.cc:2804] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:38.682252 14785 raft_consensus.cc:493] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.685416 14756 raft_consensus.cc:740] T 9bfd1e022dbc48a7805a15dbf9324642 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac3d4d81e96f412caaff09b3861ef2ff, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.687011 14756 consensus_queue.cc:260] T 9bfd1e022dbc48a7805a15dbf9324642 P ac3d4d81e96f412caaff09b3861ef2ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.690037 14757 tablet_bootstrap.cc:654] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.689829 14640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b395b43c234641fc985583e8ec908250" candidate_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3cd29370cc54e1099c022714674d225"
I20260709 13:17:38.685712 14785 raft_consensus.cc:3060] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.694348 14640 raft_consensus.cc:3060] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.695353 14756 ts_tablet_manager.cc:1434] T 9bfd1e022dbc48a7805a15dbf9324642 P ac3d4d81e96f412caaff09b3861ef2ff: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:17:38.685741 14786 raft_consensus.cc:697] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Becoming Leader. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Running, Role: LEADER
I20260709 13:17:38.696307 14756 tablet_bootstrap.cc:492] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff: Bootstrap starting.
I20260709 13:17:38.697026 14786 consensus_queue.cc:237] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.701721 14756 tablet_bootstrap.cc:654] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.703012 14785 raft_consensus.cc:515] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.706168 14639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd1e022dbc48a7805a15dbf9324642" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3cd29370cc54e1099c022714674d225"
I20260709 13:17:38.706914 14714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd1e022dbc48a7805a15dbf9324642" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
I20260709 13:17:38.707551 14714 raft_consensus.cc:3060] T 9bfd1e022dbc48a7805a15dbf9324642 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.706938 14639 raft_consensus.cc:3060] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.692981 14564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b395b43c234641fc985583e8ec908250" candidate_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
I20260709 13:17:38.691236 14758 tablet_bootstrap.cc:654] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.713940 14640 raft_consensus.cc:2468] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac3d4d81e96f412caaff09b3861ef2ff in term 1.
I20260709 13:17:38.716449 14679 leader_election.cc:304] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac3d4d81e96f412caaff09b3861ef2ff, d3cd29370cc54e1099c022714674d225; no voters: 
I20260709 13:17:38.717350 14764 raft_consensus.cc:2804] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:38.718235 14564 raft_consensus.cc:3060] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.718968 14756 tablet_bootstrap.cc:492] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff: No bootstrap required, opened a new log
I20260709 13:17:38.719465 14756 ts_tablet_manager.cc:1403] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.000s
I20260709 13:17:38.720012 14714 raft_consensus.cc:2468] T 9bfd1e022dbc48a7805a15dbf9324642 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 1.
I20260709 13:17:38.721311 14529 leader_election.cc:304] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, ac3d4d81e96f412caaff09b3861ef2ff; no voters: 
I20260709 13:17:38.722141 14786 raft_consensus.cc:2804] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:38.722481 14756 raft_consensus.cc:359] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.722780 14786 raft_consensus.cc:697] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Becoming Leader. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Running, Role: LEADER
I20260709 13:17:38.724082 14763 raft_consensus.cc:493] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.724596 14763 raft_consensus.cc:515] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.726409 14564 raft_consensus.cc:2468] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac3d4d81e96f412caaff09b3861ef2ff in term 1.
I20260709 13:17:38.723325 14785 leader_election.cc:290] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 election: Requested vote from peers d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:38.727622 14763 leader_election.cc:290] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:38.723670 14786 consensus_queue.cc:237] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.728916 14457 catalog_manager.cc:5697] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.738049 14756 raft_consensus.cc:385] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.738488 14756 raft_consensus.cc:740] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac3d4d81e96f412caaff09b3861ef2ff, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.739311 14756 consensus_queue.cc:260] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.741278 14639 raft_consensus.cc:2468] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 1.
I20260709 13:17:38.741464 14758 tablet_bootstrap.cc:492] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225: No bootstrap required, opened a new log
I20260709 13:17:38.742179 14758 ts_tablet_manager.cc:1403] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225: Time spent bootstrapping tablet: real 0.062s	user 0.011s	sys 0.002s
I20260709 13:17:38.743894 14757 tablet_bootstrap.cc:492] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3: No bootstrap required, opened a new log
I20260709 13:17:38.744418 14757 ts_tablet_manager.cc:1403] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3: Time spent bootstrapping tablet: real 0.073s	user 0.018s	sys 0.008s
I20260709 13:17:38.745376 14758 raft_consensus.cc:359] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.746114 14756 ts_tablet_manager.cc:1434] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff: Time spent starting tablet: real 0.026s	user 0.010s	sys 0.000s
I20260709 13:17:38.746203 14758 raft_consensus.cc:385] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.747108 14758 raft_consensus.cc:740] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3cd29370cc54e1099c022714674d225, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.747301 14757 raft_consensus.cc:359] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.748143 14757 raft_consensus.cc:385] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.748005 14758 consensus_queue.cc:260] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.748474 14757 raft_consensus.cc:740] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.749265 14757 consensus_queue.cc:260] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.751377 14757 ts_tablet_manager.cc:1434] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:17:38.752449 14757 tablet_bootstrap.cc:492] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3: Bootstrap starting.
I20260709 13:17:38.747141 14756 tablet_bootstrap.cc:492] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff: Bootstrap starting.
I20260709 13:17:38.755293 14763 raft_consensus.cc:493] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.755769 14763 raft_consensus.cc:515] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.757757 14763 leader_election.cc:290] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:38.758729 14757 tablet_bootstrap.cc:654] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.762156 14758 ts_tablet_manager.cc:1434] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:17:38.763319 14758 tablet_bootstrap.cc:492] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225: Bootstrap starting.
I20260709 13:17:38.770534 14757 tablet_bootstrap.cc:492] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3: No bootstrap required, opened a new log
I20260709 13:17:38.771075 14757 ts_tablet_manager.cc:1403] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.007s
I20260709 13:17:38.772066 14756 tablet_bootstrap.cc:654] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.773770 14757 raft_consensus.cc:359] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.774616 14757 raft_consensus.cc:385] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.774946 14757 raft_consensus.cc:740] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.775520 14758 tablet_bootstrap.cc:654] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.775682 14757 consensus_queue.cc:260] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.777814 14757 ts_tablet_manager.cc:1434] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:17:38.780581 14764 raft_consensus.cc:697] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 LEADER]: Becoming Leader. State: Replica: ac3d4d81e96f412caaff09b3861ef2ff, State: Running, Role: LEADER
I20260709 13:17:38.781544 14764 consensus_queue.cc:237] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } }
I20260709 13:17:38.782469 14786 raft_consensus.cc:493] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.782980 14786 raft_consensus.cc:515] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.785115 14786 leader_election.cc:290] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:38.786441 14758 tablet_bootstrap.cc:492] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225: No bootstrap required, opened a new log
I20260709 13:17:38.786803 14714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4c219a8c864700b9f22ff2a9f23e68" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" is_pre_election: true
I20260709 13:17:38.787081 14758 ts_tablet_manager.cc:1403] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.003s
I20260709 13:17:38.787556 14714 raft_consensus.cc:2468] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 0.
I20260709 13:17:38.789049 14529 leader_election.cc:304] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, ac3d4d81e96f412caaff09b3861ef2ff; no voters: 
I20260709 13:17:38.789881 14786 raft_consensus.cc:2804] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:38.790289 14786 raft_consensus.cc:493] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.790341 14639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4c219a8c864700b9f22ff2a9f23e68" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3cd29370cc54e1099c022714674d225" is_pre_election: true
I20260709 13:17:38.790835 14786 raft_consensus.cc:3060] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.790373 14758 raft_consensus.cc:359] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.791368 14758 raft_consensus.cc:385] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.791337 14639 raft_consensus.cc:2468] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 0.
I20260709 13:17:38.791857 14758 raft_consensus.cc:740] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3cd29370cc54e1099c022714674d225, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.792754 14758 consensus_queue.cc:260] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.795382 14758 ts_tablet_manager.cc:1434] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 13:17:38.796530 14758 tablet_bootstrap.cc:492] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225: Bootstrap starting.
I20260709 13:17:38.800393 14785 raft_consensus.cc:493] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.800961 14785 raft_consensus.cc:515] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.803185 14785 leader_election.cc:290] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:38.803474 14758 tablet_bootstrap.cc:654] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.805760 14714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a36368cb545c46bd929582b28a3fece5" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" is_pre_election: true
I20260709 13:17:38.811460 14714 raft_consensus.cc:2468] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 0.
I20260709 13:17:38.813850 14529 leader_election.cc:304] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, ac3d4d81e96f412caaff09b3861ef2ff; no voters: 
I20260709 13:17:38.814121 14786 raft_consensus.cc:515] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.814767 14785 raft_consensus.cc:2804] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:38.815142 14785 raft_consensus.cc:493] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.815479 14785 raft_consensus.cc:3060] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.816217 14786 leader_election.cc:290] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 election: Requested vote from peers d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:38.823493 14640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4c219a8c864700b9f22ff2a9f23e68" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3cd29370cc54e1099c022714674d225"
I20260709 13:17:38.830768 14640 raft_consensus.cc:3060] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.823239 14714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4c219a8c864700b9f22ff2a9f23e68" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
I20260709 13:17:38.834261 14714 raft_consensus.cc:3060] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.822556 14639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a36368cb545c46bd929582b28a3fece5" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3cd29370cc54e1099c022714674d225" is_pre_election: true
I20260709 13:17:38.836073 14756 tablet_bootstrap.cc:492] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff: No bootstrap required, opened a new log
I20260709 13:17:38.836861 14756 ts_tablet_manager.cc:1403] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff: Time spent bootstrapping tablet: real 0.090s	user 0.035s	sys 0.011s
I20260709 13:17:38.837025 14639 raft_consensus.cc:2468] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 0.
I20260709 13:17:38.854441 14785 raft_consensus.cc:515] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.856902 14785 leader_election.cc:290] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 election: Requested vote from peers d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:38.858503 14713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a36368cb545c46bd929582b28a3fece5" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
I20260709 13:17:38.841015 14640 raft_consensus.cc:2468] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 1.
I20260709 13:17:38.859228 14713 raft_consensus.cc:3060] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.851295 14714 raft_consensus.cc:2468] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 1.
I20260709 13:17:38.865314 14564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d1ea631a1bb4cf8a4f25e155142c0d3" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" is_pre_election: true
I20260709 13:17:38.864481 14639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a36368cb545c46bd929582b28a3fece5" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3cd29370cc54e1099c022714674d225"
I20260709 13:17:38.866993 14639 raft_consensus.cc:3060] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.867242 14529 leader_election.cc:304] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, d3cd29370cc54e1099c022714674d225; no voters: 
I20260709 13:17:38.866020 14564 raft_consensus.cc:2468] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3cd29370cc54e1099c022714674d225 in term 0.
I20260709 13:17:38.869380 14563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c246b0d9436a4b30819c7ce5f07fbd77" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" is_pre_election: true
I20260709 13:17:38.870930 14563 raft_consensus.cc:2468] T c246b0d9436a4b30819c7ce5f07fbd77 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3cd29370cc54e1099c022714674d225 in term 0.
I20260709 13:17:38.871416 14756 raft_consensus.cc:359] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.872190 14604 leader_election.cc:304] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, d3cd29370cc54e1099c022714674d225; no voters: 
I20260709 13:17:38.873207 14604 leader_election.cc:304] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, d3cd29370cc54e1099c022714674d225; no voters: 
I20260709 13:17:38.873804 14758 tablet_bootstrap.cc:492] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225: No bootstrap required, opened a new log
I20260709 13:17:38.874267 14758 ts_tablet_manager.cc:1403] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225: Time spent bootstrapping tablet: real 0.078s	user 0.044s	sys 0.013s
I20260709 13:17:38.874461 14763 raft_consensus.cc:2804] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:38.872215 14756 raft_consensus.cc:385] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.874840 14763 raft_consensus.cc:493] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.875291 14756 raft_consensus.cc:740] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac3d4d81e96f412caaff09b3861ef2ff, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.875559 14763 raft_consensus.cc:3060] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.876072 14756 consensus_queue.cc:260] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.870066 14785 raft_consensus.cc:2804] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:38.884084 14790 raft_consensus.cc:493] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.881009 14785 raft_consensus.cc:697] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Becoming Leader. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Running, Role: LEADER
I20260709 13:17:38.879521 14460 catalog_manager.cc:5697] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff reported cstate change: term changed from 0 to 1, leader changed from <none> to ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67). New cstate: current_term: 1 leader_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.884884 14790 raft_consensus.cc:515] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.886188 14785 consensus_queue.cc:237] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.887679 14756 ts_tablet_manager.cc:1434] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.001s
I20260709 13:17:38.893237 14790 leader_election.cc:290] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713), d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643)
I20260709 13:17:38.894006 14639 raft_consensus.cc:2468] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 1.
I20260709 13:17:38.901070 14756 tablet_bootstrap.cc:492] T 2c99ec13aa4c4ff5af744e67c5164098 P ac3d4d81e96f412caaff09b3861ef2ff: Bootstrap starting.
I20260709 13:17:38.903188 14529 leader_election.cc:304] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, d3cd29370cc54e1099c022714674d225; no voters: 
I20260709 13:17:38.904145 14786 raft_consensus.cc:2804] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:38.904695 14786 raft_consensus.cc:697] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Becoming Leader. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Running, Role: LEADER
I20260709 13:17:38.906694 14563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d1ea631a1bb4cf8a4f25e155142c0d3" candidate_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" is_pre_election: true
I20260709 13:17:38.907342 14563 raft_consensus.cc:2468] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac3d4d81e96f412caaff09b3861ef2ff in term 0.
I20260709 13:17:38.907914 14763 raft_consensus.cc:515] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.909070 14679 leader_election.cc:304] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, ac3d4d81e96f412caaff09b3861ef2ff; no voters: 
I20260709 13:17:38.909890 14790 raft_consensus.cc:2804] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:38.910269 14790 raft_consensus.cc:493] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.910594 14790 raft_consensus.cc:3060] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.914204 14713 raft_consensus.cc:2468] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 1.
I20260709 13:17:38.924558 14758 raft_consensus.cc:359] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.929160 14758 raft_consensus.cc:385] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.929473 14758 raft_consensus.cc:740] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3cd29370cc54e1099c022714674d225, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.930200 14758 consensus_queue.cc:260] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.912321 14763 leader_election.cc:290] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 1 election: Requested vote from peers 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:38.931880 14714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d1ea631a1bb4cf8a4f25e155142c0d3" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" is_pre_election: true
I20260709 13:17:38.929695 14639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d1ea631a1bb4cf8a4f25e155142c0d3" candidate_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3cd29370cc54e1099c022714674d225" is_pre_election: true
I20260709 13:17:38.923723 14790 raft_consensus.cc:515] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.915114 14563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d1ea631a1bb4cf8a4f25e155142c0d3" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
I20260709 13:17:38.936816 14639 raft_consensus.cc:2393] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ac3d4d81e96f412caaff09b3861ef2ff in current term 1: Already voted for candidate d3cd29370cc54e1099c022714674d225 in this term.
I20260709 13:17:38.937757 14711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d1ea631a1bb4cf8a4f25e155142c0d3" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
I20260709 13:17:38.934255 14763 raft_consensus.cc:2804] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:38.924351 14457 catalog_manager.cc:5697] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.934614 14712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c246b0d9436a4b30819c7ce5f07fbd77" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" is_pre_election: true
I20260709 13:17:38.943439 14758 ts_tablet_manager.cc:1434] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225: Time spent starting tablet: real 0.069s	user 0.009s	sys 0.007s
I20260709 13:17:38.938606 14563 raft_consensus.cc:3060] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.942031 14712 raft_consensus.cc:2468] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3cd29370cc54e1099c022714674d225 in term 0.
I20260709 13:17:38.944473 14714 raft_consensus.cc:2393] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3cd29370cc54e1099c022714674d225 in current term 1: Already voted for candidate ac3d4d81e96f412caaff09b3861ef2ff in this term.
I20260709 13:17:38.941025 14763 raft_consensus.cc:493] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.946157 14763 raft_consensus.cc:3060] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.942278 14790 leader_election.cc:290] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 1 election: Requested vote from peers 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713), d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643)
I20260709 13:17:38.965106 14564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d1ea631a1bb4cf8a4f25e155142c0d3" candidate_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
I20260709 13:17:38.942416 14786 consensus_queue.cc:237] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.966589 14785 raft_consensus.cc:493] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.969430 14785 raft_consensus.cc:515] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:38.976527 14639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d1ea631a1bb4cf8a4f25e155142c0d3" candidate_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3cd29370cc54e1099c022714674d225"
I20260709 13:17:38.978552 14639 raft_consensus.cc:2393] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ac3d4d81e96f412caaff09b3861ef2ff in current term 1: Already voted for candidate d3cd29370cc54e1099c022714674d225 in this term.
I20260709 13:17:38.979306 14785 leader_election.cc:290] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:38.980615 14758 tablet_bootstrap.cc:492] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225: Bootstrap starting.
I20260709 13:17:38.981822 14756 tablet_bootstrap.cc:654] T 2c99ec13aa4c4ff5af744e67c5164098 P ac3d4d81e96f412caaff09b3861ef2ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.981765 14714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c99ec13aa4c4ff5af744e67c5164098" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" is_pre_election: true
I20260709 13:17:38.986902 14639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c99ec13aa4c4ff5af744e67c5164098" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3cd29370cc54e1099c022714674d225" is_pre_election: true
I20260709 13:17:38.987535 14639 raft_consensus.cc:2468] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 0.
W20260709 13:17:38.988037 14529 leader_election.cc:343] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:38.988896 14529 leader_election.cc:304] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, d3cd29370cc54e1099c022714674d225; no voters: ac3d4d81e96f412caaff09b3861ef2ff
I20260709 13:17:38.990392 14785 raft_consensus.cc:2804] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:38.992018 14785 raft_consensus.cc:493] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.992447 14785 raft_consensus.cc:3060] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.994014 14679 leader_election.cc:304] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac3d4d81e96f412caaff09b3861ef2ff; no voters: 8c779a0db43c47378f25bcb2bbeb54e3, d3cd29370cc54e1099c022714674d225
I20260709 13:17:38.994932 14790 raft_consensus.cc:2749] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:17:38.995285 14758 tablet_bootstrap.cc:654] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:39.006317 14763 raft_consensus.cc:515] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.009641 14804 raft_consensus.cc:493] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.010107 14763 leader_election.cc:290] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 1 election: Requested vote from peers 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:39.010212 14804 raft_consensus.cc:515] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.012374 14804 leader_election.cc:290] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:39.015481 14712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42fa1c328df4d64818288ee87e30a46" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" is_pre_election: true
I20260709 13:17:39.017136 14712 raft_consensus.cc:2393] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3cd29370cc54e1099c022714674d225 in current term 1: Already voted for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in this term.
I20260709 13:17:39.016796 14714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c246b0d9436a4b30819c7ce5f07fbd77" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
I20260709 13:17:39.017298 14564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c246b0d9436a4b30819c7ce5f07fbd77" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
I20260709 13:17:39.017586 14714 raft_consensus.cc:3060] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:39.018054 14564 raft_consensus.cc:3060] T c246b0d9436a4b30819c7ce5f07fbd77 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:39.019526 14785 raft_consensus.cc:515] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.021664 14785 leader_election.cc:290] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 election: Requested vote from peers d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:39.018080 14562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42fa1c328df4d64818288ee87e30a46" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" is_pre_election: true
I20260709 13:17:39.023880 14639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c99ec13aa4c4ff5af744e67c5164098" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3cd29370cc54e1099c022714674d225"
I20260709 13:17:39.024459 14639 raft_consensus.cc:3060] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:39.024612 14604 leader_election.cc:304] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3cd29370cc54e1099c022714674d225; no voters: 8c779a0db43c47378f25bcb2bbeb54e3, ac3d4d81e96f412caaff09b3861ef2ff
I20260709 13:17:39.025533 14712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c99ec13aa4c4ff5af744e67c5164098" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
I20260709 13:17:39.025862 14714 raft_consensus.cc:2468] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3cd29370cc54e1099c022714674d225 in term 1.
I20260709 13:17:39.026026 14804 raft_consensus.cc:3060] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:39.021108 14563 raft_consensus.cc:2468] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3cd29370cc54e1099c022714674d225 in term 1.
W20260709 13:17:39.028528 14529 leader_election.cc:343] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:39.027823 14604 leader_election.cc:304] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac3d4d81e96f412caaff09b3861ef2ff, d3cd29370cc54e1099c022714674d225; no voters: 
I20260709 13:17:39.032382 14604 leader_election.cc:304] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, d3cd29370cc54e1099c022714674d225; no voters: ac3d4d81e96f412caaff09b3861ef2ff
I20260709 13:17:39.033013 14756 tablet_bootstrap.cc:492] T 2c99ec13aa4c4ff5af744e67c5164098 P ac3d4d81e96f412caaff09b3861ef2ff: No bootstrap required, opened a new log
I20260709 13:17:39.033313 14758 tablet_bootstrap.cc:492] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225: No bootstrap required, opened a new log
I20260709 13:17:39.033574 14806 raft_consensus.cc:2804] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:39.033749 14758 ts_tablet_manager.cc:1403] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225: Time spent bootstrapping tablet: real 0.053s	user 0.008s	sys 0.012s
I20260709 13:17:39.033557 14756 ts_tablet_manager.cc:1403] T 2c99ec13aa4c4ff5af744e67c5164098 P ac3d4d81e96f412caaff09b3861ef2ff: Time spent bootstrapping tablet: real 0.141s	user 0.017s	sys 0.001s
I20260709 13:17:39.038678 14564 raft_consensus.cc:2468] T c246b0d9436a4b30819c7ce5f07fbd77 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3cd29370cc54e1099c022714674d225 in term 1.
I20260709 13:17:39.034300 14763 raft_consensus.cc:2804] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:39.041210 14756 raft_consensus.cc:359] T 2c99ec13aa4c4ff5af744e67c5164098 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.041988 14756 raft_consensus.cc:385] T 2c99ec13aa4c4ff5af744e67c5164098 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:39.041141 14639 raft_consensus.cc:2468] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 1.
I20260709 13:17:39.042917 14806 raft_consensus.cc:697] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 1 LEADER]: Becoming Leader. State: Replica: d3cd29370cc54e1099c022714674d225, State: Running, Role: LEADER
I20260709 13:17:39.043032 14460 catalog_manager.cc:5697] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:39.044065 14806 consensus_queue.cc:237] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.046164 14804 raft_consensus.cc:2749] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:39.045559 14460 catalog_manager.cc:5697] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:39.042351 14756 raft_consensus.cc:740] T 2c99ec13aa4c4ff5af744e67c5164098 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac3d4d81e96f412caaff09b3861ef2ff, State: Initialized, Role: FOLLOWER
I20260709 13:17:39.049851 14756 consensus_queue.cc:260] T 2c99ec13aa4c4ff5af744e67c5164098 P ac3d4d81e96f412caaff09b3861ef2ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.045471 14758 raft_consensus.cc:359] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.051784 14758 raft_consensus.cc:385] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:39.052130 14758 raft_consensus.cc:740] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3cd29370cc54e1099c022714674d225, State: Initialized, Role: FOLLOWER
I20260709 13:17:39.056173 14756 ts_tablet_manager.cc:1434] T 2c99ec13aa4c4ff5af744e67c5164098 P ac3d4d81e96f412caaff09b3861ef2ff: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:17:39.057519 14529 leader_election.cc:304] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, d3cd29370cc54e1099c022714674d225; no voters: ac3d4d81e96f412caaff09b3861ef2ff
I20260709 13:17:39.060667 14758 consensus_queue.cc:260] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.055852 14763 raft_consensus.cc:697] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [term 1 LEADER]: Becoming Leader. State: Replica: d3cd29370cc54e1099c022714674d225, State: Running, Role: LEADER
I20260709 13:17:39.061558 14785 raft_consensus.cc:2804] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:39.064370 14785 raft_consensus.cc:697] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Becoming Leader. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Running, Role: LEADER
I20260709 13:17:39.063369 14763 consensus_queue.cc:237] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.063632 14758 ts_tablet_manager.cc:1434] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 13:17:39.065214 14785 consensus_queue.cc:237] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.078708 14459 catalog_manager.cc:5697] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3cd29370cc54e1099c022714674d225 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "d3cd29370cc54e1099c022714674d225" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:39.079708 14458 catalog_manager.cc:5697] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:39.080284 14459 catalog_manager.cc:5697] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3cd29370cc54e1099c022714674d225 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "d3cd29370cc54e1099c022714674d225" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:39.087628 14785 raft_consensus.cc:493] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.088161 14785 raft_consensus.cc:515] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.091356 14639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accbd816047c4054a6f506623e82f905" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3cd29370cc54e1099c022714674d225" is_pre_election: true
I20260709 13:17:39.092053 14639 raft_consensus.cc:2468] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 0.
I20260709 13:17:39.093214 14529 leader_election.cc:304] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, d3cd29370cc54e1099c022714674d225; no voters: 
I20260709 13:17:39.094040 14786 raft_consensus.cc:2804] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:39.094408 14786 raft_consensus.cc:493] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:39.094726 14786 raft_consensus.cc:3060] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:39.096757 14785 leader_election.cc:290] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:39.097978 14714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accbd816047c4054a6f506623e82f905" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" is_pre_election: true
I20260709 13:17:39.098656 14714 raft_consensus.cc:2468] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 0.
I20260709 13:17:39.101471 14786 raft_consensus.cc:515] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.103612 14786 leader_election.cc:290] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 election: Requested vote from peers d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:39.104306 14639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accbd816047c4054a6f506623e82f905" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3cd29370cc54e1099c022714674d225"
I20260709 13:17:39.104544 14714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accbd816047c4054a6f506623e82f905" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
I20260709 13:17:39.104909 14639 raft_consensus.cc:3060] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:39.105048 14714 raft_consensus.cc:3060] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:39.110231 14714 raft_consensus.cc:2468] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 1.
I20260709 13:17:39.111447 14529 leader_election.cc:304] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, ac3d4d81e96f412caaff09b3861ef2ff; no voters: 
I20260709 13:17:39.111474 14639 raft_consensus.cc:2468] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 1.
I20260709 13:17:39.112677 14786 raft_consensus.cc:2804] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:39.113173 14786 raft_consensus.cc:697] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Becoming Leader. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Running, Role: LEADER
I20260709 13:17:39.113740 14762 consensus_queue.cc:1048] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:39.115382 14786 consensus_queue.cc:237] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.132266 14785 consensus_queue.cc:1048] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:39.132875 14459 catalog_manager.cc:5697] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:39.177683 14785 consensus_queue.cc:1048] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:39.183733 14790 raft_consensus.cc:493] T 2c99ec13aa4c4ff5af744e67c5164098 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.184176 14790 raft_consensus.cc:515] T 2c99ec13aa4c4ff5af744e67c5164098 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.186095 14790 leader_election.cc:290] T 2c99ec13aa4c4ff5af744e67c5164098 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713), d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643)
I20260709 13:17:39.186720 14564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c99ec13aa4c4ff5af744e67c5164098" candidate_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" is_pre_election: true
I20260709 13:17:39.187716 14639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c99ec13aa4c4ff5af744e67c5164098" candidate_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3cd29370cc54e1099c022714674d225" is_pre_election: true
I20260709 13:17:39.188599 14639 raft_consensus.cc:2393] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ac3d4d81e96f412caaff09b3861ef2ff in current term 1: Already voted for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in this term.
I20260709 13:17:39.189584 14679 leader_election.cc:304] T 2c99ec13aa4c4ff5af744e67c5164098 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac3d4d81e96f412caaff09b3861ef2ff; no voters: 8c779a0db43c47378f25bcb2bbeb54e3, d3cd29370cc54e1099c022714674d225
I20260709 13:17:39.190625 14790 raft_consensus.cc:3060] T 2c99ec13aa4c4ff5af744e67c5164098 P ac3d4d81e96f412caaff09b3861ef2ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:39.193045 14785 consensus_queue.cc:1048] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:17:39.198414 14790 raft_consensus.cc:2749] T 2c99ec13aa4c4ff5af744e67c5164098 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:39.228830 14712 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b395b43c234641fc985583e8ec908250"
dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
mode: GRACEFUL
new_leader_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
 from {username='slave'} at 127.0.0.1:60020
I20260709 13:17:39.229446 14712 raft_consensus.cc:606] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 LEADER]: Received request to transfer leadership to TS 8c779a0db43c47378f25bcb2bbeb54e3
I20260709 13:17:39.253052 14564 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "accbd816047c4054a6f506623e82f905"
dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
mode: GRACEFUL
new_leader_uuid: "d3cd29370cc54e1099c022714674d225"
 from {username='slave'} at 127.0.0.1:45440
I20260709 13:17:39.253589 14564 raft_consensus.cc:606] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Received request to transfer leadership to TS d3cd29370cc54e1099c022714674d225
I20260709 13:17:39.261358 14564 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bfd1e022dbc48a7805a15dbf9324642"
dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
mode: GRACEFUL
new_leader_uuid: "d3cd29370cc54e1099c022714674d225"
 from {username='slave'} at 127.0.0.1:45440
I20260709 13:17:39.261893 14564 raft_consensus.cc:606] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Received request to transfer leadership to TS d3cd29370cc54e1099c022714674d225
I20260709 13:17:39.266804 14564 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c99ec13aa4c4ff5af744e67c5164098"
dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
mode: GRACEFUL
new_leader_uuid: "d3cd29370cc54e1099c022714674d225"
 from {username='slave'} at 127.0.0.1:45440
I20260709 13:17:39.267289 14564 raft_consensus.cc:606] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Received request to transfer leadership to TS d3cd29370cc54e1099c022714674d225
I20260709 13:17:39.281389 14639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c246b0d9436a4b30819c7ce5f07fbd77"
dest_uuid: "d3cd29370cc54e1099c022714674d225"
mode: GRACEFUL
new_leader_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
 from {username='slave'} at 127.0.0.1:41756
I20260709 13:17:39.281949 14639 raft_consensus.cc:606] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [term 1 LEADER]: Received request to transfer leadership to TS ac3d4d81e96f412caaff09b3861ef2ff
I20260709 13:17:39.345834 14785 consensus_queue.cc:1048] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:39.359941 14786 consensus_queue.cc:1048] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:39.373090 14786 consensus_queue.cc:1048] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:39.385958 14785 consensus_queue.cc:1048] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:39.399823 14790 consensus_queue.cc:1048] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:39.411506 14790 consensus_queue.cc:1048] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:39.429250 14790 raft_consensus.cc:993] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff: : Instructing follower 8c779a0db43c47378f25bcb2bbeb54e3 to start an election
I20260709 13:17:39.429852 14790 raft_consensus.cc:1081] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 LEADER]: Signalling peer 8c779a0db43c47378f25bcb2bbeb54e3 to start an election
I20260709 13:17:39.431821 14564 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b395b43c234641fc985583e8ec908250"
dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
 from {username='slave'} at 127.0.0.1:45424
I20260709 13:17:39.432435 14564 raft_consensus.cc:493] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:39.432793 14564 raft_consensus.cc:3060] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:39.437736 14564 raft_consensus.cc:515] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } }
I20260709 13:17:39.440109 14564 leader_election.cc:290] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 2 election: Requested vote from peers ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673), d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643)
I20260709 13:17:39.440943 14712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b395b43c234641fc985583e8ec908250" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
I20260709 13:17:39.441373 14763 consensus_queue.cc:1048] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:39.441574 14712 raft_consensus.cc:3055] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:39.442289 14712 raft_consensus.cc:740] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ac3d4d81e96f412caaff09b3861ef2ff, State: Running, Role: LEADER
I20260709 13:17:39.443185 14712 consensus_queue.cc:260] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } }
I20260709 13:17:39.444562 14712 raft_consensus.cc:3060] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:39.447566 14763 consensus_queue.cc:1048] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:39.450893 14640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b395b43c234641fc985583e8ec908250" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3cd29370cc54e1099c022714674d225"
I20260709 13:17:39.451488 14640 raft_consensus.cc:3060] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:39.466512 14712 raft_consensus.cc:2468] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 2.
I20260709 13:17:39.467805 14529 leader_election.cc:304] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, ac3d4d81e96f412caaff09b3861ef2ff; no voters: 
I20260709 13:17:39.468688 14785 raft_consensus.cc:2804] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:39.469170 14785 raft_consensus.cc:697] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 LEADER]: Becoming Leader. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Running, Role: LEADER
I20260709 13:17:39.467789 14640 raft_consensus.cc:2468] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 2.
I20260709 13:17:39.469985 14785 consensus_queue.cc:237] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } }
I20260709 13:17:39.477860 14457 catalog_manager.cc:5697] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 reported cstate change: term changed from 1 to 2, leader changed from ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67) to 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65). New cstate: current_term: 2 leader_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:39.480834 14763 consensus_queue.cc:1048] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:39.488483 14785 consensus_queue.cc:1048] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:39.496304 14786 raft_consensus.cc:993] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3: : Instructing follower d3cd29370cc54e1099c022714674d225 to start an election
I20260709 13:17:39.496763 14786 raft_consensus.cc:1081] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Signalling peer d3cd29370cc54e1099c022714674d225 to start an election
I20260709 13:17:39.497406 14763 raft_consensus.cc:993] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225: : Instructing follower ac3d4d81e96f412caaff09b3861ef2ff to start an election
I20260709 13:17:39.497836 14804 raft_consensus.cc:1081] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [term 1 LEADER]: Signalling peer ac3d4d81e96f412caaff09b3861ef2ff to start an election
I20260709 13:17:39.499125 14639 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2c99ec13aa4c4ff5af744e67c5164098"
dest_uuid: "d3cd29370cc54e1099c022714674d225"
 from {username='slave'} at 127.0.0.1:41754
I20260709 13:17:39.501258 14714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c246b0d9436a4b30819c7ce5f07fbd77"
dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
 from {username='slave'} at 127.0.0.1:60018
I20260709 13:17:39.504853 14714 raft_consensus.cc:493] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:39.505266 14714 raft_consensus.cc:3060] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:39.507130 14804 consensus_queue.cc:1048] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:39.511075 14762 consensus_queue.cc:1048] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:39.511909 14714 raft_consensus.cc:515] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.515992 14785 consensus_queue.cc:1048] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:39.520925 14563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c246b0d9436a4b30819c7ce5f07fbd77" candidate_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
I20260709 13:17:39.523310 14639 raft_consensus.cc:493] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:39.523819 14639 raft_consensus.cc:3060] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:39.528573 14640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c246b0d9436a4b30819c7ce5f07fbd77" candidate_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3cd29370cc54e1099c022714674d225"
I20260709 13:17:39.529235 14640 raft_consensus.cc:3055] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:39.529552 14640 raft_consensus.cc:740] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3cd29370cc54e1099c022714674d225, State: Running, Role: LEADER
I20260709 13:17:39.534690 14640 consensus_queue.cc:260] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.530500 14714 leader_election.cc:290] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 2 election: Requested vote from peers 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713), d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643)
I20260709 13:17:39.535728 14639 raft_consensus.cc:515] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.539337 14640 raft_consensus.cc:3060] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:39.542379 14639 leader_election.cc:290] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 2 election: Requested vote from peers 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:39.543795 14711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c99ec13aa4c4ff5af744e67c5164098" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
I20260709 13:17:39.546636 14564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c99ec13aa4c4ff5af744e67c5164098" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
I20260709 13:17:39.547257 14564 raft_consensus.cc:3055] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:39.547339 14762 consensus_queue.cc:1048] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:39.549659 14640 raft_consensus.cc:2468] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac3d4d81e96f412caaff09b3861ef2ff in term 2.
I20260709 13:17:39.550927 14679 leader_election.cc:304] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ac3d4d81e96f412caaff09b3861ef2ff, d3cd29370cc54e1099c022714674d225; no voters: 8c779a0db43c47378f25bcb2bbeb54e3
I20260709 13:17:39.551870 14764 raft_consensus.cc:2804] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:39.547637 14564 raft_consensus.cc:740] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Running, Role: LEADER
I20260709 13:17:39.552366 14764 raft_consensus.cc:697] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 LEADER]: Becoming Leader. State: Replica: ac3d4d81e96f412caaff09b3861ef2ff, State: Running, Role: LEADER
I20260709 13:17:39.553259 14764 consensus_queue.cc:237] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.553373 14564 consensus_queue.cc:260] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.554675 14564 raft_consensus.cc:3060] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:39.571416 14762 raft_consensus.cc:993] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3: : Instructing follower d3cd29370cc54e1099c022714674d225 to start an election
I20260709 13:17:39.571897 14762 raft_consensus.cc:1081] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Signalling peer d3cd29370cc54e1099c022714674d225 to start an election
I20260709 13:17:39.573675 14564 raft_consensus.cc:2468] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3cd29370cc54e1099c022714674d225 in term 2.
I20260709 13:17:39.580335 14604 leader_election.cc:304] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, d3cd29370cc54e1099c022714674d225; no voters: ac3d4d81e96f412caaff09b3861ef2ff
I20260709 13:17:39.581557 14640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "accbd816047c4054a6f506623e82f905"
dest_uuid: "d3cd29370cc54e1099c022714674d225"
 from {username='slave'} at 127.0.0.1:41754
I20260709 13:17:39.581997 14763 raft_consensus.cc:2804] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:39.582495 14763 raft_consensus.cc:697] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225 [term 2 LEADER]: Becoming Leader. State: Replica: d3cd29370cc54e1099c022714674d225, State: Running, Role: LEADER
I20260709 13:17:39.582862 14640 raft_consensus.cc:493] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:39.583280 14640 raft_consensus.cc:3060] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:39.583302 14763 consensus_queue.cc:237] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.590413 14640 raft_consensus.cc:515] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.596335 14640 leader_election.cc:290] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 2 election: Requested vote from peers 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:39.597558 14714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accbd816047c4054a6f506623e82f905" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
I20260709 13:17:39.599880 14714 raft_consensus.cc:3060] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:39.597638 14564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accbd816047c4054a6f506623e82f905" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
I20260709 13:17:39.602530 14564 raft_consensus.cc:3055] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:39.602990 14564 raft_consensus.cc:740] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Running, Role: LEADER
I20260709 13:17:39.604012 14564 consensus_queue.cc:260] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.605242 14564 raft_consensus.cc:3060] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:39.606860 14714 raft_consensus.cc:2468] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3cd29370cc54e1099c022714674d225 in term 2.
I20260709 13:17:39.608201 14604 leader_election.cc:304] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac3d4d81e96f412caaff09b3861ef2ff, d3cd29370cc54e1099c022714674d225; no voters: 
I20260709 13:17:39.609051 14804 raft_consensus.cc:2804] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:39.609527 14804 raft_consensus.cc:697] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [term 2 LEADER]: Becoming Leader. State: Replica: d3cd29370cc54e1099c022714674d225, State: Running, Role: LEADER
I20260709 13:17:39.610432 14804 consensus_queue.cc:237] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.616947 14564 raft_consensus.cc:2468] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3cd29370cc54e1099c022714674d225 in term 2.
I20260709 13:17:39.616798 14460 catalog_manager.cc:5697] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff reported cstate change: term changed from 1 to 2, leader changed from d3cd29370cc54e1099c022714674d225 (127.14.23.66) to ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67). New cstate: current_term: 2 leader_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:39.626524 14458 catalog_manager.cc:5697] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225 reported cstate change: term changed from 1 to 2, leader changed from 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65) to d3cd29370cc54e1099c022714674d225 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "d3cd29370cc54e1099c022714674d225" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:39.640053 14458 catalog_manager.cc:5697] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 reported cstate change: term changed from 1 to 2, leader changed from 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65) to d3cd29370cc54e1099c022714674d225 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "d3cd29370cc54e1099c022714674d225" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:39.664466 14786 raft_consensus.cc:993] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3: : Instructing follower d3cd29370cc54e1099c022714674d225 to start an election
I20260709 13:17:39.664928 14762 raft_consensus.cc:1081] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Signalling peer d3cd29370cc54e1099c022714674d225 to start an election
I20260709 13:17:39.666383 14640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9bfd1e022dbc48a7805a15dbf9324642"
dest_uuid: "d3cd29370cc54e1099c022714674d225"
 from {username='slave'} at 127.0.0.1:41754
I20260709 13:17:39.666877 14640 raft_consensus.cc:493] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:39.667181 14640 raft_consensus.cc:3060] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:39.671875 14640 raft_consensus.cc:515] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.673460 14640 leader_election.cc:290] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 2 election: Requested vote from peers 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:39.674504 14564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd1e022dbc48a7805a15dbf9324642" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
I20260709 13:17:39.675033 14564 raft_consensus.cc:3055] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:39.675340 14564 raft_consensus.cc:740] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Running, Role: LEADER
I20260709 13:17:39.675973 14564 consensus_queue.cc:260] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.676856 14564 raft_consensus.cc:3060] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:39.681407 14564 raft_consensus.cc:2468] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3cd29370cc54e1099c022714674d225 in term 2.
I20260709 13:17:39.674854 14714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd1e022dbc48a7805a15dbf9324642" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
I20260709 13:17:39.682039 14714 raft_consensus.cc:3060] T 9bfd1e022dbc48a7805a15dbf9324642 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:39.682576 14604 leader_election.cc:304] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, d3cd29370cc54e1099c022714674d225; no voters: 
I20260709 13:17:39.683423 14804 raft_consensus.cc:2804] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:39.683926 14804 raft_consensus.cc:697] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [term 2 LEADER]: Becoming Leader. State: Replica: d3cd29370cc54e1099c022714674d225, State: Running, Role: LEADER
I20260709 13:17:39.684739 14804 consensus_queue.cc:237] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:39.687443 14714 raft_consensus.cc:2468] T 9bfd1e022dbc48a7805a15dbf9324642 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3cd29370cc54e1099c022714674d225 in term 2.
I20260709 13:17:39.692654 14458 catalog_manager.cc:5697] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 reported cstate change: term changed from 1 to 2, leader changed from 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65) to d3cd29370cc54e1099c022714674d225 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "d3cd29370cc54e1099c022714674d225" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:39.915809 14640 raft_consensus.cc:1275] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Refusing update from remote peer 8c779a0db43c47378f25bcb2bbeb54e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:39.917280 14762 consensus_queue.cc:1048] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:39.925616 14714 raft_consensus.cc:1275] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Refusing update from remote peer 8c779a0db43c47378f25bcb2bbeb54e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:39.927572 14785 consensus_queue.cc:1048] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:39.954762 14640 raft_consensus.cc:1275] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Refusing update from remote peer ac3d4d81e96f412caaff09b3861ef2ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:39.956351 14764 consensus_queue.cc:1048] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:39.964109 14564 raft_consensus.cc:3060] T c246b0d9436a4b30819c7ce5f07fbd77 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:39.972713 14564 raft_consensus.cc:1275] T c246b0d9436a4b30819c7ce5f07fbd77 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Refusing update from remote peer ac3d4d81e96f412caaff09b3861ef2ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:39.973951 14841 consensus_queue.cc:1048] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:40.035061 14714 raft_consensus.cc:3060] T 2c99ec13aa4c4ff5af744e67c5164098 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:40.042728 14714 raft_consensus.cc:1275] T 2c99ec13aa4c4ff5af744e67c5164098 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Refusing update from remote peer d3cd29370cc54e1099c022714674d225: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:40.044166 14804 consensus_queue.cc:1048] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:40.056452 14564 raft_consensus.cc:1275] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Refusing update from remote peer d3cd29370cc54e1099c022714674d225: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:40.057972 14806 consensus_queue.cc:1048] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:40.112010 14564 raft_consensus.cc:1275] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Refusing update from remote peer d3cd29370cc54e1099c022714674d225: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:40.113207 14806 consensus_queue.cc:1048] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:40.120762 14712 raft_consensus.cc:1275] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Refusing update from remote peer d3cd29370cc54e1099c022714674d225: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:40.122681 14806 consensus_queue.cc:1048] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:40.140744 14564 raft_consensus.cc:1275] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Refusing update from remote peer d3cd29370cc54e1099c022714674d225: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:40.141975 14842 consensus_queue.cc:1048] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:40.151656 14712 raft_consensus.cc:1275] T 9bfd1e022dbc48a7805a15dbf9324642 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Refusing update from remote peer d3cd29370cc54e1099c022714674d225: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:40.154686 14804 consensus_queue.cc:1048] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:42.311825 14429 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:17:42.312721 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:42.328644 14640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d1ea631a1bb4cf8a4f25e155142c0d3"
dest_uuid: "d3cd29370cc54e1099c022714674d225"
mode: GRACEFUL
new_leader_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
 from {username='slave'} at 127.0.0.1:50356
I20260709 13:17:42.329133 14640 raft_consensus.cc:606] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 1 LEADER]: Received request to transfer leadership to TS ac3d4d81e96f412caaff09b3861ef2ff
I20260709 13:17:42.339674 14564 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a36368cb545c46bd929582b28a3fece5"
dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
mode: GRACEFUL
new_leader_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
 from {username='slave'} at 127.0.0.1:58844
I20260709 13:17:42.340163 14564 raft_consensus.cc:606] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Received request to transfer leadership to TS ac3d4d81e96f412caaff09b3861ef2ff
I20260709 13:17:42.341249 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:42.341739 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:42.588579 14804 raft_consensus.cc:993] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225: : Instructing follower ac3d4d81e96f412caaff09b3861ef2ff to start an election
I20260709 13:17:42.588996 14804 raft_consensus.cc:1081] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 1 LEADER]: Signalling peer ac3d4d81e96f412caaff09b3861ef2ff to start an election
I20260709 13:17:42.590416 14712 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5d1ea631a1bb4cf8a4f25e155142c0d3"
dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
 from {username='slave'} at 127.0.0.1:60018
I20260709 13:17:42.590883 14712 raft_consensus.cc:493] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:42.591176 14712 raft_consensus.cc:3060] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.595537 14712 raft_consensus.cc:515] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:42.597251 14712 leader_election.cc:290] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 2 election: Requested vote from peers 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713), d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643)
I20260709 13:17:42.598047 14564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d1ea631a1bb4cf8a4f25e155142c0d3" candidate_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
I20260709 13:17:42.598397 14640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d1ea631a1bb4cf8a4f25e155142c0d3" candidate_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3cd29370cc54e1099c022714674d225"
I20260709 13:17:42.598609 14564 raft_consensus.cc:3060] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.598901 14640 raft_consensus.cc:3055] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:42.599166 14640 raft_consensus.cc:740] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3cd29370cc54e1099c022714674d225, State: Running, Role: LEADER
I20260709 13:17:42.599772 14640 consensus_queue.cc:260] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:42.600657 14640 raft_consensus.cc:3060] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.603410 14564 raft_consensus.cc:2468] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac3d4d81e96f412caaff09b3861ef2ff in term 2.
I20260709 13:17:42.604548 14679 leader_election.cc:304] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, ac3d4d81e96f412caaff09b3861ef2ff; no voters: 
I20260709 13:17:42.605247 14848 raft_consensus.cc:2804] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:42.606472 14640 raft_consensus.cc:2468] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac3d4d81e96f412caaff09b3861ef2ff in term 2.
I20260709 13:17:42.606463 14848 raft_consensus.cc:697] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 LEADER]: Becoming Leader. State: Replica: ac3d4d81e96f412caaff09b3861ef2ff, State: Running, Role: LEADER
I20260709 13:17:42.607928 14848 consensus_queue.cc:237] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:42.615039 14460 catalog_manager.cc:5697] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff reported cstate change: term changed from 1 to 2, leader changed from d3cd29370cc54e1099c022714674d225 (127.14.23.66) to ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67). New cstate: current_term: 2 leader_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:42.718613 14846 raft_consensus.cc:993] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3: : Instructing follower ac3d4d81e96f412caaff09b3861ef2ff to start an election
I20260709 13:17:42.718994 14860 raft_consensus.cc:1081] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Signalling peer ac3d4d81e96f412caaff09b3861ef2ff to start an election
I20260709 13:17:42.720434 14712 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a36368cb545c46bd929582b28a3fece5"
dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
 from {username='slave'} at 127.0.0.1:60008
I20260709 13:17:42.720919 14712 raft_consensus.cc:493] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:42.721247 14712 raft_consensus.cc:3060] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.725903 14712 raft_consensus.cc:515] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:42.728008 14712 leader_election.cc:290] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 2 election: Requested vote from peers 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713), d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643)
I20260709 13:17:42.728919 14564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a36368cb545c46bd929582b28a3fece5" candidate_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
I20260709 13:17:42.729111 14640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a36368cb545c46bd929582b28a3fece5" candidate_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3cd29370cc54e1099c022714674d225"
I20260709 13:17:42.729430 14564 raft_consensus.cc:3055] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:42.729748 14564 raft_consensus.cc:740] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Running, Role: LEADER
I20260709 13:17:42.729642 14640 raft_consensus.cc:3060] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.730506 14564 consensus_queue.cc:260] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:42.731501 14564 raft_consensus.cc:3060] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.735052 14640 raft_consensus.cc:2468] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac3d4d81e96f412caaff09b3861ef2ff in term 2.
I20260709 13:17:42.736091 14564 raft_consensus.cc:2468] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac3d4d81e96f412caaff09b3861ef2ff in term 2.
I20260709 13:17:42.736128 14679 leader_election.cc:304] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac3d4d81e96f412caaff09b3861ef2ff, d3cd29370cc54e1099c022714674d225; no voters: 
I20260709 13:17:42.736879 14848 raft_consensus.cc:2804] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:42.737255 14848 raft_consensus.cc:697] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 LEADER]: Becoming Leader. State: Replica: ac3d4d81e96f412caaff09b3861ef2ff, State: Running, Role: LEADER
I20260709 13:17:42.737905 14848 consensus_queue.cc:237] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:42.744618 14460 catalog_manager.cc:5697] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff reported cstate change: term changed from 1 to 2, leader changed from 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65) to ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67). New cstate: current_term: 2 leader_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:43.079849 14564 raft_consensus.cc:1275] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Refusing update from remote peer ac3d4d81e96f412caaff09b3861ef2ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:43.080931 14848 consensus_queue.cc:1048] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:43.090507 14640 raft_consensus.cc:1275] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Refusing update from remote peer ac3d4d81e96f412caaff09b3861ef2ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:43.092587 14854 consensus_queue.cc:1048] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:43.300652 14640 raft_consensus.cc:1275] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Refusing update from remote peer ac3d4d81e96f412caaff09b3861ef2ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:43.301868 14854 consensus_queue.cc:1048] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:43.310380 14564 raft_consensus.cc:1275] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Refusing update from remote peer ac3d4d81e96f412caaff09b3861ef2ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:43.311723 14854 consensus_queue.cc:1048] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:43.342577 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:43.352414 14564 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b395b43c234641fc985583e8ec908250"
dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
mode: GRACEFUL
new_leader_uuid: "d3cd29370cc54e1099c022714674d225"
 from {username='slave'} at 127.0.0.1:45440
I20260709 13:17:43.352901 14564 raft_consensus.cc:606] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 LEADER]: Received request to transfer leadership to TS d3cd29370cc54e1099c022714674d225
I20260709 13:17:43.357435 14564 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c4c219a8c864700b9f22ff2a9f23e68"
dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
mode: GRACEFUL
new_leader_uuid: "d3cd29370cc54e1099c022714674d225"
 from {username='slave'} at 127.0.0.1:45440
I20260709 13:17:43.357950 14564 raft_consensus.cc:606] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Received request to transfer leadership to TS d3cd29370cc54e1099c022714674d225
I20260709 13:17:43.362548 14714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a36368cb545c46bd929582b28a3fece5"
dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
mode: GRACEFUL
new_leader_uuid: "d3cd29370cc54e1099c022714674d225"
 from {username='slave'} at 127.0.0.1:60020
I20260709 13:17:43.363070 14714 raft_consensus.cc:606] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 LEADER]: Received request to transfer leadership to TS d3cd29370cc54e1099c022714674d225
I20260709 13:17:43.367759 14564 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b42fa1c328df4d64818288ee87e30a46"
dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
mode: GRACEFUL
new_leader_uuid: "d3cd29370cc54e1099c022714674d225"
 from {username='slave'} at 127.0.0.1:45440
I20260709 13:17:43.368233 14564 raft_consensus.cc:606] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Received request to transfer leadership to TS d3cd29370cc54e1099c022714674d225
I20260709 13:17:43.374966 14714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d1ea631a1bb4cf8a4f25e155142c0d3"
dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
mode: GRACEFUL
new_leader_uuid: "d3cd29370cc54e1099c022714674d225"
 from {username='slave'} at 127.0.0.1:60020
I20260709 13:17:43.375494 14714 raft_consensus.cc:606] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 LEADER]: Received request to transfer leadership to TS d3cd29370cc54e1099c022714674d225
I20260709 13:17:43.632855 14860 raft_consensus.cc:993] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3: : Instructing follower d3cd29370cc54e1099c022714674d225 to start an election
I20260709 13:17:43.633280 14868 raft_consensus.cc:1081] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Signalling peer d3cd29370cc54e1099c022714674d225 to start an election
I20260709 13:17:43.634732 14640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5c4c219a8c864700b9f22ff2a9f23e68"
dest_uuid: "d3cd29370cc54e1099c022714674d225"
 from {username='slave'} at 127.0.0.1:41754
I20260709 13:17:43.635260 14640 raft_consensus.cc:493] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:43.635532 14640 raft_consensus.cc:3060] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.639586 14640 raft_consensus.cc:515] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:43.641041 14640 leader_election.cc:290] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 2 election: Requested vote from peers 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:43.642004 14564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4c219a8c864700b9f22ff2a9f23e68" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
I20260709 13:17:43.642187 14714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4c219a8c864700b9f22ff2a9f23e68" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
I20260709 13:17:43.642570 14564 raft_consensus.cc:3055] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:43.642694 14714 raft_consensus.cc:3060] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.642850 14564 raft_consensus.cc:740] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Running, Role: LEADER
I20260709 13:17:43.643556 14564 consensus_queue.cc:260] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:43.644372 14564 raft_consensus.cc:3060] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.647545 14714 raft_consensus.cc:2468] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3cd29370cc54e1099c022714674d225 in term 2.
I20260709 13:17:43.648900 14604 leader_election.cc:304] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac3d4d81e96f412caaff09b3861ef2ff, d3cd29370cc54e1099c022714674d225; no voters: 
I20260709 13:17:43.649107 14564 raft_consensus.cc:2468] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3cd29370cc54e1099c022714674d225 in term 2.
I20260709 13:17:43.649796 14804 raft_consensus.cc:2804] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:43.650161 14860 raft_consensus.cc:993] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3: : Instructing follower d3cd29370cc54e1099c022714674d225 to start an election
I20260709 13:17:43.650807 14804 raft_consensus.cc:697] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [term 2 LEADER]: Becoming Leader. State: Replica: d3cd29370cc54e1099c022714674d225, State: Running, Role: LEADER
I20260709 13:17:43.651125 14868 raft_consensus.cc:1081] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 LEADER]: Signalling peer d3cd29370cc54e1099c022714674d225 to start an election
I20260709 13:17:43.651685 14804 consensus_queue.cc:237] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:43.653215 14640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b395b43c234641fc985583e8ec908250"
dest_uuid: "d3cd29370cc54e1099c022714674d225"
 from {username='slave'} at 127.0.0.1:41754
I20260709 13:17:43.653661 14640 raft_consensus.cc:493] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:43.653898 14640 raft_consensus.cc:3060] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:43.659715 14864 raft_consensus.cc:993] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff: : Instructing follower d3cd29370cc54e1099c022714674d225 to start an election
I20260709 13:17:43.660151 14864 raft_consensus.cc:1081] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 LEADER]: Signalling peer d3cd29370cc54e1099c022714674d225 to start an election
I20260709 13:17:43.660145 14640 raft_consensus.cc:515] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } }
I20260709 13:17:43.661968 14638 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5d1ea631a1bb4cf8a4f25e155142c0d3"
dest_uuid: "d3cd29370cc54e1099c022714674d225"
 from {username='slave'} at 127.0.0.1:41746
I20260709 13:17:43.662510 14638 raft_consensus.cc:493] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:43.662900 14638 raft_consensus.cc:3060] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:43.664147 14640 leader_election.cc:290] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 3 election: Requested vote from peers ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673), 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713)
I20260709 13:17:43.664706 14714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b395b43c234641fc985583e8ec908250" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
I20260709 13:17:43.665295 14714 raft_consensus.cc:3060] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:43.669404 14564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b395b43c234641fc985583e8ec908250" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
I20260709 13:17:43.669528 14638 raft_consensus.cc:515] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:43.670001 14564 raft_consensus.cc:3055] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:43.670346 14564 raft_consensus.cc:740] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Running, Role: LEADER
I20260709 13:17:43.670387 14458 catalog_manager.cc:5697] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 reported cstate change: term changed from 1 to 2, leader changed from 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65) to d3cd29370cc54e1099c022714674d225 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "d3cd29370cc54e1099c022714674d225" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:43.671367 14714 raft_consensus.cc:2468] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3cd29370cc54e1099c022714674d225 in term 3.
I20260709 13:17:43.671140 14564 consensus_queue.cc:260] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } }
I20260709 13:17:43.671837 14638 leader_election.cc:290] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 3 election: Requested vote from peers 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:43.672256 14564 raft_consensus.cc:3060] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:43.673030 14563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d1ea631a1bb4cf8a4f25e155142c0d3" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
I20260709 13:17:43.673581 14563 raft_consensus.cc:3060] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:43.676222 14714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d1ea631a1bb4cf8a4f25e155142c0d3" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
I20260709 13:17:43.676779 14714 raft_consensus.cc:3055] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:43.677067 14714 raft_consensus.cc:740] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ac3d4d81e96f412caaff09b3861ef2ff, State: Running, Role: LEADER
I20260709 13:17:43.677817 14714 consensus_queue.cc:260] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:43.679044 14714 raft_consensus.cc:3060] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:43.679975 14563 raft_consensus.cc:2468] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3cd29370cc54e1099c022714674d225 in term 3.
I20260709 13:17:43.680289 14860 raft_consensus.cc:993] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3: : Instructing follower d3cd29370cc54e1099c022714674d225 to start an election
I20260709 13:17:43.680693 14860 raft_consensus.cc:1081] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Signalling peer d3cd29370cc54e1099c022714674d225 to start an election
I20260709 13:17:43.681735 14604 leader_election.cc:304] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, d3cd29370cc54e1099c022714674d225; no voters: 
I20260709 13:17:43.682348 14638 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b42fa1c328df4d64818288ee87e30a46"
dest_uuid: "d3cd29370cc54e1099c022714674d225"
 from {username='slave'} at 127.0.0.1:41754
I20260709 13:17:43.682554 14804 raft_consensus.cc:2804] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:43.683027 14804 raft_consensus.cc:697] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 3 LEADER]: Becoming Leader. State: Replica: d3cd29370cc54e1099c022714674d225, State: Running, Role: LEADER
I20260709 13:17:43.683933 14564 raft_consensus.cc:2468] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3cd29370cc54e1099c022714674d225 in term 3.
I20260709 13:17:43.683830 14804 consensus_queue.cc:237] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:43.685956 14714 raft_consensus.cc:2468] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3cd29370cc54e1099c022714674d225 in term 3.
I20260709 13:17:43.683542 14604 leader_election.cc:304] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac3d4d81e96f412caaff09b3861ef2ff, d3cd29370cc54e1099c022714674d225; no voters: 
I20260709 13:17:43.687551 14871 raft_consensus.cc:2804] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:43.688138 14871 raft_consensus.cc:697] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 [term 3 LEADER]: Becoming Leader. State: Replica: d3cd29370cc54e1099c022714674d225, State: Running, Role: LEADER
I20260709 13:17:43.683005 14638 raft_consensus.cc:493] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:43.688840 14638 raft_consensus.cc:3060] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.689180 14871 consensus_queue.cc:237] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } }
I20260709 13:17:43.695609 14638 raft_consensus.cc:515] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:43.697717 14638 leader_election.cc:290] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 2 election: Requested vote from peers 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:43.699015 14458 catalog_manager.cc:5697] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 reported cstate change: term changed from 2 to 3, leader changed from ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67) to d3cd29370cc54e1099c022714674d225 (127.14.23.66). New cstate: current_term: 3 leader_uuid: "d3cd29370cc54e1099c022714674d225" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:43.700165 14714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42fa1c328df4d64818288ee87e30a46" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
I20260709 13:17:43.700778 14714 raft_consensus.cc:3060] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.700578 14458 catalog_manager.cc:5697] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 reported cstate change: term changed from 2 to 3, leader changed from 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65) to d3cd29370cc54e1099c022714674d225 (127.14.23.66). New cstate: current_term: 3 leader_uuid: "d3cd29370cc54e1099c022714674d225" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:43.702601 14564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42fa1c328df4d64818288ee87e30a46" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
I20260709 13:17:43.703190 14564 raft_consensus.cc:3055] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:43.703527 14564 raft_consensus.cc:740] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Running, Role: LEADER
I20260709 13:17:43.704202 14564 consensus_queue.cc:260] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:43.705354 14564 raft_consensus.cc:3060] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.706867 14714 raft_consensus.cc:2468] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3cd29370cc54e1099c022714674d225 in term 2.
I20260709 13:17:43.707840 14604 leader_election.cc:304] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac3d4d81e96f412caaff09b3861ef2ff, d3cd29370cc54e1099c022714674d225; no voters: 
I20260709 13:17:43.708472 14871 raft_consensus.cc:2804] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:43.708988 14871 raft_consensus.cc:697] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [term 2 LEADER]: Becoming Leader. State: Replica: d3cd29370cc54e1099c022714674d225, State: Running, Role: LEADER
I20260709 13:17:43.713325 14564 raft_consensus.cc:2468] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3cd29370cc54e1099c022714674d225 in term 2.
I20260709 13:17:43.709872 14871 consensus_queue.cc:237] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:43.722136 14458 catalog_manager.cc:5697] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 reported cstate change: term changed from 1 to 2, leader changed from 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65) to d3cd29370cc54e1099c022714674d225 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "d3cd29370cc54e1099c022714674d225" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:43.812325 14864 raft_consensus.cc:993] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff: : Instructing follower d3cd29370cc54e1099c022714674d225 to start an election
I20260709 13:17:43.812701 14854 raft_consensus.cc:1081] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 LEADER]: Signalling peer d3cd29370cc54e1099c022714674d225 to start an election
I20260709 13:17:43.814153 14638 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a36368cb545c46bd929582b28a3fece5"
dest_uuid: "d3cd29370cc54e1099c022714674d225"
 from {username='slave'} at 127.0.0.1:41746
I20260709 13:17:43.814611 14638 raft_consensus.cc:493] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:43.814868 14638 raft_consensus.cc:3060] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:43.819363 14638 raft_consensus.cc:515] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:43.820881 14638 leader_election.cc:290] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 3 election: Requested vote from peers 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:43.821741 14563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a36368cb545c46bd929582b28a3fece5" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
I20260709 13:17:43.822046 14714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a36368cb545c46bd929582b28a3fece5" candidate_uuid: "d3cd29370cc54e1099c022714674d225" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
I20260709 13:17:43.822288 14563 raft_consensus.cc:3060] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:43.822588 14714 raft_consensus.cc:3055] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:43.822852 14714 raft_consensus.cc:740] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ac3d4d81e96f412caaff09b3861ef2ff, State: Running, Role: LEADER
I20260709 13:17:43.823462 14714 consensus_queue.cc:260] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:43.824360 14714 raft_consensus.cc:3060] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:43.827108 14563 raft_consensus.cc:2468] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3cd29370cc54e1099c022714674d225 in term 3.
I20260709 13:17:43.828155 14604 leader_election.cc:304] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, d3cd29370cc54e1099c022714674d225; no voters: 
I20260709 13:17:43.828586 14714 raft_consensus.cc:2468] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3cd29370cc54e1099c022714674d225 in term 3.
I20260709 13:17:43.829015 14871 raft_consensus.cc:2804] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:43.829509 14871 raft_consensus.cc:697] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225 [term 3 LEADER]: Becoming Leader. State: Replica: d3cd29370cc54e1099c022714674d225, State: Running, Role: LEADER
I20260709 13:17:43.830265 14871 consensus_queue.cc:237] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:43.835942 14458 catalog_manager.cc:5697] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225 reported cstate change: term changed from 2 to 3, leader changed from ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67) to d3cd29370cc54e1099c022714674d225 (127.14.23.66). New cstate: current_term: 3 leader_uuid: "d3cd29370cc54e1099c022714674d225" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:44.071981 14563 raft_consensus.cc:1275] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Refusing update from remote peer d3cd29370cc54e1099c022714674d225: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:44.073225 14871 consensus_queue.cc:1048] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:44.083092 14714 raft_consensus.cc:1275] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Refusing update from remote peer d3cd29370cc54e1099c022714674d225: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:44.085093 14804 consensus_queue.cc:1048] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:44.087318 14714 raft_consensus.cc:1275] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Refusing update from remote peer d3cd29370cc54e1099c022714674d225: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:44.088634 14804 consensus_queue.cc:1048] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:44.098220 14563 raft_consensus.cc:1275] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Refusing update from remote peer d3cd29370cc54e1099c022714674d225: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:44.101145 14885 consensus_queue.cc:1048] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:44.107287 14714 raft_consensus.cc:1275] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Refusing update from remote peer d3cd29370cc54e1099c022714674d225: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:44.108887 14885 consensus_queue.cc:1048] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:44.119011 14563 raft_consensus.cc:1275] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Refusing update from remote peer d3cd29370cc54e1099c022714674d225: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:44.121181 14804 consensus_queue.cc:1048] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:44.170305 14563 raft_consensus.cc:1275] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Refusing update from remote peer d3cd29370cc54e1099c022714674d225: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:44.171471 14804 consensus_queue.cc:1048] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:44.180150 14714 raft_consensus.cc:1275] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Refusing update from remote peer d3cd29370cc54e1099c022714674d225: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:44.181855 14871 consensus_queue.cc:1048] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:44.327291 14714 raft_consensus.cc:1275] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Refusing update from remote peer d3cd29370cc54e1099c022714674d225: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:44.328662 14871 consensus_queue.cc:1048] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:44.338980 14563 raft_consensus.cc:1275] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Refusing update from remote peer d3cd29370cc54e1099c022714674d225: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:44.341229 14885 consensus_queue.cc:1048] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:46.385282 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:46.392448 14638 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c4c219a8c864700b9f22ff2a9f23e68"
dest_uuid: "d3cd29370cc54e1099c022714674d225"
mode: GRACEFUL
new_leader_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
 from {username='slave'} at 127.0.0.1:50356
I20260709 13:17:46.393246 14638 raft_consensus.cc:606] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [term 2 LEADER]: Received request to transfer leadership to TS 8c779a0db43c47378f25bcb2bbeb54e3
I20260709 13:17:46.396180 14638 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d1ea631a1bb4cf8a4f25e155142c0d3"
dest_uuid: "d3cd29370cc54e1099c022714674d225"
mode: GRACEFUL
new_leader_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
 from {username='slave'} at 127.0.0.1:50356
I20260709 13:17:46.396669 14638 raft_consensus.cc:606] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 3 LEADER]: Received request to transfer leadership to TS 8c779a0db43c47378f25bcb2bbeb54e3
I20260709 13:17:46.399092 14638 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bfd1e022dbc48a7805a15dbf9324642"
dest_uuid: "d3cd29370cc54e1099c022714674d225"
mode: GRACEFUL
new_leader_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
 from {username='slave'} at 127.0.0.1:50356
I20260709 13:17:46.399547 14638 raft_consensus.cc:606] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [term 2 LEADER]: Received request to transfer leadership to TS 8c779a0db43c47378f25bcb2bbeb54e3
I20260709 13:17:46.401832 14638 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "accbd816047c4054a6f506623e82f905"
dest_uuid: "d3cd29370cc54e1099c022714674d225"
mode: GRACEFUL
new_leader_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
 from {username='slave'} at 127.0.0.1:50356
I20260709 13:17:46.402336 14638 raft_consensus.cc:606] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [term 2 LEADER]: Received request to transfer leadership to TS ac3d4d81e96f412caaff09b3861ef2ff
I20260709 13:17:46.404615 14638 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b42fa1c328df4d64818288ee87e30a46"
dest_uuid: "d3cd29370cc54e1099c022714674d225"
mode: GRACEFUL
new_leader_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
 from {username='slave'} at 127.0.0.1:50356
I20260709 13:17:46.405076 14638 raft_consensus.cc:606] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [term 2 LEADER]: Received request to transfer leadership to TS ac3d4d81e96f412caaff09b3861ef2ff
I20260709 13:17:46.406168 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:17:46.406750 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:46.696982 14893 raft_consensus.cc:993] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225: : Instructing follower ac3d4d81e96f412caaff09b3861ef2ff to start an election
I20260709 13:17:46.697528 14895 raft_consensus.cc:1081] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [term 2 LEADER]: Signalling peer ac3d4d81e96f412caaff09b3861ef2ff to start an election
I20260709 13:17:46.699033 14714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b42fa1c328df4d64818288ee87e30a46"
dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
 from {username='slave'} at 127.0.0.1:60018
I20260709 13:17:46.699486 14714 raft_consensus.cc:493] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:46.699815 14714 raft_consensus.cc:3060] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:46.704767 14714 raft_consensus.cc:515] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:46.706384 14714 leader_election.cc:290] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 3 election: Requested vote from peers 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713), d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643)
I20260709 13:17:46.707484 14562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42fa1c328df4d64818288ee87e30a46" candidate_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
I20260709 13:17:46.707487 14638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b42fa1c328df4d64818288ee87e30a46" candidate_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d3cd29370cc54e1099c022714674d225"
I20260709 13:17:46.708324 14562 raft_consensus.cc:3060] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:46.708390 14638 raft_consensus.cc:3055] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:46.708714 14638 raft_consensus.cc:740] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d3cd29370cc54e1099c022714674d225, State: Running, Role: LEADER
I20260709 13:17:46.709403 14638 consensus_queue.cc:260] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:46.710386 14638 raft_consensus.cc:3060] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:46.713485 14562 raft_consensus.cc:2468] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac3d4d81e96f412caaff09b3861ef2ff in term 3.
I20260709 13:17:46.714968 14679 leader_election.cc:304] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, ac3d4d81e96f412caaff09b3861ef2ff; no voters: 
I20260709 13:17:46.715406 14638 raft_consensus.cc:2468] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac3d4d81e96f412caaff09b3861ef2ff in term 3.
I20260709 13:17:46.715978 14854 raft_consensus.cc:2804] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:46.716945 14854 raft_consensus.cc:697] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 LEADER]: Becoming Leader. State: Replica: ac3d4d81e96f412caaff09b3861ef2ff, State: Running, Role: LEADER
I20260709 13:17:46.717830 14854 consensus_queue.cc:237] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:46.724972 14460 catalog_manager.cc:5697] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff reported cstate change: term changed from 2 to 3, leader changed from d3cd29370cc54e1099c022714674d225 (127.14.23.66) to ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67). New cstate: current_term: 3 leader_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:46.747682 14893 raft_consensus.cc:993] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225: : Instructing follower ac3d4d81e96f412caaff09b3861ef2ff to start an election
I20260709 13:17:46.748103 14895 raft_consensus.cc:1081] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [term 2 LEADER]: Signalling peer ac3d4d81e96f412caaff09b3861ef2ff to start an election
I20260709 13:17:46.749455 14714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "accbd816047c4054a6f506623e82f905"
dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
 from {username='slave'} at 127.0.0.1:60018
I20260709 13:17:46.749887 14714 raft_consensus.cc:493] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:46.750190 14714 raft_consensus.cc:3060] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:46.754475 14714 raft_consensus.cc:515] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:46.756031 14714 leader_election.cc:290] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 3 election: Requested vote from peers 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713), d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643)
I20260709 13:17:46.756718 14562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accbd816047c4054a6f506623e82f905" candidate_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
I20260709 13:17:46.757061 14638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accbd816047c4054a6f506623e82f905" candidate_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d3cd29370cc54e1099c022714674d225"
I20260709 13:17:46.757237 14562 raft_consensus.cc:3060] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:46.757546 14638 raft_consensus.cc:3055] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:46.757822 14638 raft_consensus.cc:740] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d3cd29370cc54e1099c022714674d225, State: Running, Role: LEADER
I20260709 13:17:46.758482 14638 consensus_queue.cc:260] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:46.759342 14638 raft_consensus.cc:3060] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:46.762180 14562 raft_consensus.cc:2468] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac3d4d81e96f412caaff09b3861ef2ff in term 3.
I20260709 13:17:46.763195 14679 leader_election.cc:304] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, ac3d4d81e96f412caaff09b3861ef2ff; no voters: 
I20260709 13:17:46.764041 14854 raft_consensus.cc:2804] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:46.764318 14638 raft_consensus.cc:2468] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac3d4d81e96f412caaff09b3861ef2ff in term 3.
I20260709 13:17:46.764442 14854 raft_consensus.cc:697] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 LEADER]: Becoming Leader. State: Replica: ac3d4d81e96f412caaff09b3861ef2ff, State: Running, Role: LEADER
I20260709 13:17:46.765395 14854 consensus_queue.cc:237] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:46.772104 14460 catalog_manager.cc:5697] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff reported cstate change: term changed from 2 to 3, leader changed from d3cd29370cc54e1099c022714674d225 (127.14.23.66) to ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67). New cstate: current_term: 3 leader_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:46.802928 14893 raft_consensus.cc:993] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225: : Instructing follower 8c779a0db43c47378f25bcb2bbeb54e3 to start an election
I20260709 13:17:46.803390 14895 raft_consensus.cc:1081] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 3 LEADER]: Signalling peer 8c779a0db43c47378f25bcb2bbeb54e3 to start an election
I20260709 13:17:46.804849 14562 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5d1ea631a1bb4cf8a4f25e155142c0d3"
dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
 from {username='slave'} at 127.0.0.1:45436
I20260709 13:17:46.805325 14562 raft_consensus.cc:493] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:46.805620 14562 raft_consensus.cc:3060] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:46.809947 14562 raft_consensus.cc:515] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:46.811755 14562 leader_election.cc:290] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 4 election: Requested vote from peers d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:46.812562 14638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d1ea631a1bb4cf8a4f25e155142c0d3" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d3cd29370cc54e1099c022714674d225"
I20260709 13:17:46.812942 14714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d1ea631a1bb4cf8a4f25e155142c0d3" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
I20260709 13:17:46.813128 14638 raft_consensus.cc:3055] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:46.813375 14714 raft_consensus.cc:3060] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:46.813483 14638 raft_consensus.cc:740] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d3cd29370cc54e1099c022714674d225, State: Running, Role: LEADER
I20260709 13:17:46.814242 14638 consensus_queue.cc:260] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:46.815111 14638 raft_consensus.cc:3060] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:46.817466 14714 raft_consensus.cc:2468] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 4.
I20260709 13:17:46.818650 14529 leader_election.cc:304] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, ac3d4d81e96f412caaff09b3861ef2ff; no voters: 
I20260709 13:17:46.820238 14638 raft_consensus.cc:2468] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 4.
I20260709 13:17:46.820454 14903 raft_consensus.cc:2804] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:46.821761 14903 raft_consensus.cc:697] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 4 LEADER]: Becoming Leader. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Running, Role: LEADER
I20260709 13:17:46.822609 14903 consensus_queue.cc:237] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:46.829326 14460 catalog_manager.cc:5697] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 reported cstate change: term changed from 3 to 4, leader changed from d3cd29370cc54e1099c022714674d225 (127.14.23.66) to 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65). New cstate: current_term: 4 leader_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:46.910110 14893 raft_consensus.cc:993] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225: : Instructing follower 8c779a0db43c47378f25bcb2bbeb54e3 to start an election
I20260709 13:17:46.910501 14895 raft_consensus.cc:1081] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [term 2 LEADER]: Signalling peer 8c779a0db43c47378f25bcb2bbeb54e3 to start an election
I20260709 13:17:46.911857 14562 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9bfd1e022dbc48a7805a15dbf9324642"
dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
 from {username='slave'} at 127.0.0.1:45436
I20260709 13:17:46.912353 14562 raft_consensus.cc:493] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:46.912657 14562 raft_consensus.cc:3060] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:46.917514 14562 raft_consensus.cc:515] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:46.919171 14562 leader_election.cc:290] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 3 election: Requested vote from peers d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:46.920002 14638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd1e022dbc48a7805a15dbf9324642" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d3cd29370cc54e1099c022714674d225"
I20260709 13:17:46.920166 14714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfd1e022dbc48a7805a15dbf9324642" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
I20260709 13:17:46.920657 14714 raft_consensus.cc:3060] T 9bfd1e022dbc48a7805a15dbf9324642 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:46.920632 14638 raft_consensus.cc:3055] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:46.921161 14638 raft_consensus.cc:740] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d3cd29370cc54e1099c022714674d225, State: Running, Role: LEADER
I20260709 13:17:46.922026 14638 consensus_queue.cc:260] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:46.923302 14638 raft_consensus.cc:3060] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:46.927029 14714 raft_consensus.cc:2468] T 9bfd1e022dbc48a7805a15dbf9324642 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 3.
I20260709 13:17:46.928109 14529 leader_election.cc:304] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, ac3d4d81e96f412caaff09b3861ef2ff; no voters: 
I20260709 13:17:46.928797 14903 raft_consensus.cc:2804] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:46.929262 14903 raft_consensus.cc:697] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 LEADER]: Becoming Leader. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Running, Role: LEADER
I20260709 13:17:46.930110 14638 raft_consensus.cc:2468] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 3.
I20260709 13:17:46.930224 14903 consensus_queue.cc:237] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:46.932550 14893 raft_consensus.cc:993] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225: : Instructing follower 8c779a0db43c47378f25bcb2bbeb54e3 to start an election
I20260709 13:17:46.933056 14895 raft_consensus.cc:1081] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [term 2 LEADER]: Signalling peer 8c779a0db43c47378f25bcb2bbeb54e3 to start an election
I20260709 13:17:46.934552 14562 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5c4c219a8c864700b9f22ff2a9f23e68"
dest_uuid: "8c779a0db43c47378f25bcb2bbeb54e3"
 from {username='slave'} at 127.0.0.1:45436
I20260709 13:17:46.935122 14562 raft_consensus.cc:493] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:46.935446 14562 raft_consensus.cc:3060] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:46.938125 14460 catalog_manager.cc:5697] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 reported cstate change: term changed from 2 to 3, leader changed from d3cd29370cc54e1099c022714674d225 (127.14.23.66) to 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65). New cstate: current_term: 3 leader_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:46.940315 14562 raft_consensus.cc:515] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:46.942196 14562 leader_election.cc:290] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 3 election: Requested vote from peers d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643), ac3d4d81e96f412caaff09b3861ef2ff (127.14.23.67:42673)
I20260709 13:17:46.942828 14638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4c219a8c864700b9f22ff2a9f23e68" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d3cd29370cc54e1099c022714674d225"
I20260709 13:17:46.943361 14638 raft_consensus.cc:3055] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:46.943591 14714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4c219a8c864700b9f22ff2a9f23e68" candidate_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ac3d4d81e96f412caaff09b3861ef2ff"
I20260709 13:17:46.943732 14638 raft_consensus.cc:740] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d3cd29370cc54e1099c022714674d225, State: Running, Role: LEADER
I20260709 13:17:46.944147 14714 raft_consensus.cc:3060] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:46.944590 14638 consensus_queue.cc:260] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:46.945848 14638 raft_consensus.cc:3060] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:46.949193 14714 raft_consensus.cc:2468] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 3.
I20260709 13:17:46.950129 14529 leader_election.cc:304] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c779a0db43c47378f25bcb2bbeb54e3, ac3d4d81e96f412caaff09b3861ef2ff; no voters: 
I20260709 13:17:46.950732 14903 raft_consensus.cc:2804] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:46.951203 14903 raft_consensus.cc:697] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 LEADER]: Becoming Leader. State: Replica: 8c779a0db43c47378f25bcb2bbeb54e3, State: Running, Role: LEADER
I20260709 13:17:46.951665 14638 raft_consensus.cc:2468] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c779a0db43c47378f25bcb2bbeb54e3 in term 3.
I20260709 13:17:46.952023 14903 consensus_queue.cc:237] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } }
I20260709 13:17:46.958923 14460 catalog_manager.cc:5697] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 reported cstate change: term changed from 2 to 3, leader changed from d3cd29370cc54e1099c022714674d225 (127.14.23.66) to 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65). New cstate: current_term: 3 leader_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:47.101084 14638 raft_consensus.cc:1275] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Refusing update from remote peer ac3d4d81e96f412caaff09b3861ef2ff: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:47.102432 14854 consensus_queue.cc:1048] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:47.113330 14562 raft_consensus.cc:1275] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Refusing update from remote peer ac3d4d81e96f412caaff09b3861ef2ff: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:47.114773 14854 consensus_queue.cc:1048] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:47.171361 14638 raft_consensus.cc:1275] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Refusing update from remote peer ac3d4d81e96f412caaff09b3861ef2ff: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:47.172576 14854 consensus_queue.cc:1048] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:47.181453 14562 raft_consensus.cc:1275] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Refusing update from remote peer ac3d4d81e96f412caaff09b3861ef2ff: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:47.183035 14910 consensus_queue.cc:1048] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c779a0db43c47378f25bcb2bbeb54e3" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 34713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:47.321026 14638 raft_consensus.cc:1275] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 4 FOLLOWER]: Refusing update from remote peer 8c779a0db43c47378f25bcb2bbeb54e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:47.322211 14903 consensus_queue.cc:1048] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:47.324862 14714 raft_consensus.cc:1275] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 4 FOLLOWER]: Refusing update from remote peer 8c779a0db43c47378f25bcb2bbeb54e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:47.326460 14904 consensus_queue.cc:1048] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:47.367686 14714 raft_consensus.cc:1275] T 9bfd1e022dbc48a7805a15dbf9324642 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Refusing update from remote peer 8c779a0db43c47378f25bcb2bbeb54e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:47.368880 14903 consensus_queue.cc:1048] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac3d4d81e96f412caaff09b3861ef2ff" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:47.378942 14638 raft_consensus.cc:1275] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Refusing update from remote peer 8c779a0db43c47378f25bcb2bbeb54e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:47.380389 14918 consensus_queue.cc:1048] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:47.407626 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:47.412846 14429 tablet_server.cc:179] TabletServer@127.14.23.65:0 shutting down...
I20260709 13:17:47.438565 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:47.439239 14429 tablet_replica.cc:333] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3: stopping tablet replica
I20260709 13:17:47.439939 14429 raft_consensus.cc:2243] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:17:47.440719 14429 raft_consensus.cc:2272] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.443585 14429 tablet_replica.cc:333] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3: stopping tablet replica
I20260709 13:17:47.444113 14429 raft_consensus.cc:2243] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:47.444546 14429 raft_consensus.cc:2272] T accbd816047c4054a6f506623e82f905 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.446568 14429 tablet_replica.cc:333] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3: stopping tablet replica
I20260709 13:17:47.447060 14429 raft_consensus.cc:2243] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:47.447521 14429 raft_consensus.cc:2272] T a36368cb545c46bd929582b28a3fece5 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.449222 14429 tablet_replica.cc:333] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3: stopping tablet replica
I20260709 13:17:47.449796 14429 raft_consensus.cc:2243] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:47.450361 14429 raft_consensus.cc:2272] T 2c99ec13aa4c4ff5af744e67c5164098 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.451269 14638 raft_consensus.cc:1275] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Refusing update from remote peer 8c779a0db43c47378f25bcb2bbeb54e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:47.452260 14429 tablet_replica.cc:333] T c246b0d9436a4b30819c7ce5f07fbd77 P 8c779a0db43c47378f25bcb2bbeb54e3: stopping tablet replica
I20260709 13:17:47.452482 14918 consensus_queue.cc:1048] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3cd29370cc54e1099c022714674d225" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:47.452780 14429 raft_consensus.cc:2243] T c246b0d9436a4b30819c7ce5f07fbd77 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:47.453233 14429 raft_consensus.cc:2272] T c246b0d9436a4b30819c7ce5f07fbd77 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.455225 14429 tablet_replica.cc:333] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3: stopping tablet replica
I20260709 13:17:47.455822 14429 raft_consensus.cc:2243] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:47.456261 14429 raft_consensus.cc:2272] T b42fa1c328df4d64818288ee87e30a46 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.458976 14429 tablet_replica.cc:333] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3: stopping tablet replica
I20260709 13:17:47.459668 14429 raft_consensus.cc:2243] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:47.460330 14429 raft_consensus.cc:2272] T 5c4c219a8c864700b9f22ff2a9f23e68 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.462941 14429 tablet_replica.cc:333] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3: stopping tablet replica
I20260709 13:17:47.463658 14429 raft_consensus.cc:2243] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:47.463769 14714 raft_consensus.cc:1275] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Refusing update from remote peer 8c779a0db43c47378f25bcb2bbeb54e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:47.464818 14429 raft_consensus.cc:2272] T 9bfd1e022dbc48a7805a15dbf9324642 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.467177 14429 tablet_replica.cc:333] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3: stopping tablet replica
I20260709 13:17:47.467760 14429 raft_consensus.cc:2243] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:47.468171 14429 raft_consensus.cc:2272] T b395b43c234641fc985583e8ec908250 P 8c779a0db43c47378f25bcb2bbeb54e3 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:47.558645 14604 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.23.65:34713: connect: Connection refused (error 111)
I20260709 13:17:47.561868 14429 tablet_server.cc:196] TabletServer@127.14.23.65:0 shutdown complete.
W20260709 13:17:47.562988 14604 consensus_peers.cc:597] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 -> Peer 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713): Couldn't send request to peer 8c779a0db43c47378f25bcb2bbeb54e3. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:34713: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:47.578840 14679 consensus_peers.cc:597] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff -> Peer 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713): Couldn't send request to peer 8c779a0db43c47378f25bcb2bbeb54e3. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:34713: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:47.585595 14429 tablet_server.cc:179] TabletServer@127.14.23.66:0 shutting down...
W20260709 13:17:47.591230 14679 consensus_peers.cc:597] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff -> Peer d3cd29370cc54e1099c022714674d225 (127.14.23.66:40643): Couldn't send request to peer d3cd29370cc54e1099c022714674d225. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:47.609898 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:47.610574 14429 tablet_replica.cc:333] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225: stopping tablet replica
I20260709 13:17:47.611109 14429 raft_consensus.cc:2243] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:47.611547 14429 raft_consensus.cc:2272] T accbd816047c4054a6f506623e82f905 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.613430 14429 tablet_replica.cc:333] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225: stopping tablet replica
I20260709 13:17:47.614018 14429 raft_consensus.cc:2243] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:47.614760 14429 raft_consensus.cc:2272] T 2c99ec13aa4c4ff5af744e67c5164098 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.616526 14429 tablet_replica.cc:333] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225: stopping tablet replica
I20260709 13:17:47.616995 14429 raft_consensus.cc:2243] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:47.617400 14429 raft_consensus.cc:2272] T c246b0d9436a4b30819c7ce5f07fbd77 P d3cd29370cc54e1099c022714674d225 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.618999 14429 tablet_replica.cc:333] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225: stopping tablet replica
I20260709 13:17:47.619475 14429 raft_consensus.cc:2243] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:47.619831 14429 raft_consensus.cc:2272] T b42fa1c328df4d64818288ee87e30a46 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.621728 14429 tablet_replica.cc:333] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225: stopping tablet replica
I20260709 13:17:47.622232 14429 raft_consensus.cc:2243] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:47.622606 14429 raft_consensus.cc:2272] T 5c4c219a8c864700b9f22ff2a9f23e68 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.624583 14429 tablet_replica.cc:333] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225: stopping tablet replica
I20260709 13:17:47.625063 14429 raft_consensus.cc:2243] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:47.625744 14429 raft_consensus.cc:2272] T a36368cb545c46bd929582b28a3fece5 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.627767 14429 tablet_replica.cc:333] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225: stopping tablet replica
I20260709 13:17:47.628293 14429 raft_consensus.cc:2243] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:47.629057 14429 raft_consensus.cc:2272] T b395b43c234641fc985583e8ec908250 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.630825 14429 tablet_replica.cc:333] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225: stopping tablet replica
I20260709 13:17:47.631275 14429 raft_consensus.cc:2243] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:47.631660 14429 raft_consensus.cc:2272] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P d3cd29370cc54e1099c022714674d225 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.633764 14429 tablet_replica.cc:333] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225: stopping tablet replica
I20260709 13:17:47.634219 14429 raft_consensus.cc:2243] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:47.634611 14429 raft_consensus.cc:2272] T 9bfd1e022dbc48a7805a15dbf9324642 P d3cd29370cc54e1099c022714674d225 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:47.641335 14679 consensus_peers.cc:597] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff -> Peer 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713): Couldn't send request to peer 8c779a0db43c47378f25bcb2bbeb54e3. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:34713: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:47.664531 14429 tablet_server.cc:196] TabletServer@127.14.23.66:0 shutdown complete.
W20260709 13:17:47.682173 14679 consensus_peers.cc:597] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff -> Peer 8c779a0db43c47378f25bcb2bbeb54e3 (127.14.23.65:34713): Couldn't send request to peer 8c779a0db43c47378f25bcb2bbeb54e3. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:34713: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:47.682998 14429 tablet_server.cc:179] TabletServer@127.14.23.67:0 shutting down...
I20260709 13:17:47.705145 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:47.705722 14429 tablet_replica.cc:333] T 2c99ec13aa4c4ff5af744e67c5164098 P ac3d4d81e96f412caaff09b3861ef2ff: stopping tablet replica
I20260709 13:17:47.706327 14429 raft_consensus.cc:2243] T 2c99ec13aa4c4ff5af744e67c5164098 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:47.706785 14429 raft_consensus.cc:2272] T 2c99ec13aa4c4ff5af744e67c5164098 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.708422 14429 tablet_replica.cc:333] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff: stopping tablet replica
I20260709 13:17:47.708889 14429 raft_consensus.cc:2243] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:47.709594 14429 raft_consensus.cc:2272] T c246b0d9436a4b30819c7ce5f07fbd77 P ac3d4d81e96f412caaff09b3861ef2ff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.711262 14429 tablet_replica.cc:333] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff: stopping tablet replica
I20260709 13:17:47.711767 14429 raft_consensus.cc:2243] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:47.712441 14429 raft_consensus.cc:2272] T accbd816047c4054a6f506623e82f905 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.714390 14429 tablet_replica.cc:333] T 9bfd1e022dbc48a7805a15dbf9324642 P ac3d4d81e96f412caaff09b3861ef2ff: stopping tablet replica
I20260709 13:17:47.714879 14429 raft_consensus.cc:2243] T 9bfd1e022dbc48a7805a15dbf9324642 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:47.715297 14429 raft_consensus.cc:2272] T 9bfd1e022dbc48a7805a15dbf9324642 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.717682 14429 tablet_replica.cc:333] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff: stopping tablet replica
I20260709 13:17:47.718195 14429 raft_consensus.cc:2243] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:47.718616 14429 raft_consensus.cc:2272] T a36368cb545c46bd929582b28a3fece5 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.720204 14429 tablet_replica.cc:333] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff: stopping tablet replica
I20260709 13:17:47.720646 14429 raft_consensus.cc:2243] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:47.721037 14429 raft_consensus.cc:2272] T b395b43c234641fc985583e8ec908250 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.722724 14429 tablet_replica.cc:333] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff: stopping tablet replica
I20260709 13:17:47.723158 14429 raft_consensus.cc:2243] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:47.723534 14429 raft_consensus.cc:2272] T 5d1ea631a1bb4cf8a4f25e155142c0d3 P ac3d4d81e96f412caaff09b3861ef2ff [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.725417 14429 tablet_replica.cc:333] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff: stopping tablet replica
I20260709 13:17:47.725870 14429 raft_consensus.cc:2243] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:47.726675 14429 raft_consensus.cc:2272] T b42fa1c328df4d64818288ee87e30a46 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.728560 14429 tablet_replica.cc:333] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff: stopping tablet replica
I20260709 13:17:47.728976 14429 raft_consensus.cc:2243] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:47.729414 14429 raft_consensus.cc:2272] T 5c4c219a8c864700b9f22ff2a9f23e68 P ac3d4d81e96f412caaff09b3861ef2ff [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.754956 14429 tablet_server.cc:196] TabletServer@127.14.23.67:0 shutdown complete.
I20260709 13:17:47.770841 14429 master.cc:562] Master@127.14.23.126:34187 shutting down...
I20260709 13:17:47.940542 14429 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:47.941134 14429 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.941479 14429 tablet_replica.cc:333] T 00000000000000000000000000000000 P 0c9c7bb62d7a4ab4947c8a1a6f8c8748: stopping tablet replica
I20260709 13:17:47.961508 14429 master.cc:584] Master@127.14.23.126:34187 shutdown complete.
8c779a0db43c47378f25bcb2bbeb54e3  4
ac3d4d81e96f412caaff09b3861ef2ff  1
d3cd29370cc54e1099c022714674d225  4
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (12273 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:17:48.000969 14429 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.23.126:43765
I20260709 13:17:48.002230 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:48.008363 14926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:48.008780 14927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:48.009294 14929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:48.010614 14429 server_base.cc:1061] running on GCE node
I20260709 13:17:48.011466 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:48.011670 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:48.011842 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603068011824 us; error 0 us; skew 500 ppm
I20260709 13:17:48.012379 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:48.014989 14429 webserver.cc:533] Webserver started at http://127.14.23.126:45563/ using document root <none> and password file <none>
I20260709 13:17:48.015512 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:48.015697 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:48.015991 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:48.017175 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603055563165-14429-0/minicluster-data/master-0-root/instance:
uuid: "1b56247107e2429391d454af42a0f55f"
format_stamp: "Formatted at 2026-07-09 13:17:48 on dist-test-slave-twc1"
I20260709 13:17:48.021878 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:17:48.025514 14934 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:48.026322 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:17:48.026583 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603055563165-14429-0/minicluster-data/master-0-root
uuid: "1b56247107e2429391d454af42a0f55f"
format_stamp: "Formatted at 2026-07-09 13:17:48 on dist-test-slave-twc1"
I20260709 13:17:48.026830 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603055563165-14429-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603055563165-14429-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603055563165-14429-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:48.052793 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:48.054045 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:48.090430 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.126:43765
I20260709 13:17:48.090523 14985 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.126:43765 every 8 connection(s)
I20260709 13:17:48.094192 14986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.103554 14986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f: Bootstrap starting.
I20260709 13:17:48.108119 14986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.112279 14986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f: No bootstrap required, opened a new log
I20260709 13:17:48.114384 14986 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b56247107e2429391d454af42a0f55f" member_type: VOTER }
I20260709 13:17:48.114796 14986 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.115017 14986 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b56247107e2429391d454af42a0f55f, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.115600 14986 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b56247107e2429391d454af42a0f55f" member_type: VOTER }
I20260709 13:17:48.116053 14986 raft_consensus.cc:399] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:17:48.116261 14986 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:17:48.116492 14986 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:48.120499 14986 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b56247107e2429391d454af42a0f55f" member_type: VOTER }
I20260709 13:17:48.121019 14986 leader_election.cc:304] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b56247107e2429391d454af42a0f55f; no voters: 
I20260709 13:17:48.122298 14986 leader_election.cc:290] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:17:48.122669 14989 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:48.123970 14989 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f [term 1 LEADER]: Becoming Leader. State: Replica: 1b56247107e2429391d454af42a0f55f, State: Running, Role: LEADER
I20260709 13:17:48.124639 14989 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b56247107e2429391d454af42a0f55f" member_type: VOTER }
I20260709 13:17:48.125198 14986 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:17:48.129307 14991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b56247107e2429391d454af42a0f55f. Latest consensus state: current_term: 1 leader_uuid: "1b56247107e2429391d454af42a0f55f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b56247107e2429391d454af42a0f55f" member_type: VOTER } }
I20260709 13:17:48.130218 14991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:48.132160 14996 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:17:48.132118 14990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b56247107e2429391d454af42a0f55f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b56247107e2429391d454af42a0f55f" member_type: VOTER } }
I20260709 13:17:48.132916 14990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:48.137883 14996 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:17:48.140293 14429 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:17:48.146782 14996 catalog_manager.cc:1383] Generated new cluster ID: ca78e98e8a3747d4a802ad64e22346ff
I20260709 13:17:48.147032 14996 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:17:48.164482 14996 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:17:48.165688 14996 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:17:48.180558 14996 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f: Generated new TSK 0
I20260709 13:17:48.181124 14996 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:17:48.209110 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:48.214887 15007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:48.215864 15008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:48.218449 14429 server_base.cc:1061] running on GCE node
W20260709 13:17:48.218420 15010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:48.219496 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:48.219692 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:48.219812 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603068219801 us; error 0 us; skew 500 ppm
I20260709 13:17:48.220310 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:48.222700 14429 webserver.cc:533] Webserver started at http://127.14.23.65:40861/ using document root <none> and password file <none>
I20260709 13:17:48.223162 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:48.223312 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:48.223520 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:48.224520 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603055563165-14429-0/minicluster-data/ts-0-root/instance:
uuid: "4d322b2e1af04b48b88b588e012740d7"
format_stamp: "Formatted at 2026-07-09 13:17:48 on dist-test-slave-twc1"
I20260709 13:17:48.229091 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:17:48.232273 15015 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:48.232995 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:17:48.233279 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603055563165-14429-0/minicluster-data/ts-0-root
uuid: "4d322b2e1af04b48b88b588e012740d7"
format_stamp: "Formatted at 2026-07-09 13:17:48 on dist-test-slave-twc1"
I20260709 13:17:48.233564 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603055563165-14429-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603055563165-14429-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603055563165-14429-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:48.250203 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:48.251423 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:48.252848 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:48.255148 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:48.255338 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:48.255517 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:48.255661 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:48.295533 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.65:43629
I20260709 13:17:48.295625 15077 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.65:43629 every 8 connection(s)
I20260709 13:17:48.300176 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:48.307716 15082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:48.309619 15083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:48.313117 15085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:48.313597 14429 server_base.cc:1061] running on GCE node
I20260709 13:17:48.314888 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:48.315153 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:48.315346 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603068315330 us; error 0 us; skew 500 ppm
I20260709 13:17:48.316004 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:48.317586 15078 heartbeater.cc:344] Connected to a master server at 127.14.23.126:43765
I20260709 13:17:48.317981 15078 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:48.318811 14429 webserver.cc:533] Webserver started at http://127.14.23.66:38805/ using document root <none> and password file <none>
I20260709 13:17:48.318907 15078 heartbeater.cc:507] Master 127.14.23.126:43765 requested a full tablet report, sending...
I20260709 13:17:48.319481 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:48.319741 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:48.320044 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:48.321192 14951 ts_manager.cc:194] Registered new tserver with Master: 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629)
I20260709 13:17:48.321260 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603055563165-14429-0/minicluster-data/ts-1-root/instance:
uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
format_stamp: "Formatted at 2026-07-09 13:17:48 on dist-test-slave-twc1"
I20260709 13:17:48.322983 14951 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44994
I20260709 13:17:48.326130 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 13:17:48.329728 15090 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:48.330505 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:17:48.330802 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603055563165-14429-0/minicluster-data/ts-1-root
uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
format_stamp: "Formatted at 2026-07-09 13:17:48 on dist-test-slave-twc1"
I20260709 13:17:48.331063 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603055563165-14429-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603055563165-14429-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603055563165-14429-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:48.345813 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:48.347095 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:48.348567 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:48.350869 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:48.351074 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:48.351302 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:48.351464 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:48.389365 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.66:42825
I20260709 13:17:48.389465 15152 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.66:42825 every 8 connection(s)
I20260709 13:17:48.393841 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:48.400497 15156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:48.402290 15157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:48.403860 14429 server_base.cc:1061] running on GCE node
W20260709 13:17:48.404050 15159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:48.405133 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:48.405411 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:48.405618 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603068405598 us; error 0 us; skew 500 ppm
I20260709 13:17:48.406343 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:48.406669 15153 heartbeater.cc:344] Connected to a master server at 127.14.23.126:43765
I20260709 13:17:48.407012 15153 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:48.407874 15153 heartbeater.cc:507] Master 127.14.23.126:43765 requested a full tablet report, sending...
I20260709 13:17:48.408902 14429 webserver.cc:533] Webserver started at http://127.14.23.67:35445/ using document root <none> and password file <none>
I20260709 13:17:48.409510 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:48.409744 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:48.410091 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:48.410039 14951 ts_manager.cc:194] Registered new tserver with Master: 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825)
I20260709 13:17:48.411696 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603055563165-14429-0/minicluster-data/ts-2-root/instance:
uuid: "52976c323d7a4e97bf1415f69c8834ad"
format_stamp: "Formatted at 2026-07-09 13:17:48 on dist-test-slave-twc1"
I20260709 13:17:48.411882 14951 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45000
I20260709 13:17:48.416592 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:17:48.419976 15164 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:48.420707 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:17:48.421005 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603055563165-14429-0/minicluster-data/ts-2-root
uuid: "52976c323d7a4e97bf1415f69c8834ad"
format_stamp: "Formatted at 2026-07-09 13:17:48 on dist-test-slave-twc1"
I20260709 13:17:48.421277 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603055563165-14429-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603055563165-14429-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603055563165-14429-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:48.441648 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:48.442919 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:48.444412 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:48.446661 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:48.446874 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:48.447093 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:48.447239 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:48.485234 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.67:36917
I20260709 13:17:48.485321 15226 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.67:36917 every 8 connection(s)
I20260709 13:17:48.498319 15227 heartbeater.cc:344] Connected to a master server at 127.14.23.126:43765
I20260709 13:17:48.498679 15227 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:48.499430 15227 heartbeater.cc:507] Master 127.14.23.126:43765 requested a full tablet report, sending...
I20260709 13:17:48.501351 14951 ts_manager.cc:194] Registered new tserver with Master: 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:48.501463 14429 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013267302s
I20260709 13:17:48.503127 14951 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45016
I20260709 13:17:48.526979 14951 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45026:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:48.529814 14951 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:48.605556 15042 tablet_service.cc:1511] Processing CreateTablet for tablet 6baa6e039e11445488bb6c0086c79cf6 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:17:48.606673 15043 tablet_service.cc:1511] Processing CreateTablet for tablet eb35caf64cd84f77a7807ec904c0e340 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:17:48.607097 15042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6baa6e039e11445488bb6c0086c79cf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.608016 15043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb35caf64cd84f77a7807ec904c0e340. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.608484 15040 tablet_service.cc:1511] Processing CreateTablet for tablet db57138f7b4e42ac852be5436039207e (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:17:48.609761 15040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db57138f7b4e42ac852be5436039207e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.611099 15041 tablet_service.cc:1511] Processing CreateTablet for tablet 5ee7c7dfa5424b0bb683bb62badf9658 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:17:48.612963 15041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ee7c7dfa5424b0bb683bb62badf9658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.614849 15037 tablet_service.cc:1511] Processing CreateTablet for tablet fba2aa7361fd4aad8cb86213305039be (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:17:48.615224 15038 tablet_service.cc:1511] Processing CreateTablet for tablet 7809a2a83ca241a4ba6bc8015cc802c6 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:17:48.616207 15037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fba2aa7361fd4aad8cb86213305039be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.607081 15039 tablet_service.cc:1511] Processing CreateTablet for tablet 3b1769efa1d947e78a05c89546a0e917 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:17:48.617502 15039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b1769efa1d947e78a05c89546a0e917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.619066 15036 tablet_service.cc:1511] Processing CreateTablet for tablet c0060b39cf81428da22c632f18e594b6 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:17:48.620304 15036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0060b39cf81428da22c632f18e594b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.622371 15038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7809a2a83ca241a4ba6bc8015cc802c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.623119 15035 tablet_service.cc:1511] Processing CreateTablet for tablet f16bebdc1ec542578d9b1dfef1160556 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:17:48.624409 15035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f16bebdc1ec542578d9b1dfef1160556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.633078 15034 tablet_service.cc:1511] Processing CreateTablet for tablet 49fac936bede4d7491681c3d913a1fd0 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:17:48.634802 15034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49fac936bede4d7491681c3d913a1fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.672478 15188 tablet_service.cc:1511] Processing CreateTablet for tablet 3b1769efa1d947e78a05c89546a0e917 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:17:48.673892 15188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b1769efa1d947e78a05c89546a0e917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.673730 15192 tablet_service.cc:1511] Processing CreateTablet for tablet eb35caf64cd84f77a7807ec904c0e340 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:17:48.675083 15192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb35caf64cd84f77a7807ec904c0e340. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.677476 15242 tablet_bootstrap.cc:492] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7: Bootstrap starting.
I20260709 13:17:48.677958 15115 tablet_service.cc:1511] Processing CreateTablet for tablet db57138f7b4e42ac852be5436039207e (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:17:48.679356 15115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db57138f7b4e42ac852be5436039207e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.680648 15113 tablet_service.cc:1511] Processing CreateTablet for tablet 7809a2a83ca241a4ba6bc8015cc802c6 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:17:48.681986 15113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7809a2a83ca241a4ba6bc8015cc802c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.681845 15189 tablet_service.cc:1511] Processing CreateTablet for tablet db57138f7b4e42ac852be5436039207e (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:17:48.683184 15189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db57138f7b4e42ac852be5436039207e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.683046 15116 tablet_service.cc:1511] Processing CreateTablet for tablet 5ee7c7dfa5424b0bb683bb62badf9658 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:17:48.684417 15116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ee7c7dfa5424b0bb683bb62badf9658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.685380 15242 tablet_bootstrap.cc:654] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.686185 15112 tablet_service.cc:1511] Processing CreateTablet for tablet fba2aa7361fd4aad8cb86213305039be (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:17:48.686992 15187 tablet_service.cc:1511] Processing CreateTablet for tablet 7809a2a83ca241a4ba6bc8015cc802c6 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:17:48.687428 15112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fba2aa7361fd4aad8cb86213305039be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.688306 15187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7809a2a83ca241a4ba6bc8015cc802c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.688730 15118 tablet_service.cc:1511] Processing CreateTablet for tablet eb35caf64cd84f77a7807ec904c0e340 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:17:48.679009 15191 tablet_service.cc:1511] Processing CreateTablet for tablet 6baa6e039e11445488bb6c0086c79cf6 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:17:48.691084 15111 tablet_service.cc:1511] Processing CreateTablet for tablet c0060b39cf81428da22c632f18e594b6 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:17:48.691231 15185 tablet_service.cc:1511] Processing CreateTablet for tablet c0060b39cf81428da22c632f18e594b6 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:17:48.687547 15117 tablet_service.cc:1511] Processing CreateTablet for tablet 6baa6e039e11445488bb6c0086c79cf6 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:17:48.691186 15191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6baa6e039e11445488bb6c0086c79cf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.691620 15118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb35caf64cd84f77a7807ec904c0e340. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.680684 15190 tablet_service.cc:1511] Processing CreateTablet for tablet 5ee7c7dfa5424b0bb683bb62badf9658 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:17:48.694427 15184 tablet_service.cc:1511] Processing CreateTablet for tablet f16bebdc1ec542578d9b1dfef1160556 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:17:48.695391 15190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ee7c7dfa5424b0bb683bb62badf9658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.679492 15114 tablet_service.cc:1511] Processing CreateTablet for tablet 3b1769efa1d947e78a05c89546a0e917 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:17:48.696576 15186 tablet_service.cc:1511] Processing CreateTablet for tablet fba2aa7361fd4aad8cb86213305039be (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:17:48.694451 15117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6baa6e039e11445488bb6c0086c79cf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.700001 15186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fba2aa7361fd4aad8cb86213305039be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.700577 15111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0060b39cf81428da22c632f18e594b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.699628 15110 tablet_service.cc:1511] Processing CreateTablet for tablet f16bebdc1ec542578d9b1dfef1160556 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:17:48.701771 15114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b1769efa1d947e78a05c89546a0e917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.701848 15184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f16bebdc1ec542578d9b1dfef1160556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.703323 15110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f16bebdc1ec542578d9b1dfef1160556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.704509 15109 tablet_service.cc:1511] Processing CreateTablet for tablet 49fac936bede4d7491681c3d913a1fd0 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:17:48.706036 15109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49fac936bede4d7491681c3d913a1fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.704540 15185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0060b39cf81428da22c632f18e594b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.736922 15183 tablet_service.cc:1511] Processing CreateTablet for tablet 49fac936bede4d7491681c3d913a1fd0 (DEFAULT_TABLE table=test-workload [id=bee4066a38794dbfa149f9c44804278c]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:17:48.738534 15183 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49fac936bede4d7491681c3d913a1fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.824052 15244 tablet_bootstrap.cc:492] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3: Bootstrap starting.
I20260709 13:17:48.827911 15242 tablet_bootstrap.cc:492] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7: No bootstrap required, opened a new log
I20260709 13:17:48.834282 15244 tablet_bootstrap.cc:654] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.836702 15242 ts_tablet_manager.cc:1403] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7: Time spent bootstrapping tablet: real 0.162s	user 0.054s	sys 0.088s
I20260709 13:17:48.838595 15245 tablet_bootstrap.cc:492] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad: Bootstrap starting.
I20260709 13:17:48.839911 15242 raft_consensus.cc:359] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:48.840567 15242 raft_consensus.cc:385] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.840880 15242 raft_consensus.cc:740] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.841444 15242 consensus_queue.cc:260] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:48.843605 15244 tablet_bootstrap.cc:492] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3: No bootstrap required, opened a new log
I20260709 13:17:48.843648 15078 heartbeater.cc:499] Master 127.14.23.126:43765 was elected leader, sending a full tablet report...
I20260709 13:17:48.843966 15242 ts_tablet_manager.cc:1434] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:17:48.844441 15244 ts_tablet_manager.cc:1403] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.000s
I20260709 13:17:48.844940 15245 tablet_bootstrap.cc:654] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.845099 15242 tablet_bootstrap.cc:492] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7: Bootstrap starting.
I20260709 13:17:48.847326 15244 raft_consensus.cc:359] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:48.848047 15244 raft_consensus.cc:385] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.848372 15244 raft_consensus.cc:740] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11b6d8a2d2034e7d8e53e9df74226ac3, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.849054 15244 consensus_queue.cc:260] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:48.851749 15242 tablet_bootstrap.cc:654] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.860028 15245 tablet_bootstrap.cc:492] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad: No bootstrap required, opened a new log
I20260709 13:17:48.860515 15245 ts_tablet_manager.cc:1403] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad: Time spent bootstrapping tablet: real 0.022s	user 0.003s	sys 0.012s
I20260709 13:17:48.863354 15245 raft_consensus.cc:359] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:48.864068 15245 raft_consensus.cc:385] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.864375 15245 raft_consensus.cc:740] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52976c323d7a4e97bf1415f69c8834ad, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.865342 15245 consensus_queue.cc:260] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:48.874577 15245 ts_tablet_manager.cc:1434] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad: Time spent starting tablet: real 0.014s	user 0.013s	sys 0.002s
I20260709 13:17:48.875268 15227 heartbeater.cc:499] Master 127.14.23.126:43765 was elected leader, sending a full tablet report...
I20260709 13:17:48.876521 15153 heartbeater.cc:499] Master 127.14.23.126:43765 was elected leader, sending a full tablet report...
I20260709 13:17:48.876906 15244 ts_tablet_manager.cc:1434] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3: Time spent starting tablet: real 0.032s	user 0.012s	sys 0.020s
I20260709 13:17:48.877666 15247 raft_consensus.cc:493] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.877898 15244 tablet_bootstrap.cc:492] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3: Bootstrap starting.
I20260709 13:17:48.878139 15247 raft_consensus.cc:515] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:48.881875 15247 leader_election.cc:290] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:48.884550 15244 tablet_bootstrap.cc:654] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.882548 15245 tablet_bootstrap.cc:492] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad: Bootstrap starting.
I20260709 13:17:48.891076 15242 tablet_bootstrap.cc:492] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7: No bootstrap required, opened a new log
I20260709 13:17:48.892040 15242 ts_tablet_manager.cc:1403] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7: Time spent bootstrapping tablet: real 0.047s	user 0.032s	sys 0.000s
I20260709 13:17:48.895608 15249 raft_consensus.cc:493] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.896200 15249 raft_consensus.cc:515] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:48.898648 15245 tablet_bootstrap.cc:654] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.904454 15249 leader_election.cc:290] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:48.906596 15242 raft_consensus.cc:359] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:48.907451 15242 raft_consensus.cc:385] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.907816 15242 raft_consensus.cc:740] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.908735 15242 consensus_queue.cc:260] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:48.913105 15242 ts_tablet_manager.cc:1434] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.001s
I20260709 13:17:48.914181 15242 tablet_bootstrap.cc:492] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7: Bootstrap starting.
I20260709 13:17:48.918402 15128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db57138f7b4e42ac852be5436039207e" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" is_pre_election: true
I20260709 13:17:48.920756 15242 tablet_bootstrap.cc:654] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.924149 15202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db57138f7b4e42ac852be5436039207e" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52976c323d7a4e97bf1415f69c8834ad" is_pre_election: true
W20260709 13:17:48.925450 15018 leader_election.cc:343] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:48.928201 15019 leader_election.cc:343] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:48.930138 15053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b1769efa1d947e78a05c89546a0e917" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d322b2e1af04b48b88b588e012740d7" is_pre_election: true
I20260709 13:17:48.930506 15019 leader_election.cc:304] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4d322b2e1af04b48b88b588e012740d7; no voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 52976c323d7a4e97bf1415f69c8834ad
W20260709 13:17:48.931756 15094 leader_election.cc:343] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:48.932150 15202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b1769efa1d947e78a05c89546a0e917" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52976c323d7a4e97bf1415f69c8834ad" is_pre_election: true
I20260709 13:17:48.932844 15247 raft_consensus.cc:2749] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:17:48.935593 15094 leader_election.cc:343] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:48.936050 15094 leader_election.cc:304] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3; no voters: 4d322b2e1af04b48b88b588e012740d7, 52976c323d7a4e97bf1415f69c8834ad
I20260709 13:17:48.937139 15249 raft_consensus.cc:2749] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:48.940447 15244 tablet_bootstrap.cc:492] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3: No bootstrap required, opened a new log
I20260709 13:17:48.941032 15244 ts_tablet_manager.cc:1403] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3: Time spent bootstrapping tablet: real 0.063s	user 0.015s	sys 0.014s
I20260709 13:17:48.943856 15244 raft_consensus.cc:359] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:48.944711 15244 raft_consensus.cc:385] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.945266 15244 raft_consensus.cc:740] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11b6d8a2d2034e7d8e53e9df74226ac3, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.945320 15245 tablet_bootstrap.cc:492] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad: No bootstrap required, opened a new log
I20260709 13:17:48.945956 15245 ts_tablet_manager.cc:1403] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad: Time spent bootstrapping tablet: real 0.064s	user 0.011s	sys 0.004s
I20260709 13:17:48.945997 15244 consensus_queue.cc:260] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:48.948235 15244 ts_tablet_manager.cc:1434] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.004s
I20260709 13:17:48.948918 15242 tablet_bootstrap.cc:492] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7: No bootstrap required, opened a new log
I20260709 13:17:48.949167 15244 tablet_bootstrap.cc:492] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3: Bootstrap starting.
I20260709 13:17:48.948784 15245 raft_consensus.cc:359] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:48.949560 15242 ts_tablet_manager.cc:1403] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7: Time spent bootstrapping tablet: real 0.036s	user 0.023s	sys 0.011s
I20260709 13:17:48.949546 15245 raft_consensus.cc:385] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.950078 15245 raft_consensus.cc:740] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52976c323d7a4e97bf1415f69c8834ad, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.950940 15245 consensus_queue.cc:260] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:48.952821 15242 raft_consensus.cc:359] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:48.953608 15242 raft_consensus.cc:385] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.953965 15242 raft_consensus.cc:740] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.954882 15242 consensus_queue.cc:260] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:48.955907 15244 tablet_bootstrap.cc:654] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.959676 15242 ts_tablet_manager.cc:1434] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:17:48.960753 15242 tablet_bootstrap.cc:492] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7: Bootstrap starting.
I20260709 13:17:48.965330 15245 ts_tablet_manager.cc:1434] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.001s
I20260709 13:17:48.967279 15242 tablet_bootstrap.cc:654] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.970643 15245 tablet_bootstrap.cc:492] T fba2aa7361fd4aad8cb86213305039be P 52976c323d7a4e97bf1415f69c8834ad: Bootstrap starting.
I20260709 13:17:48.976296 15242 tablet_bootstrap.cc:492] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7: No bootstrap required, opened a new log
I20260709 13:17:48.976742 15242 ts_tablet_manager.cc:1403] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.003s
I20260709 13:17:48.978227 15245 tablet_bootstrap.cc:654] T fba2aa7361fd4aad8cb86213305039be P 52976c323d7a4e97bf1415f69c8834ad: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.979234 15242 raft_consensus.cc:359] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:48.979771 15242 raft_consensus.cc:385] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.980005 15242 raft_consensus.cc:740] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.980662 15244 tablet_bootstrap.cc:492] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3: No bootstrap required, opened a new log
I20260709 13:17:48.980636 15242 consensus_queue.cc:260] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:48.981246 15244 ts_tablet_manager.cc:1403] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3: Time spent bootstrapping tablet: real 0.032s	user 0.019s	sys 0.011s
I20260709 13:17:48.982717 15242 ts_tablet_manager.cc:1434] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:17:48.983747 15242 tablet_bootstrap.cc:492] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7: Bootstrap starting.
I20260709 13:17:48.984550 15244 raft_consensus.cc:359] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:48.985363 15244 raft_consensus.cc:385] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.985688 15244 raft_consensus.cc:740] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11b6d8a2d2034e7d8e53e9df74226ac3, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.986636 15244 consensus_queue.cc:260] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:48.990492 15242 tablet_bootstrap.cc:654] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.993139 15244 ts_tablet_manager.cc:1434] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:17:48.994357 15244 tablet_bootstrap.cc:492] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3: Bootstrap starting.
I20260709 13:17:48.996039 15245 tablet_bootstrap.cc:492] T fba2aa7361fd4aad8cb86213305039be P 52976c323d7a4e97bf1415f69c8834ad: No bootstrap required, opened a new log
I20260709 13:17:48.996524 15245 ts_tablet_manager.cc:1403] T fba2aa7361fd4aad8cb86213305039be P 52976c323d7a4e97bf1415f69c8834ad: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.013s
I20260709 13:17:48.996888 15242 tablet_bootstrap.cc:492] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7: No bootstrap required, opened a new log
I20260709 13:17:48.997427 15242 ts_tablet_manager.cc:1403] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.001s
I20260709 13:17:48.999593 15245 raft_consensus.cc:359] T fba2aa7361fd4aad8cb86213305039be P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.000322 15245 raft_consensus.cc:385] T fba2aa7361fd4aad8cb86213305039be P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.000604 15245 raft_consensus.cc:740] T fba2aa7361fd4aad8cb86213305039be P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52976c323d7a4e97bf1415f69c8834ad, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.000873 15244 tablet_bootstrap.cc:654] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.001415 15245 consensus_queue.cc:260] T fba2aa7361fd4aad8cb86213305039be P 52976c323d7a4e97bf1415f69c8834ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.003896 15242 raft_consensus.cc:359] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.004791 15242 raft_consensus.cc:385] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.005124 15242 raft_consensus.cc:740] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.005338 15245 ts_tablet_manager.cc:1434] T fba2aa7361fd4aad8cb86213305039be P 52976c323d7a4e97bf1415f69c8834ad: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:17:49.006301 15245 tablet_bootstrap.cc:492] T c0060b39cf81428da22c632f18e594b6 P 52976c323d7a4e97bf1415f69c8834ad: Bootstrap starting.
I20260709 13:17:49.005903 15242 consensus_queue.cc:260] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.007688 15244 tablet_bootstrap.cc:492] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3: No bootstrap required, opened a new log
I20260709 13:17:49.008224 15244 ts_tablet_manager.cc:1403] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.001s
I20260709 13:17:49.010004 15242 ts_tablet_manager.cc:1434] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.003s
I20260709 13:17:49.011051 15242 tablet_bootstrap.cc:492] T 49fac936bede4d7491681c3d913a1fd0 P 4d322b2e1af04b48b88b588e012740d7: Bootstrap starting.
I20260709 13:17:49.011353 15244 raft_consensus.cc:359] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.012140 15244 raft_consensus.cc:385] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.012497 15244 raft_consensus.cc:740] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11b6d8a2d2034e7d8e53e9df74226ac3, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.013279 15244 consensus_queue.cc:260] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.015560 15244 ts_tablet_manager.cc:1434] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:17:49.016716 15244 tablet_bootstrap.cc:492] T 49fac936bede4d7491681c3d913a1fd0 P 11b6d8a2d2034e7d8e53e9df74226ac3: Bootstrap starting.
I20260709 13:17:49.017884 15242 tablet_bootstrap.cc:654] T 49fac936bede4d7491681c3d913a1fd0 P 4d322b2e1af04b48b88b588e012740d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.020395 15245 tablet_bootstrap.cc:654] T c0060b39cf81428da22c632f18e594b6 P 52976c323d7a4e97bf1415f69c8834ad: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.023517 15244 tablet_bootstrap.cc:654] T 49fac936bede4d7491681c3d913a1fd0 P 11b6d8a2d2034e7d8e53e9df74226ac3: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.033082 15242 tablet_bootstrap.cc:492] T 49fac936bede4d7491681c3d913a1fd0 P 4d322b2e1af04b48b88b588e012740d7: No bootstrap required, opened a new log
I20260709 13:17:49.033588 15242 ts_tablet_manager.cc:1403] T 49fac936bede4d7491681c3d913a1fd0 P 4d322b2e1af04b48b88b588e012740d7: Time spent bootstrapping tablet: real 0.023s	user 0.007s	sys 0.009s
I20260709 13:17:49.036324 15242 raft_consensus.cc:359] T 49fac936bede4d7491681c3d913a1fd0 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.036900 15242 raft_consensus.cc:385] T 49fac936bede4d7491681c3d913a1fd0 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.037132 15242 raft_consensus.cc:740] T 49fac936bede4d7491681c3d913a1fd0 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.037788 15242 consensus_queue.cc:260] T 49fac936bede4d7491681c3d913a1fd0 P 4d322b2e1af04b48b88b588e012740d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.039975 15242 ts_tablet_manager.cc:1434] T 49fac936bede4d7491681c3d913a1fd0 P 4d322b2e1af04b48b88b588e012740d7: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 13:17:49.040966 15242 tablet_bootstrap.cc:492] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7: Bootstrap starting.
I20260709 13:17:49.044669 15245 tablet_bootstrap.cc:492] T c0060b39cf81428da22c632f18e594b6 P 52976c323d7a4e97bf1415f69c8834ad: No bootstrap required, opened a new log
I20260709 13:17:49.045197 15245 ts_tablet_manager.cc:1403] T c0060b39cf81428da22c632f18e594b6 P 52976c323d7a4e97bf1415f69c8834ad: Time spent bootstrapping tablet: real 0.039s	user 0.009s	sys 0.019s
I20260709 13:17:49.045727 15244 tablet_bootstrap.cc:492] T 49fac936bede4d7491681c3d913a1fd0 P 11b6d8a2d2034e7d8e53e9df74226ac3: No bootstrap required, opened a new log
I20260709 13:17:49.046270 15244 ts_tablet_manager.cc:1403] T 49fac936bede4d7491681c3d913a1fd0 P 11b6d8a2d2034e7d8e53e9df74226ac3: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.019s
I20260709 13:17:49.048178 15245 raft_consensus.cc:359] T c0060b39cf81428da22c632f18e594b6 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.048936 15245 raft_consensus.cc:385] T c0060b39cf81428da22c632f18e594b6 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.049324 15245 raft_consensus.cc:740] T c0060b39cf81428da22c632f18e594b6 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52976c323d7a4e97bf1415f69c8834ad, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.048974 15244 raft_consensus.cc:359] T 49fac936bede4d7491681c3d913a1fd0 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.049712 15242 tablet_bootstrap.cc:654] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.049739 15244 raft_consensus.cc:385] T 49fac936bede4d7491681c3d913a1fd0 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.050279 15244 raft_consensus.cc:740] T 49fac936bede4d7491681c3d913a1fd0 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11b6d8a2d2034e7d8e53e9df74226ac3, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.051010 15244 consensus_queue.cc:260] T 49fac936bede4d7491681c3d913a1fd0 P 11b6d8a2d2034e7d8e53e9df74226ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.052517 15245 consensus_queue.cc:260] T c0060b39cf81428da22c632f18e594b6 P 52976c323d7a4e97bf1415f69c8834ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.053232 15244 ts_tablet_manager.cc:1434] T 49fac936bede4d7491681c3d913a1fd0 P 11b6d8a2d2034e7d8e53e9df74226ac3: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:17:49.055097 15245 ts_tablet_manager.cc:1434] T c0060b39cf81428da22c632f18e594b6 P 52976c323d7a4e97bf1415f69c8834ad: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.002s
I20260709 13:17:49.056124 15245 tablet_bootstrap.cc:492] T 6baa6e039e11445488bb6c0086c79cf6 P 52976c323d7a4e97bf1415f69c8834ad: Bootstrap starting.
I20260709 13:17:49.058840 15244 tablet_bootstrap.cc:492] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3: Bootstrap starting.
I20260709 13:17:49.063879 15245 tablet_bootstrap.cc:654] T 6baa6e039e11445488bb6c0086c79cf6 P 52976c323d7a4e97bf1415f69c8834ad: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.065636 15244 tablet_bootstrap.cc:654] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.072711 15242 tablet_bootstrap.cc:492] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7: No bootstrap required, opened a new log
I20260709 13:17:49.073222 15242 ts_tablet_manager.cc:1403] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7: Time spent bootstrapping tablet: real 0.032s	user 0.020s	sys 0.006s
I20260709 13:17:49.073628 15244 tablet_bootstrap.cc:492] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3: No bootstrap required, opened a new log
I20260709 13:17:49.073657 15245 tablet_bootstrap.cc:492] T 6baa6e039e11445488bb6c0086c79cf6 P 52976c323d7a4e97bf1415f69c8834ad: No bootstrap required, opened a new log
I20260709 13:17:49.074326 15245 ts_tablet_manager.cc:1403] T 6baa6e039e11445488bb6c0086c79cf6 P 52976c323d7a4e97bf1415f69c8834ad: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.000s
I20260709 13:17:49.074326 15244 ts_tablet_manager.cc:1403] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.001s
I20260709 13:17:49.076089 15242 raft_consensus.cc:359] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.076812 15242 raft_consensus.cc:385] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.077044 15242 raft_consensus.cc:740] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.077416 15245 raft_consensus.cc:359] T 6baa6e039e11445488bb6c0086c79cf6 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.077657 15242 consensus_queue.cc:260] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.078181 15245 raft_consensus.cc:385] T 6baa6e039e11445488bb6c0086c79cf6 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.077947 15244 raft_consensus.cc:359] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.078588 15245 raft_consensus.cc:740] T 6baa6e039e11445488bb6c0086c79cf6 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52976c323d7a4e97bf1415f69c8834ad, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.078825 15244 raft_consensus.cc:385] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.079138 15244 raft_consensus.cc:740] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11b6d8a2d2034e7d8e53e9df74226ac3, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.079352 15245 consensus_queue.cc:260] T 6baa6e039e11445488bb6c0086c79cf6 P 52976c323d7a4e97bf1415f69c8834ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.079861 15244 consensus_queue.cc:260] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.081539 15245 ts_tablet_manager.cc:1434] T 6baa6e039e11445488bb6c0086c79cf6 P 52976c323d7a4e97bf1415f69c8834ad: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:17:49.082144 15247 raft_consensus.cc:493] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.082553 15245 tablet_bootstrap.cc:492] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad: Bootstrap starting.
I20260709 13:17:49.082602 15247 raft_consensus.cc:515] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.084796 15247 leader_election.cc:290] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:49.086136 15242 ts_tablet_manager.cc:1434] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:17:49.087172 15242 tablet_bootstrap.cc:492] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7: Bootstrap starting.
I20260709 13:17:49.087536 15202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba2aa7361fd4aad8cb86213305039be" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52976c323d7a4e97bf1415f69c8834ad" is_pre_election: true
I20260709 13:17:49.088147 15202 raft_consensus.cc:2468] T fba2aa7361fd4aad8cb86213305039be P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 0.
I20260709 13:17:49.089103 15244 ts_tablet_manager.cc:1434] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:17:49.089413 15245 tablet_bootstrap.cc:654] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.089596 15019 leader_election.cc:304] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d322b2e1af04b48b88b588e012740d7, 52976c323d7a4e97bf1415f69c8834ad; no voters: 
I20260709 13:17:49.090534 15247 raft_consensus.cc:2804] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:49.090907 15247 raft_consensus.cc:493] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.091285 15247 raft_consensus.cc:3060] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.093245 15242 tablet_bootstrap.cc:654] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.085947 15128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba2aa7361fd4aad8cb86213305039be" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" is_pre_election: true
I20260709 13:17:49.094357 15128 raft_consensus.cc:2468] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 0.
I20260709 13:17:49.097463 15244 tablet_bootstrap.cc:492] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3: Bootstrap starting.
I20260709 13:17:49.103422 15249 raft_consensus.cc:493] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.104036 15249 raft_consensus.cc:515] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.104735 15244 tablet_bootstrap.cc:654] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.110353 15249 leader_election.cc:290] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:49.112079 15053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7809a2a83ca241a4ba6bc8015cc802c6" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d322b2e1af04b48b88b588e012740d7" is_pre_election: true
I20260709 13:17:49.113517 15202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7809a2a83ca241a4ba6bc8015cc802c6" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52976c323d7a4e97bf1415f69c8834ad" is_pre_election: true
W20260709 13:17:49.115237 15094 leader_election.cc:343] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:49.116477 15094 leader_election.cc:343] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.116976 15094 leader_election.cc:304] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3; no voters: 4d322b2e1af04b48b88b588e012740d7, 52976c323d7a4e97bf1415f69c8834ad
I20260709 13:17:49.118199 15249 raft_consensus.cc:2749] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:49.121495 15247 raft_consensus.cc:515] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.124533 15128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba2aa7361fd4aad8cb86213305039be" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
I20260709 13:17:49.125200 15128 raft_consensus.cc:3060] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.126734 15244 tablet_bootstrap.cc:492] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3: No bootstrap required, opened a new log
I20260709 13:17:49.127303 15244 ts_tablet_manager.cc:1403] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3: Time spent bootstrapping tablet: real 0.030s	user 0.021s	sys 0.008s
I20260709 13:17:49.130529 15244 raft_consensus.cc:359] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.131340 15244 raft_consensus.cc:385] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.131649 15244 raft_consensus.cc:740] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11b6d8a2d2034e7d8e53e9df74226ac3, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.132570 15244 consensus_queue.cc:260] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
W20260709 13:17:49.135353 14999 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:49.135488 15202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba2aa7361fd4aad8cb86213305039be" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
I20260709 13:17:49.136242 15202 raft_consensus.cc:3060] T fba2aa7361fd4aad8cb86213305039be P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.137261 15244 ts_tablet_manager.cc:1434] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:17:49.138331 15244 tablet_bootstrap.cc:492] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3: Bootstrap starting.
I20260709 13:17:49.144822 15244 tablet_bootstrap.cc:654] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.149503 15202 raft_consensus.cc:2468] T fba2aa7361fd4aad8cb86213305039be P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 1.
I20260709 13:17:49.152141 15019 leader_election.cc:304] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d322b2e1af04b48b88b588e012740d7, 52976c323d7a4e97bf1415f69c8834ad; no voters: 
I20260709 13:17:49.150301 15128 raft_consensus.cc:2468] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 1.
I20260709 13:17:49.154167 15274 raft_consensus.cc:2804] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:49.155421 15251 raft_consensus.cc:493] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.155984 15251 raft_consensus.cc:515] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.156464 15244 tablet_bootstrap.cc:492] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3: No bootstrap required, opened a new log
I20260709 13:17:49.157047 15244 ts_tablet_manager.cc:1403] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.005s
I20260709 13:17:49.158864 15251 leader_election.cc:290] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825)
I20260709 13:17:49.150764 15247 leader_election.cc:290] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 election: Requested vote from peers 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:49.164279 15244 raft_consensus.cc:359] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.164994 15244 raft_consensus.cc:385] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.165268 15244 raft_consensus.cc:740] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11b6d8a2d2034e7d8e53e9df74226ac3, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.165405 15245 tablet_bootstrap.cc:492] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad: No bootstrap required, opened a new log
I20260709 13:17:49.165879 15245 ts_tablet_manager.cc:1403] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad: Time spent bootstrapping tablet: real 0.084s	user 0.027s	sys 0.030s
I20260709 13:17:49.166074 15244 consensus_queue.cc:260] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.168224 15244 ts_tablet_manager.cc:1434] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:17:49.169205 15244 tablet_bootstrap.cc:492] T 6baa6e039e11445488bb6c0086c79cf6 P 11b6d8a2d2034e7d8e53e9df74226ac3: Bootstrap starting.
I20260709 13:17:49.175791 15244 tablet_bootstrap.cc:654] T 6baa6e039e11445488bb6c0086c79cf6 P 11b6d8a2d2034e7d8e53e9df74226ac3: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.166970 15274 raft_consensus.cc:697] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 1 LEADER]: Becoming Leader. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Running, Role: LEADER
I20260709 13:17:49.184118 15274 consensus_queue.cc:237] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.185313 15245 raft_consensus.cc:359] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.186587 15245 raft_consensus.cc:385] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.186928 15245 raft_consensus.cc:740] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52976c323d7a4e97bf1415f69c8834ad, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.187747 15245 consensus_queue.cc:260] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.189162 15242 tablet_bootstrap.cc:492] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7: No bootstrap required, opened a new log
I20260709 13:17:49.189692 15242 ts_tablet_manager.cc:1403] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7: Time spent bootstrapping tablet: real 0.103s	user 0.041s	sys 0.033s
I20260709 13:17:49.190078 15245 ts_tablet_manager.cc:1434] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:17:49.191169 15245 tablet_bootstrap.cc:492] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad: Bootstrap starting.
I20260709 13:17:49.192516 15242 raft_consensus.cc:359] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.193872 15242 raft_consensus.cc:385] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.194674 15242 raft_consensus.cc:740] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.196213 15242 consensus_queue.cc:260] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.198096 15245 tablet_bootstrap.cc:654] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.198028 15128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee7c7dfa5424b0bb683bb62badf9658" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" is_pre_election: true
I20260709 13:17:49.198997 15128 raft_consensus.cc:2468] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52976c323d7a4e97bf1415f69c8834ad in term 0.
I20260709 13:17:49.200348 15167 leader_election.cc:304] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 52976c323d7a4e97bf1415f69c8834ad; no voters: 
I20260709 13:17:49.202143 15275 raft_consensus.cc:493] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.206612 15275 raft_consensus.cc:515] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.208907 15275 leader_election.cc:290] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:49.210533 15202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baa6e039e11445488bb6c0086c79cf6" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52976c323d7a4e97bf1415f69c8834ad" is_pre_election: true
I20260709 13:17:49.211216 15202 raft_consensus.cc:2468] T 6baa6e039e11445488bb6c0086c79cf6 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 0.
I20260709 13:17:49.212522 15019 leader_election.cc:304] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d322b2e1af04b48b88b588e012740d7, 52976c323d7a4e97bf1415f69c8834ad; no voters: 
I20260709 13:17:49.213404 15275 raft_consensus.cc:2804] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:49.213776 15275 raft_consensus.cc:493] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.214128 15275 raft_consensus.cc:3060] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.216619 15242 ts_tablet_manager.cc:1434] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7: Time spent starting tablet: real 0.027s	user 0.001s	sys 0.005s
I20260709 13:17:49.217741 15242 tablet_bootstrap.cc:492] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7: Bootstrap starting.
I20260709 13:17:49.215097 15251 raft_consensus.cc:2804] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:49.218544 15251 raft_consensus.cc:493] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.218935 15251 raft_consensus.cc:3060] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.214818 15128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baa6e039e11445488bb6c0086c79cf6" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" is_pre_election: true
I20260709 13:17:49.212889 14948 catalog_manager.cc:5697] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "4d322b2e1af04b48b88b588e012740d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:49.225028 15018 leader_election.cc:343] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.232491 15242 tablet_bootstrap.cc:654] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.240254 15053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee7c7dfa5424b0bb683bb62badf9658" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d322b2e1af04b48b88b588e012740d7" is_pre_election: true
I20260709 13:17:49.241549 15251 raft_consensus.cc:515] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.243234 15247 raft_consensus.cc:493] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:49.243522 15168 leader_election.cc:343] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.243966 15251 leader_election.cc:290] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 1 election: Requested vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825)
I20260709 13:17:49.252542 15244 tablet_bootstrap.cc:492] T 6baa6e039e11445488bb6c0086c79cf6 P 11b6d8a2d2034e7d8e53e9df74226ac3: No bootstrap required, opened a new log
I20260709 13:17:49.253545 15244 ts_tablet_manager.cc:1403] T 6baa6e039e11445488bb6c0086c79cf6 P 11b6d8a2d2034e7d8e53e9df74226ac3: Time spent bootstrapping tablet: real 0.085s	user 0.019s	sys 0.027s
I20260709 13:17:49.248522 15275 raft_consensus.cc:515] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.248208 15128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee7c7dfa5424b0bb683bb62badf9658" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
I20260709 13:17:49.254904 15128 raft_consensus.cc:3060] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.256425 15127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baa6e039e11445488bb6c0086c79cf6" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
I20260709 13:17:49.257555 15202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baa6e039e11445488bb6c0086c79cf6" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
W20260709 13:17:49.258139 15018 leader_election.cc:343] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.258453 15249 raft_consensus.cc:493] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.258240 15251 raft_consensus.cc:493] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.258168 15202 raft_consensus.cc:3060] T 6baa6e039e11445488bb6c0086c79cf6 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.258911 15249 raft_consensus.cc:515] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.243736 15247 raft_consensus.cc:515] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.261027 15249 leader_election.cc:290] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:49.261237 15247 leader_election.cc:290] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:49.259032 15053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee7c7dfa5424b0bb683bb62badf9658" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
W20260709 13:17:49.263616 15168 leader_election.cc:343] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.265115 15201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0060b39cf81428da22c632f18e594b6" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52976c323d7a4e97bf1415f69c8834ad" is_pre_election: true
I20260709 13:17:49.265808 15201 raft_consensus.cc:2468] T c0060b39cf81428da22c632f18e594b6 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11b6d8a2d2034e7d8e53e9df74226ac3 in term 0.
I20260709 13:17:49.266614 15275 leader_election.cc:290] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 election: Requested vote from peers 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:49.267565 15249 raft_consensus.cc:493] T 49fac936bede4d7491681c3d913a1fd0 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.268054 15249 raft_consensus.cc:515] T 49fac936bede4d7491681c3d913a1fd0 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.268432 15127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b1769efa1d947e78a05c89546a0e917" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" is_pre_election: true
I20260709 13:17:49.268934 15128 raft_consensus.cc:2468] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52976c323d7a4e97bf1415f69c8834ad in term 1.
I20260709 13:17:49.269133 15127 raft_consensus.cc:2468] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 0.
I20260709 13:17:49.270215 15167 leader_election.cc:304] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 52976c323d7a4e97bf1415f69c8834ad; no voters: 4d322b2e1af04b48b88b588e012740d7
I20260709 13:17:49.259053 15251 raft_consensus.cc:515] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.271320 15094 leader_election.cc:304] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 52976c323d7a4e97bf1415f69c8834ad; no voters: 
I20260709 13:17:49.272341 15251 leader_election.cc:290] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825)
I20260709 13:17:49.272786 15201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b1769efa1d947e78a05c89546a0e917" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52976c323d7a4e97bf1415f69c8834ad" is_pre_election: true
I20260709 13:17:49.268976 15053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0060b39cf81428da22c632f18e594b6" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d322b2e1af04b48b88b588e012740d7" is_pre_election: true
I20260709 13:17:49.273712 15245 tablet_bootstrap.cc:492] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad: No bootstrap required, opened a new log
I20260709 13:17:49.273480 15018 leader_election.cc:304] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 4d322b2e1af04b48b88b588e012740d7; no voters: 
I20260709 13:17:49.275537 15245 ts_tablet_manager.cc:1403] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad: Time spent bootstrapping tablet: real 0.085s	user 0.015s	sys 0.019s
I20260709 13:17:49.275612 15249 leader_election.cc:290] T 49fac936bede4d7491681c3d913a1fd0 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:49.276044 15247 raft_consensus.cc:2804] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:49.276166 15249 raft_consensus.cc:2804] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:49.276499 15247 raft_consensus.cc:493] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.276604 15249 raft_consensus.cc:493] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.276924 15247 raft_consensus.cc:3060] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.277029 15249 raft_consensus.cc:3060] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.278975 15245 raft_consensus.cc:359] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.280011 15245 raft_consensus.cc:385] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.274986 15242 tablet_bootstrap.cc:492] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7: No bootstrap required, opened a new log
I20260709 13:17:49.279280 15053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49fac936bede4d7491681c3d913a1fd0" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d322b2e1af04b48b88b588e012740d7" is_pre_election: true
I20260709 13:17:49.280825 15242 ts_tablet_manager.cc:1403] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7: Time spent bootstrapping tablet: real 0.063s	user 0.011s	sys 0.012s
I20260709 13:17:49.280906 15053 raft_consensus.cc:2468] T 49fac936bede4d7491681c3d913a1fd0 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52976c323d7a4e97bf1415f69c8834ad in term 0.
W20260709 13:17:49.280046 15019 leader_election.cc:343] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.273537 15128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49fac936bede4d7491681c3d913a1fd0" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" is_pre_election: true
I20260709 13:17:49.272890 15251 raft_consensus.cc:2804] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:49.282234 15128 raft_consensus.cc:2468] T 49fac936bede4d7491681c3d913a1fd0 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52976c323d7a4e97bf1415f69c8834ad in term 0.
I20260709 13:17:49.282706 15251 raft_consensus.cc:697] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 1 LEADER]: Becoming Leader. State: Replica: 52976c323d7a4e97bf1415f69c8834ad, State: Running, Role: LEADER
I20260709 13:17:49.283545 15251 consensus_queue.cc:237] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.283814 15242 raft_consensus.cc:359] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.284559 15242 raft_consensus.cc:385] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.284905 15242 raft_consensus.cc:740] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.285055 15247 raft_consensus.cc:515] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.285657 15242 consensus_queue.cc:260] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.286232 15167 leader_election.cc:304] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 52976c323d7a4e97bf1415f69c8834ad; no voters: 
I20260709 13:17:49.287457 15283 raft_consensus.cc:2804] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:49.287830 15283 raft_consensus.cc:493] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.287973 15128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b1769efa1d947e78a05c89546a0e917" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
I20260709 13:17:49.288064 15242 ts_tablet_manager.cc:1434] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:17:49.288563 15128 raft_consensus.cc:3060] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.288942 15245 raft_consensus.cc:740] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52976c323d7a4e97bf1415f69c8834ad, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.288213 15283 raft_consensus.cc:3060] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.274464 15202 raft_consensus.cc:2468] T 6baa6e039e11445488bb6c0086c79cf6 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 1.
I20260709 13:17:49.291076 15242 tablet_bootstrap.cc:492] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7: Bootstrap starting.
I20260709 13:17:49.291646 15247 leader_election.cc:290] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 election: Requested vote from peers 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:49.290005 15245 consensus_queue.cc:260] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.291095 15244 raft_consensus.cc:359] T 6baa6e039e11445488bb6c0086c79cf6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.292816 15244 raft_consensus.cc:385] T 6baa6e039e11445488bb6c0086c79cf6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.293040 15244 raft_consensus.cc:740] T 6baa6e039e11445488bb6c0086c79cf6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11b6d8a2d2034e7d8e53e9df74226ac3, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.293776 15244 consensus_queue.cc:260] T 6baa6e039e11445488bb6c0086c79cf6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.294260 15019 leader_election.cc:304] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4d322b2e1af04b48b88b588e012740d7, 52976c323d7a4e97bf1415f69c8834ad; no voters: 11b6d8a2d2034e7d8e53e9df74226ac3
I20260709 13:17:49.295871 15244 ts_tablet_manager.cc:1434] T 6baa6e039e11445488bb6c0086c79cf6 P 11b6d8a2d2034e7d8e53e9df74226ac3: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.000s
I20260709 13:17:49.298632 15275 raft_consensus.cc:2804] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:49.300442 15275 raft_consensus.cc:697] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 [term 1 LEADER]: Becoming Leader. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Running, Role: LEADER
I20260709 13:17:49.302232 15275 consensus_queue.cc:237] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.303804 15202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b1769efa1d947e78a05c89546a0e917" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
I20260709 13:17:49.306181 15244 tablet_bootstrap.cc:492] T eb35caf64cd84f77a7807ec904c0e340 P 11b6d8a2d2034e7d8e53e9df74226ac3: Bootstrap starting.
I20260709 13:17:49.312546 15245 ts_tablet_manager.cc:1434] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad: Time spent starting tablet: real 0.036s	user 0.013s	sys 0.004s
I20260709 13:17:49.313349 15249 raft_consensus.cc:515] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.315680 15249 leader_election.cc:290] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 1 election: Requested vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:49.317386 14948 catalog_manager.cc:5697] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67). New cstate: current_term: 1 leader_uuid: "52976c323d7a4e97bf1415f69c8834ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:49.318790 15244 tablet_bootstrap.cc:654] T eb35caf64cd84f77a7807ec904c0e340 P 11b6d8a2d2034e7d8e53e9df74226ac3: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.320755 15251 raft_consensus.cc:493] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.322886 15245 tablet_bootstrap.cc:492] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad: Bootstrap starting.
I20260709 13:17:49.322018 15247 raft_consensus.cc:493] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:49.327795 15094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:36917, user_credentials={real_user=slave}} blocked reactor thread for 56520us
I20260709 13:17:49.327499 15249 raft_consensus.cc:493] T 6baa6e039e11445488bb6c0086c79cf6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.336747 15249 raft_consensus.cc:515] T 6baa6e039e11445488bb6c0086c79cf6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.340040 15249 leader_election.cc:290] T 6baa6e039e11445488bb6c0086c79cf6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:49.323300 15283 raft_consensus.cc:515] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.345332 15245 tablet_bootstrap.cc:654] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.346094 15283 leader_election.cc:290] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 1 election: Requested vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825)
I20260709 13:17:49.346655 15128 raft_consensus.cc:2468] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 1.
I20260709 13:17:49.348110 15018 leader_election.cc:304] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 4d322b2e1af04b48b88b588e012740d7; no voters: 
I20260709 13:17:49.348734 15128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49fac936bede4d7491681c3d913a1fd0" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
I20260709 13:17:49.348987 15274 raft_consensus.cc:2804] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:49.349323 15128 raft_consensus.cc:3060] T 49fac936bede4d7491681c3d913a1fd0 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.349743 15242 tablet_bootstrap.cc:654] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.322069 15251 raft_consensus.cc:515] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.339066 15247 raft_consensus.cc:515] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
W20260709 13:17:49.355368 15019 leader_election.cc:343] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.352250 15251 leader_election.cc:290] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825)
I20260709 13:17:49.355638 15053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49fac936bede4d7491681c3d913a1fd0" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d322b2e1af04b48b88b588e012740d7" is_pre_election: true
I20260709 13:17:49.358620 15053 raft_consensus.cc:2468] T 49fac936bede4d7491681c3d913a1fd0 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11b6d8a2d2034e7d8e53e9df74226ac3 in term 0.
I20260709 13:17:49.358721 15127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7809a2a83ca241a4ba6bc8015cc802c6" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" is_pre_election: true
I20260709 13:17:49.358865 15052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0060b39cf81428da22c632f18e594b6" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
I20260709 13:17:49.359402 15127 raft_consensus.cc:2468] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52976c323d7a4e97bf1415f69c8834ad in term 0.
I20260709 13:17:49.360467 15202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49fac936bede4d7491681c3d913a1fd0" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52976c323d7a4e97bf1415f69c8834ad" is_pre_election: true
I20260709 13:17:49.360685 15247 leader_election.cc:290] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:49.361373 15202 raft_consensus.cc:2393] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11b6d8a2d2034e7d8e53e9df74226ac3 in current term 1: Already voted for candidate 52976c323d7a4e97bf1415f69c8834ad in this term.
I20260709 13:17:49.361424 15127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f16bebdc1ec542578d9b1dfef1160556" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" is_pre_election: true
I20260709 13:17:49.362208 15127 raft_consensus.cc:2468] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 0.
W20260709 13:17:49.362689 15094 leader_election.cc:343] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.362771 15167 leader_election.cc:304] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 52976c323d7a4e97bf1415f69c8834ad; no voters: 
I20260709 13:17:49.363389 15018 leader_election.cc:304] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 4d322b2e1af04b48b88b588e012740d7; no voters: 
I20260709 13:17:49.363608 15094 leader_election.cc:304] T 49fac936bede4d7491681c3d913a1fd0 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 4d322b2e1af04b48b88b588e012740d7; no voters: 
I20260709 13:17:49.364207 15251 raft_consensus.cc:2804] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:17:49.364413 15094 leader_election.cc:343] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.364784 15251 raft_consensus.cc:493] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.364806 15052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baa6e039e11445488bb6c0086c79cf6" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d322b2e1af04b48b88b588e012740d7" is_pre_election: true
I20260709 13:17:49.365543 15251 raft_consensus.cc:3060] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.361327 15247 raft_consensus.cc:493] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.366760 15274 raft_consensus.cc:697] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [term 1 LEADER]: Becoming Leader. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Running, Role: LEADER
I20260709 13:17:49.370092 15275 raft_consensus.cc:2804] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:49.378762 15274 consensus_queue.cc:237] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.381433 15128 raft_consensus.cc:2468] T 49fac936bede4d7491681c3d913a1fd0 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52976c323d7a4e97bf1415f69c8834ad in term 1.
I20260709 13:17:49.382823 15201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f16bebdc1ec542578d9b1dfef1160556" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52976c323d7a4e97bf1415f69c8834ad" is_pre_election: true
I20260709 13:17:49.387720 15249 raft_consensus.cc:2764] T 49fac936bede4d7491681c3d913a1fd0 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:17:49.389492 15167 leader_election.cc:304] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 52976c323d7a4e97bf1415f69c8834ad; no voters: 
I20260709 13:17:49.390622 15289 raft_consensus.cc:2804] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:49.390553 15202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0060b39cf81428da22c632f18e594b6" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
I20260709 13:17:49.391287 15289 raft_consensus.cc:697] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad [term 1 LEADER]: Becoming Leader. State: Replica: 52976c323d7a4e97bf1415f69c8834ad, State: Running, Role: LEADER
I20260709 13:17:49.391378 15202 raft_consensus.cc:3060] T c0060b39cf81428da22c632f18e594b6 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.367759 15247 raft_consensus.cc:515] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.392198 15289 consensus_queue.cc:237] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.380084 15275 raft_consensus.cc:493] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.392688 14948 catalog_manager.cc:5697] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "4d322b2e1af04b48b88b588e012740d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:49.395478 15275 raft_consensus.cc:3060] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.405104 15247 leader_election.cc:290] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:49.408317 15247 raft_consensus.cc:493] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.408972 15247 raft_consensus.cc:515] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.409888 15202 raft_consensus.cc:2468] T c0060b39cf81428da22c632f18e594b6 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11b6d8a2d2034e7d8e53e9df74226ac3 in term 1.
I20260709 13:17:49.413674 15052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49fac936bede4d7491681c3d913a1fd0" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
I20260709 13:17:49.414453 15052 raft_consensus.cc:3060] T 49fac936bede4d7491681c3d913a1fd0 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.422011 15201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7809a2a83ca241a4ba6bc8015cc802c6" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52976c323d7a4e97bf1415f69c8834ad" is_pre_election: true
W20260709 13:17:49.423502 15019 leader_election.cc:343] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.430289 15244 tablet_bootstrap.cc:492] T eb35caf64cd84f77a7807ec904c0e340 P 11b6d8a2d2034e7d8e53e9df74226ac3: No bootstrap required, opened a new log
I20260709 13:17:49.430728 15244 ts_tablet_manager.cc:1403] T eb35caf64cd84f77a7807ec904c0e340 P 11b6d8a2d2034e7d8e53e9df74226ac3: Time spent bootstrapping tablet: real 0.129s	user 0.014s	sys 0.003s
I20260709 13:17:49.424715 15128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7809a2a83ca241a4ba6bc8015cc802c6" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" is_pre_election: true
I20260709 13:17:49.416594 15094 leader_election.cc:304] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 52976c323d7a4e97bf1415f69c8834ad; no voters: 4d322b2e1af04b48b88b588e012740d7
I20260709 13:17:49.428676 15202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6baa6e039e11445488bb6c0086c79cf6" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52976c323d7a4e97bf1415f69c8834ad" is_pre_election: true
I20260709 13:17:49.431718 15247 leader_election.cc:290] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:49.426142 15251 raft_consensus.cc:515] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.432596 15053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7809a2a83ca241a4ba6bc8015cc802c6" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d322b2e1af04b48b88b588e012740d7" is_pre_election: true
I20260709 13:17:49.437645 15053 raft_consensus.cc:2468] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52976c323d7a4e97bf1415f69c8834ad in term 0.
I20260709 13:17:49.438514 15201 raft_consensus.cc:2393] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4d322b2e1af04b48b88b588e012740d7 in current term 1: Already voted for candidate 52976c323d7a4e97bf1415f69c8834ad in this term.
I20260709 13:17:49.436153 15249 raft_consensus.cc:2804] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:49.439385 15249 raft_consensus.cc:697] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 LEADER]: Becoming Leader. State: Replica: 11b6d8a2d2034e7d8e53e9df74226ac3, State: Running, Role: LEADER
I20260709 13:17:49.432832 15128 raft_consensus.cc:2468] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 0.
I20260709 13:17:49.440246 15249 consensus_queue.cc:237] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.440945 15018 leader_election.cc:304] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 4d322b2e1af04b48b88b588e012740d7; no voters: 
I20260709 13:17:49.442710 15293 raft_consensus.cc:2804] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:49.443094 15293 raft_consensus.cc:493] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.443416 15293 raft_consensus.cc:3060] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.444348 15244 raft_consensus.cc:359] T eb35caf64cd84f77a7807ec904c0e340 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.445439 15244 raft_consensus.cc:385] T eb35caf64cd84f77a7807ec904c0e340 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.445642 15244 raft_consensus.cc:740] T eb35caf64cd84f77a7807ec904c0e340 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11b6d8a2d2034e7d8e53e9df74226ac3, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.435626 15052 raft_consensus.cc:2468] T 49fac936bede4d7491681c3d913a1fd0 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52976c323d7a4e97bf1415f69c8834ad in term 1.
I20260709 13:17:49.448227 15285 raft_consensus.cc:493] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.448288 15275 raft_consensus.cc:515] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.448823 15285 raft_consensus.cc:515] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.447599 15242 tablet_bootstrap.cc:492] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7: No bootstrap required, opened a new log
I20260709 13:17:49.444283 15127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb35caf64cd84f77a7807ec904c0e340" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" is_pre_election: true
I20260709 13:17:49.447700 15201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb35caf64cd84f77a7807ec904c0e340" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52976c323d7a4e97bf1415f69c8834ad" is_pre_election: true
I20260709 13:17:49.451521 15285 leader_election.cc:290] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:49.452394 15242 ts_tablet_manager.cc:1403] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7: Time spent bootstrapping tablet: real 0.162s	user 0.039s	sys 0.027s
I20260709 13:17:49.453044 15244 consensus_queue.cc:260] T eb35caf64cd84f77a7807ec904c0e340 P 11b6d8a2d2034e7d8e53e9df74226ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.446067 15202 raft_consensus.cc:2393] T 6baa6e039e11445488bb6c0086c79cf6 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11b6d8a2d2034e7d8e53e9df74226ac3 in current term 1: Already voted for candidate 4d322b2e1af04b48b88b588e012740d7 in this term.
I20260709 13:17:49.455035 15127 raft_consensus.cc:2468] T eb35caf64cd84f77a7807ec904c0e340 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 0.
W20260709 13:17:49.457192 15019 leader_election.cc:343] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.458755 15052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db57138f7b4e42ac852be5436039207e" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d322b2e1af04b48b88b588e012740d7" is_pre_election: true
I20260709 13:17:49.458969 15251 leader_election.cc:290] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 1 election: Requested vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825)
I20260709 13:17:49.460037 15128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7809a2a83ca241a4ba6bc8015cc802c6" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
I20260709 13:17:49.460608 15128 raft_consensus.cc:3060] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.462752 15018 leader_election.cc:304] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 4d322b2e1af04b48b88b588e012740d7; no voters: 52976c323d7a4e97bf1415f69c8834ad
I20260709 13:17:49.462886 14951 catalog_manager.cc:5697] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67). New cstate: current_term: 1 leader_uuid: "52976c323d7a4e97bf1415f69c8834ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:49.463894 15247 raft_consensus.cc:2804] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:49.463712 15293 raft_consensus.cc:515] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.464270 15247 raft_consensus.cc:493] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.464608 15247 raft_consensus.cc:3060] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.465883 15293 leader_election.cc:290] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 election: Requested vote from peers 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:49.457109 15244 ts_tablet_manager.cc:1434] T eb35caf64cd84f77a7807ec904c0e340 P 11b6d8a2d2034e7d8e53e9df74226ac3: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:17:49.471819 15127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7809a2a83ca241a4ba6bc8015cc802c6" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
I20260709 13:17:49.472537 15275 leader_election.cc:290] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 election: Requested vote from peers 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:49.473471 15127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f16bebdc1ec542578d9b1dfef1160556" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
I20260709 13:17:49.476709 15127 raft_consensus.cc:3060] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.476542 14948 catalog_manager.cc:5697] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:49.477483 15245 tablet_bootstrap.cc:492] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad: No bootstrap required, opened a new log
I20260709 13:17:49.459486 15052 raft_consensus.cc:2468] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11b6d8a2d2034e7d8e53e9df74226ac3 in term 0.
I20260709 13:17:49.478341 15245 ts_tablet_manager.cc:1403] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad: Time spent bootstrapping tablet: real 0.156s	user 0.003s	sys 0.018s
I20260709 13:17:49.474440 15247 raft_consensus.cc:515] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.479389 15094 leader_election.cc:304] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 4d322b2e1af04b48b88b588e012740d7; no voters: 
I20260709 13:17:49.483254 15245 raft_consensus.cc:359] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.483969 15245 raft_consensus.cc:385] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.484293 15245 raft_consensus.cc:740] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52976c323d7a4e97bf1415f69c8834ad, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.484992 15052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7809a2a83ca241a4ba6bc8015cc802c6" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
I20260709 13:17:49.485498 15202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7809a2a83ca241a4ba6bc8015cc802c6" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
I20260709 13:17:49.485822 15201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f16bebdc1ec542578d9b1dfef1160556" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
I20260709 13:17:49.486356 15202 raft_consensus.cc:2393] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4d322b2e1af04b48b88b588e012740d7 in current term 1: Already voted for candidate 52976c323d7a4e97bf1415f69c8834ad in this term.
I20260709 13:17:49.485793 15052 raft_consensus.cc:2393] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 52976c323d7a4e97bf1415f69c8834ad in current term 1: Already voted for candidate 4d322b2e1af04b48b88b588e012740d7 in this term.
I20260709 13:17:49.487303 15200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db57138f7b4e42ac852be5436039207e" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52976c323d7a4e97bf1415f69c8834ad" is_pre_election: true
I20260709 13:17:49.487648 15247 leader_election.cc:290] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 election: Requested vote from peers 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:49.487901 15200 raft_consensus.cc:2468] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11b6d8a2d2034e7d8e53e9df74226ac3 in term 0.
I20260709 13:17:49.488430 15126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb35caf64cd84f77a7807ec904c0e340" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
I20260709 13:17:49.489015 15126 raft_consensus.cc:3060] T eb35caf64cd84f77a7807ec904c0e340 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:49.489288 15019 leader_election.cc:343] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.490043 15245 consensus_queue.cc:260] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.490473 15200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb35caf64cd84f77a7807ec904c0e340" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
I20260709 13:17:49.493055 15094 leader_election.cc:304] T 6baa6e039e11445488bb6c0086c79cf6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3; no voters: 4d322b2e1af04b48b88b588e012740d7, 52976c323d7a4e97bf1415f69c8834ad
I20260709 13:17:49.494015 15245 ts_tablet_manager.cc:1434] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.001s
I20260709 13:17:49.490011 15019 leader_election.cc:304] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4d322b2e1af04b48b88b588e012740d7; no voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 52976c323d7a4e97bf1415f69c8834ad
I20260709 13:17:49.493729 15249 raft_consensus.cc:2804] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:49.492146 15242 raft_consensus.cc:359] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.495455 15245 tablet_bootstrap.cc:492] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad: Bootstrap starting.
I20260709 13:17:49.495573 15242 raft_consensus.cc:385] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.495934 15242 raft_consensus.cc:740] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.496593 15242 consensus_queue.cc:260] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
W20260709 13:17:49.497830 15019 leader_election.cc:343] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.499171 15247 raft_consensus.cc:2749] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:17:49.499312 15285 raft_consensus.cc:3060] T 6baa6e039e11445488bb6c0086c79cf6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.498875 14951 catalog_manager.cc:5697] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "4d322b2e1af04b48b88b588e012740d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:49.495500 15249 raft_consensus.cc:493] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.500308 15249 raft_consensus.cc:3060] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.500782 15242 ts_tablet_manager.cc:1434] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7: Time spent starting tablet: real 0.048s	user 0.005s	sys 0.001s
I20260709 13:17:49.508635 15285 raft_consensus.cc:2749] T 6baa6e039e11445488bb6c0086c79cf6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:49.508698 15128 raft_consensus.cc:2468] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52976c323d7a4e97bf1415f69c8834ad in term 1.
I20260709 13:17:49.510869 15126 raft_consensus.cc:2468] T eb35caf64cd84f77a7807ec904c0e340 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 1.
I20260709 13:17:49.511299 15167 leader_election.cc:304] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 52976c323d7a4e97bf1415f69c8834ad; no voters: 4d322b2e1af04b48b88b588e012740d7
I20260709 13:17:49.512048 15251 raft_consensus.cc:2804] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:49.512180 15018 leader_election.cc:304] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 4d322b2e1af04b48b88b588e012740d7; no voters: 52976c323d7a4e97bf1415f69c8834ad
I20260709 13:17:49.512566 15251 raft_consensus.cc:697] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 1 LEADER]: Becoming Leader. State: Replica: 52976c323d7a4e97bf1415f69c8834ad, State: Running, Role: LEADER
I20260709 13:17:49.513403 15251 consensus_queue.cc:237] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.514546 15247 raft_consensus.cc:2804] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:49.515151 15245 tablet_bootstrap.cc:654] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.515072 15247 raft_consensus.cc:697] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 1 LEADER]: Becoming Leader. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Running, Role: LEADER
I20260709 13:17:49.516279 15247 consensus_queue.cc:237] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.521250 15127 raft_consensus.cc:2468] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 1.
I20260709 13:17:49.523092 15018 leader_election.cc:304] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 4d322b2e1af04b48b88b588e012740d7; no voters: 52976c323d7a4e97bf1415f69c8834ad
I20260709 13:17:49.524343 15275 raft_consensus.cc:2804] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:49.524868 15275 raft_consensus.cc:697] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [term 1 LEADER]: Becoming Leader. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Running, Role: LEADER
I20260709 13:17:49.525807 15275 consensus_queue.cc:237] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.530893 14948 catalog_manager.cc:5697] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67). New cstate: current_term: 1 leader_uuid: "52976c323d7a4e97bf1415f69c8834ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:49.524822 15249 raft_consensus.cc:515] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.541263 14951 catalog_manager.cc:5697] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "4d322b2e1af04b48b88b588e012740d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:49.543057 15249 leader_election.cc:290] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 1 election: Requested vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:49.545724 15200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db57138f7b4e42ac852be5436039207e" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
I20260709 13:17:49.547284 15200 raft_consensus.cc:3060] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.547489 15052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db57138f7b4e42ac852be5436039207e" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
I20260709 13:17:49.547832 15245 tablet_bootstrap.cc:492] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad: No bootstrap required, opened a new log
I20260709 13:17:49.548252 15245 ts_tablet_manager.cc:1403] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad: Time spent bootstrapping tablet: real 0.053s	user 0.018s	sys 0.008s
I20260709 13:17:49.549270 15052 raft_consensus.cc:3060] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.550570 15245 raft_consensus.cc:359] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.551193 15245 raft_consensus.cc:385] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.551442 15245 raft_consensus.cc:740] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52976c323d7a4e97bf1415f69c8834ad, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.552178 15245 consensus_queue.cc:260] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.553944 15200 raft_consensus.cc:2468] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11b6d8a2d2034e7d8e53e9df74226ac3 in term 1.
I20260709 13:17:49.554168 15245 ts_tablet_manager.cc:1434] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 13:17:49.555126 15245 tablet_bootstrap.cc:492] T f16bebdc1ec542578d9b1dfef1160556 P 52976c323d7a4e97bf1415f69c8834ad: Bootstrap starting.
I20260709 13:17:49.555143 15094 leader_election.cc:304] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 52976c323d7a4e97bf1415f69c8834ad; no voters: 
I20260709 13:17:49.556121 15249 raft_consensus.cc:2804] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:49.556809 15249 raft_consensus.cc:697] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 LEADER]: Becoming Leader. State: Replica: 11b6d8a2d2034e7d8e53e9df74226ac3, State: Running, Role: LEADER
I20260709 13:17:49.557606 15249 consensus_queue.cc:237] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.546772 15275 raft_consensus.cc:493] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.559970 14950 catalog_manager.cc:5697] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "4d322b2e1af04b48b88b588e012740d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:49.561224 15245 tablet_bootstrap.cc:654] T f16bebdc1ec542578d9b1dfef1160556 P 52976c323d7a4e97bf1415f69c8834ad: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.561074 15275 raft_consensus.cc:515] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.563843 15127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee7c7dfa5424b0bb683bb62badf9658" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" is_pre_election: true
I20260709 13:17:49.564605 15127 raft_consensus.cc:2393] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4d322b2e1af04b48b88b588e012740d7 in current term 1: Already voted for candidate 52976c323d7a4e97bf1415f69c8834ad in this term.
I20260709 13:17:49.568377 15200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee7c7dfa5424b0bb683bb62badf9658" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52976c323d7a4e97bf1415f69c8834ad" is_pre_election: true
I20260709 13:17:49.569836 15019 leader_election.cc:304] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4d322b2e1af04b48b88b588e012740d7; no voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 52976c323d7a4e97bf1415f69c8834ad
I20260709 13:17:49.570670 15247 raft_consensus.cc:3060] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.571247 15275 leader_election.cc:290] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:49.575470 15052 raft_consensus.cc:2468] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11b6d8a2d2034e7d8e53e9df74226ac3 in term 1.
I20260709 13:17:49.580588 15247 raft_consensus.cc:2749] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:49.582233 14950 catalog_manager.cc:5697] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:49.583581 15245 tablet_bootstrap.cc:492] T f16bebdc1ec542578d9b1dfef1160556 P 52976c323d7a4e97bf1415f69c8834ad: No bootstrap required, opened a new log
I20260709 13:17:49.583968 15245 ts_tablet_manager.cc:1403] T f16bebdc1ec542578d9b1dfef1160556 P 52976c323d7a4e97bf1415f69c8834ad: Time spent bootstrapping tablet: real 0.029s	user 0.022s	sys 0.004s
I20260709 13:17:49.586395 15245 raft_consensus.cc:359] T f16bebdc1ec542578d9b1dfef1160556 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.586993 15245 raft_consensus.cc:385] T f16bebdc1ec542578d9b1dfef1160556 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.587195 15245 raft_consensus.cc:740] T f16bebdc1ec542578d9b1dfef1160556 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52976c323d7a4e97bf1415f69c8834ad, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.587797 15245 consensus_queue.cc:260] T f16bebdc1ec542578d9b1dfef1160556 P 52976c323d7a4e97bf1415f69c8834ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.589540 15245 ts_tablet_manager.cc:1434] T f16bebdc1ec542578d9b1dfef1160556 P 52976c323d7a4e97bf1415f69c8834ad: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:17:49.603351 15247 consensus_queue.cc:1048] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:49.613881 15293 consensus_queue.cc:1048] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:49.672711 15251 raft_consensus.cc:493] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.673192 15251 raft_consensus.cc:515] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.673837 15200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ee7c7dfa5424b0bb683bb62badf9658"
dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
mode: GRACEFUL
new_leader_uuid: "4d322b2e1af04b48b88b588e012740d7"
 from {username='slave'} at 127.0.0.1:42682
I20260709 13:17:49.674332 15200 raft_consensus.cc:606] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 1 LEADER]: Received request to transfer leadership to TS 4d322b2e1af04b48b88b588e012740d7
I20260709 13:17:49.674950 15251 leader_election.cc:290] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825)
I20260709 13:17:49.675758 15052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb35caf64cd84f77a7807ec904c0e340" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d322b2e1af04b48b88b588e012740d7" is_pre_election: true
I20260709 13:17:49.676101 15127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb35caf64cd84f77a7807ec904c0e340" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" is_pre_election: true
I20260709 13:17:49.676805 15127 raft_consensus.cc:2393] T eb35caf64cd84f77a7807ec904c0e340 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52976c323d7a4e97bf1415f69c8834ad in current term 1: Already voted for candidate 4d322b2e1af04b48b88b588e012740d7 in this term.
I20260709 13:17:49.677712 15167 leader_election.cc:304] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52976c323d7a4e97bf1415f69c8834ad; no voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 4d322b2e1af04b48b88b588e012740d7
I20260709 13:17:49.678382 15251 raft_consensus.cc:3060] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.683213 15251 raft_consensus.cc:2749] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:49.687052 15127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db57138f7b4e42ac852be5436039207e"
dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
mode: GRACEFUL
new_leader_uuid: "4d322b2e1af04b48b88b588e012740d7"
 from {username='slave'} at 127.0.0.1:52616
I20260709 13:17:49.688037 15127 raft_consensus.cc:606] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 LEADER]: Received request to transfer leadership to TS 4d322b2e1af04b48b88b588e012740d7
I20260709 13:17:49.689714 15251 consensus_queue.cc:1048] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:49.697350 15200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7809a2a83ca241a4ba6bc8015cc802c6"
dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
mode: GRACEFUL
new_leader_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
 from {username='slave'} at 127.0.0.1:42682
I20260709 13:17:49.699666 15200 raft_consensus.cc:606] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 1 LEADER]: Received request to transfer leadership to TS 11b6d8a2d2034e7d8e53e9df74226ac3
I20260709 13:17:49.704007 15251 consensus_queue.cc:1048] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:49.715526 15283 raft_consensus.cc:993] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad: : Instructing follower 4d322b2e1af04b48b88b588e012740d7 to start an election
I20260709 13:17:49.716025 15289 raft_consensus.cc:1081] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 1 LEADER]: Signalling peer 4d322b2e1af04b48b88b588e012740d7 to start an election
I20260709 13:17:49.717624 15052 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ee7c7dfa5424b0bb683bb62badf9658"
dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
 from {username='slave'} at 127.0.0.1:52514
I20260709 13:17:49.718286 15052 raft_consensus.cc:493] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:49.718703 15052 raft_consensus.cc:3060] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:49.719316 15053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fba2aa7361fd4aad8cb86213305039be"
dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
mode: GRACEFUL
new_leader_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
 from {username='slave'} at 127.0.0.1:52524
I20260709 13:17:49.719755 15053 raft_consensus.cc:606] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 1 LEADER]: Received request to transfer leadership to TS 11b6d8a2d2034e7d8e53e9df74226ac3
I20260709 13:17:49.724035 15052 raft_consensus.cc:515] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.726629 15128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee7c7dfa5424b0bb683bb62badf9658" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
I20260709 13:17:49.727246 15128 raft_consensus.cc:3060] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:49.727404 15053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f16bebdc1ec542578d9b1dfef1160556"
dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
mode: GRACEFUL
new_leader_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
 from {username='slave'} at 127.0.0.1:52524
I20260709 13:17:49.727960 15200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee7c7dfa5424b0bb683bb62badf9658" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
I20260709 13:17:49.728065 15052 leader_election.cc:290] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 2 election: Requested vote from peers 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:49.728560 15200 raft_consensus.cc:3055] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:49.728881 15200 raft_consensus.cc:740] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52976c323d7a4e97bf1415f69c8834ad, State: Running, Role: LEADER
I20260709 13:17:49.729554 15200 consensus_queue.cc:260] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.730479 15200 raft_consensus.cc:3060] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:49.732604 15128 raft_consensus.cc:2468] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 2.
I20260709 13:17:49.733578 15018 leader_election.cc:304] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 4d322b2e1af04b48b88b588e012740d7; no voters: 
I20260709 13:17:49.727988 15053 raft_consensus.cc:606] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [term 1 LEADER]: Received request to transfer leadership to TS 11b6d8a2d2034e7d8e53e9df74226ac3
I20260709 13:17:49.734391 15293 raft_consensus.cc:2804] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:49.734836 15293 raft_consensus.cc:697] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 2 LEADER]: Becoming Leader. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Running, Role: LEADER
I20260709 13:17:49.735615 15293 consensus_queue.cc:237] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.736868 15200 raft_consensus.cc:2468] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 2.
I20260709 13:17:49.742168 14951 catalog_manager.cc:5697] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 reported cstate change: term changed from 1 to 2, leader changed from 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67) to 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65). New cstate: current_term: 2 leader_uuid: "4d322b2e1af04b48b88b588e012740d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:49.781770 15293 consensus_queue.cc:1048] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:49.791695 15274 consensus_queue.cc:1048] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:49.802487 15274 consensus_queue.cc:1048] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:49.812070 15200 raft_consensus.cc:3060] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.816653 15275 raft_consensus.cc:493] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.817166 15275 raft_consensus.cc:515] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.818998 15275 leader_election.cc:290] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:49.820225 15202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0060b39cf81428da22c632f18e594b6" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52976c323d7a4e97bf1415f69c8834ad" is_pre_election: true
I20260709 13:17:49.820509 15275 consensus_queue.cc:1048] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:49.821051 15202 raft_consensus.cc:2393] T c0060b39cf81428da22c632f18e594b6 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4d322b2e1af04b48b88b588e012740d7 in current term 1: Already voted for candidate 11b6d8a2d2034e7d8e53e9df74226ac3 in this term.
I20260709 13:17:49.822717 15128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0060b39cf81428da22c632f18e594b6" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" is_pre_election: true
I20260709 13:17:49.823617 15053 raft_consensus.cc:3060] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.824203 15018 leader_election.cc:304] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4d322b2e1af04b48b88b588e012740d7; no voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 52976c323d7a4e97bf1415f69c8834ad
I20260709 13:17:49.827888 15289 consensus_queue.cc:1048] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:49.833729 15274 raft_consensus.cc:2749] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:49.836710 15249 consensus_queue.cc:1048] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:49.841166 15289 consensus_queue.cc:1048] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:49.861007 15321 consensus_queue.cc:1048] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:49.939270 15289 consensus_queue.cc:1048] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:49.945727 15283 raft_consensus.cc:993] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad: : Instructing follower 11b6d8a2d2034e7d8e53e9df74226ac3 to start an election
I20260709 13:17:49.946358 15287 raft_consensus.cc:1081] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 1 LEADER]: Signalling peer 11b6d8a2d2034e7d8e53e9df74226ac3 to start an election
I20260709 13:17:49.948096 15128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7809a2a83ca241a4ba6bc8015cc802c6"
dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
 from {username='slave'} at 127.0.0.1:52610
I20260709 13:17:49.949291 15128 raft_consensus.cc:493] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:49.949729 15128 raft_consensus.cc:3060] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:49.954259 15289 consensus_queue.cc:1048] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:49.956624 15128 raft_consensus.cc:515] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.959957 15053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7809a2a83ca241a4ba6bc8015cc802c6" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
I20260709 13:17:49.962416 15128 leader_election.cc:290] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 2 election: Requested vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:49.963683 15202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7809a2a83ca241a4ba6bc8015cc802c6" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
I20260709 13:17:49.964293 15202 raft_consensus.cc:3055] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:49.964341 15275 consensus_queue.cc:1048] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:49.964715 15202 raft_consensus.cc:740] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52976c323d7a4e97bf1415f69c8834ad, State: Running, Role: LEADER
I20260709 13:17:49.965607 15202 consensus_queue.cc:260] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.968061 15202 raft_consensus.cc:3060] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:49.975176 15202 raft_consensus.cc:2468] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11b6d8a2d2034e7d8e53e9df74226ac3 in term 2.
I20260709 13:17:49.976070 15247 consensus_queue.cc:1048] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:49.978027 15094 leader_election.cc:304] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 52976c323d7a4e97bf1415f69c8834ad; no voters: 4d322b2e1af04b48b88b588e012740d7
I20260709 13:17:49.979121 15321 raft_consensus.cc:2804] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:49.979589 15321 raft_consensus.cc:697] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 LEADER]: Becoming Leader. State: Replica: 11b6d8a2d2034e7d8e53e9df74226ac3, State: Running, Role: LEADER
I20260709 13:17:49.980376 15321 consensus_queue.cc:237] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:49.988265 14950 catalog_manager.cc:5697] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 reported cstate change: term changed from 1 to 2, leader changed from 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67) to 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:50.011904 15289 raft_consensus.cc:493] T f16bebdc1ec542578d9b1dfef1160556 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.012409 15289 raft_consensus.cc:515] T f16bebdc1ec542578d9b1dfef1160556 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:50.014040 15289 leader_election.cc:290] T f16bebdc1ec542578d9b1dfef1160556 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825)
I20260709 13:17:50.014904 15053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f16bebdc1ec542578d9b1dfef1160556" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d322b2e1af04b48b88b588e012740d7" is_pre_election: true
I20260709 13:17:50.014995 15128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f16bebdc1ec542578d9b1dfef1160556" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" is_pre_election: true
I20260709 13:17:50.015614 15128 raft_consensus.cc:2393] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52976c323d7a4e97bf1415f69c8834ad in current term 1: Already voted for candidate 4d322b2e1af04b48b88b588e012740d7 in this term.
I20260709 13:17:50.016521 15167 leader_election.cc:304] T f16bebdc1ec542578d9b1dfef1160556 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52976c323d7a4e97bf1415f69c8834ad; no voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 4d322b2e1af04b48b88b588e012740d7
I20260709 13:17:50.017164 15289 raft_consensus.cc:3060] T f16bebdc1ec542578d9b1dfef1160556 P 52976c323d7a4e97bf1415f69c8834ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:50.021792 15289 raft_consensus.cc:2749] T f16bebdc1ec542578d9b1dfef1160556 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:50.024883 15275 raft_consensus.cc:993] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7: : Instructing follower 11b6d8a2d2034e7d8e53e9df74226ac3 to start an election
I20260709 13:17:50.025301 15247 raft_consensus.cc:1081] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 1 LEADER]: Signalling peer 11b6d8a2d2034e7d8e53e9df74226ac3 to start an election
I20260709 13:17:50.026775 15128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fba2aa7361fd4aad8cb86213305039be"
dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
 from {username='slave'} at 127.0.0.1:52594
I20260709 13:17:50.027248 15128 raft_consensus.cc:493] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:50.027619 15128 raft_consensus.cc:3060] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:50.032395 15128 raft_consensus.cc:515] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:50.034049 15128 leader_election.cc:290] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 2 election: Requested vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:50.034910 15053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba2aa7361fd4aad8cb86213305039be" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
I20260709 13:17:50.035151 15200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba2aa7361fd4aad8cb86213305039be" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
I20260709 13:17:50.035615 15053 raft_consensus.cc:3055] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:50.035640 15200 raft_consensus.cc:3060] T fba2aa7361fd4aad8cb86213305039be P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:50.036044 15053 raft_consensus.cc:740] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Running, Role: LEADER
I20260709 13:17:50.036702 15053 consensus_queue.cc:260] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:50.037580 15053 raft_consensus.cc:3060] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:50.040484 15200 raft_consensus.cc:2468] T fba2aa7361fd4aad8cb86213305039be P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11b6d8a2d2034e7d8e53e9df74226ac3 in term 2.
I20260709 13:17:50.041679 15094 leader_election.cc:304] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 52976c323d7a4e97bf1415f69c8834ad; no voters: 
I20260709 13:17:50.042248 15053 raft_consensus.cc:2468] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11b6d8a2d2034e7d8e53e9df74226ac3 in term 2.
I20260709 13:17:50.042513 15321 raft_consensus.cc:2804] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:50.042968 15321 raft_consensus.cc:697] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 LEADER]: Becoming Leader. State: Replica: 11b6d8a2d2034e7d8e53e9df74226ac3, State: Running, Role: LEADER
I20260709 13:17:50.043716 15321 consensus_queue.cc:237] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:50.050238 14950 catalog_manager.cc:5697] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 reported cstate change: term changed from 1 to 2, leader changed from 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65) to 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:50.052570 15321 consensus_queue.cc:1048] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:50.056160 15247 consensus_queue.cc:1048] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:50.062582 15321 raft_consensus.cc:993] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3: : Instructing follower 4d322b2e1af04b48b88b588e012740d7 to start an election
I20260709 13:17:50.063019 15321 raft_consensus.cc:1081] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 LEADER]: Signalling peer 4d322b2e1af04b48b88b588e012740d7 to start an election
I20260709 13:17:50.066521 15052 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "db57138f7b4e42ac852be5436039207e"
dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
 from {username='slave'} at 127.0.0.1:52500
I20260709 13:17:50.067158 15052 raft_consensus.cc:493] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:50.067574 15052 raft_consensus.cc:3060] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:50.077283 15321 consensus_queue.cc:1048] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:50.078079 15052 raft_consensus.cc:515] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:50.079082 15247 consensus_queue.cc:1048] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:50.080802 15053 raft_consensus.cc:1240] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Rejecting Update request from peer 11b6d8a2d2034e7d8e53e9df74226ac3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:50.081230 15202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db57138f7b4e42ac852be5436039207e" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
I20260709 13:17:50.081633 15052 leader_election.cc:290] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 2 election: Requested vote from peers 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:50.085572 15128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db57138f7b4e42ac852be5436039207e" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
I20260709 13:17:50.086325 15128 raft_consensus.cc:3055] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:50.086674 15128 raft_consensus.cc:740] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11b6d8a2d2034e7d8e53e9df74226ac3, State: Running, Role: LEADER
I20260709 13:17:50.087728 15128 consensus_queue.cc:260] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:50.088862 15128 raft_consensus.cc:3060] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:50.095685 15275 raft_consensus.cc:993] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7: : Instructing follower 11b6d8a2d2034e7d8e53e9df74226ac3 to start an election
I20260709 13:17:50.096199 15247 raft_consensus.cc:1081] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [term 1 LEADER]: Signalling peer 11b6d8a2d2034e7d8e53e9df74226ac3 to start an election
I20260709 13:17:50.097972 15127 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f16bebdc1ec542578d9b1dfef1160556"
dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
 from {username='slave'} at 127.0.0.1:52594
I20260709 13:17:50.098203 15128 raft_consensus.cc:2468] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 2.
I20260709 13:17:50.098685 15127 raft_consensus.cc:493] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:50.099124 15127 raft_consensus.cc:3060] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:50.099246 15018 leader_election.cc:304] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 4d322b2e1af04b48b88b588e012740d7; no voters: 52976c323d7a4e97bf1415f69c8834ad
I20260709 13:17:50.100011 15247 raft_consensus.cc:2804] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:50.100464 15247 raft_consensus.cc:697] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [term 2 LEADER]: Becoming Leader. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Running, Role: LEADER
I20260709 13:17:50.101114 15247 consensus_queue.cc:237] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:50.104071 15127 raft_consensus.cc:515] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:50.106815 15053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f16bebdc1ec542578d9b1dfef1160556" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
I20260709 13:17:50.107394 15053 raft_consensus.cc:3055] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:50.107612 15200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f16bebdc1ec542578d9b1dfef1160556" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
I20260709 13:17:50.107724 15053 raft_consensus.cc:740] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Running, Role: LEADER
I20260709 13:17:50.108211 15200 raft_consensus.cc:3060] T f16bebdc1ec542578d9b1dfef1160556 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:50.108639 15053 consensus_queue.cc:260] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:50.108745 14951 catalog_manager.cc:5697] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 reported cstate change: term changed from 1 to 2, leader changed from 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66) to 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65). New cstate: current_term: 2 leader_uuid: "4d322b2e1af04b48b88b588e012740d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:50.109704 15053 raft_consensus.cc:3060] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:50.112039 15127 leader_election.cc:290] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 2 election: Requested vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:50.115123 15200 raft_consensus.cc:2468] T f16bebdc1ec542578d9b1dfef1160556 P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11b6d8a2d2034e7d8e53e9df74226ac3 in term 2.
I20260709 13:17:50.116259 15094 leader_election.cc:304] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 52976c323d7a4e97bf1415f69c8834ad; no voters: 
I20260709 13:17:50.116477 15053 raft_consensus.cc:2468] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11b6d8a2d2034e7d8e53e9df74226ac3 in term 2.
I20260709 13:17:50.117132 15321 raft_consensus.cc:2804] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:50.117597 15321 raft_consensus.cc:697] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 LEADER]: Becoming Leader. State: Replica: 11b6d8a2d2034e7d8e53e9df74226ac3, State: Running, Role: LEADER
I20260709 13:17:50.118394 15321 consensus_queue.cc:237] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:50.124291 14951 catalog_manager.cc:5697] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3 reported cstate change: term changed from 1 to 2, leader changed from 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65) to 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:50.280927 15200 raft_consensus.cc:1275] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Refusing update from remote peer 4d322b2e1af04b48b88b588e012740d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:50.282147 15247 consensus_queue.cc:1048] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:50.289696 15127 raft_consensus.cc:1275] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Refusing update from remote peer 4d322b2e1af04b48b88b588e012740d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:50.291563 15247 consensus_queue.cc:1048] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:50.401067 15200 raft_consensus.cc:1275] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Refusing update from remote peer 11b6d8a2d2034e7d8e53e9df74226ac3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:50.402274 15321 consensus_queue.cc:1048] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:50.409744 15053 raft_consensus.cc:3060] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:50.416469 15053 raft_consensus.cc:1275] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Refusing update from remote peer 11b6d8a2d2034e7d8e53e9df74226ac3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:50.417686 15249 consensus_queue.cc:1048] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:50.523032 15200 raft_consensus.cc:1275] T fba2aa7361fd4aad8cb86213305039be P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Refusing update from remote peer 11b6d8a2d2034e7d8e53e9df74226ac3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:50.524449 15249 consensus_queue.cc:1048] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:50.531911 15053 raft_consensus.cc:1275] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Refusing update from remote peer 11b6d8a2d2034e7d8e53e9df74226ac3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:50.534426 15285 consensus_queue.cc:1048] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:50.534922 15200 raft_consensus.cc:1275] T f16bebdc1ec542578d9b1dfef1160556 P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Refusing update from remote peer 11b6d8a2d2034e7d8e53e9df74226ac3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:50.536949 15285 consensus_queue.cc:1048] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:50.548660 15053 raft_consensus.cc:1275] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Refusing update from remote peer 11b6d8a2d2034e7d8e53e9df74226ac3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:50.550522 15331 consensus_queue.cc:1048] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:50.635491 15127 raft_consensus.cc:1275] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Refusing update from remote peer 4d322b2e1af04b48b88b588e012740d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:50.636570 15247 consensus_queue.cc:1048] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:50.644583 15200 raft_consensus.cc:3060] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:50.652671 15200 raft_consensus.cc:1275] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Refusing update from remote peer 4d322b2e1af04b48b88b588e012740d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:50.653869 15293 consensus_queue.cc:1048] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:52.768208 14429 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:17:52.768968 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:52.784408 15053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ee7c7dfa5424b0bb683bb62badf9658"
dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
mode: GRACEFUL
new_leader_uuid: "52976c323d7a4e97bf1415f69c8834ad"
 from {username='slave'} at 127.0.0.1:48290
I20260709 13:17:52.784932 15053 raft_consensus.cc:606] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 2 LEADER]: Received request to transfer leadership to TS 52976c323d7a4e97bf1415f69c8834ad
I20260709 13:17:52.787274 15053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb35caf64cd84f77a7807ec904c0e340"
dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
mode: GRACEFUL
new_leader_uuid: "52976c323d7a4e97bf1415f69c8834ad"
 from {username='slave'} at 127.0.0.1:48290
I20260709 13:17:52.787747 15053 raft_consensus.cc:606] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 1 LEADER]: Received request to transfer leadership to TS 52976c323d7a4e97bf1415f69c8834ad
I20260709 13:17:52.788830 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:52.789271 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:53.005065 15332 raft_consensus.cc:993] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7: : Instructing follower 52976c323d7a4e97bf1415f69c8834ad to start an election
I20260709 13:17:53.005594 15332 raft_consensus.cc:1081] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 1 LEADER]: Signalling peer 52976c323d7a4e97bf1415f69c8834ad to start an election
I20260709 13:17:53.007146 15202 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb35caf64cd84f77a7807ec904c0e340"
dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
 from {username='slave'} at 127.0.0.1:42670
I20260709 13:17:53.007614 15202 raft_consensus.cc:493] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:53.008010 15202 raft_consensus.cc:3060] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:53.012459 15202 raft_consensus.cc:515] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:53.014114 15202 leader_election.cc:290] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 2 election: Requested vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825)
I20260709 13:17:53.014914 15053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb35caf64cd84f77a7807ec904c0e340" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
I20260709 13:17:53.015161 15127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb35caf64cd84f77a7807ec904c0e340" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
I20260709 13:17:53.015431 15053 raft_consensus.cc:3055] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:53.015684 15127 raft_consensus.cc:3060] T eb35caf64cd84f77a7807ec904c0e340 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:53.015728 15053 raft_consensus.cc:740] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Running, Role: LEADER
I20260709 13:17:53.016534 15053 consensus_queue.cc:260] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:53.017403 15053 raft_consensus.cc:3060] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:53.020466 15127 raft_consensus.cc:2468] T eb35caf64cd84f77a7807ec904c0e340 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52976c323d7a4e97bf1415f69c8834ad in term 2.
I20260709 13:17:53.021567 15167 leader_election.cc:304] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 52976c323d7a4e97bf1415f69c8834ad; no voters: 
I20260709 13:17:53.021874 15053 raft_consensus.cc:2468] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52976c323d7a4e97bf1415f69c8834ad in term 2.
I20260709 13:17:53.022254 15289 raft_consensus.cc:2804] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:53.023237 15289 raft_consensus.cc:697] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [term 2 LEADER]: Becoming Leader. State: Replica: 52976c323d7a4e97bf1415f69c8834ad, State: Running, Role: LEADER
I20260709 13:17:53.024001 15289 consensus_queue.cc:237] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:53.030494 14951 catalog_manager.cc:5697] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad reported cstate change: term changed from 1 to 2, leader changed from 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65) to 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67). New cstate: current_term: 2 leader_uuid: "52976c323d7a4e97bf1415f69c8834ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:53.147933 15274 raft_consensus.cc:993] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7: : Instructing follower 52976c323d7a4e97bf1415f69c8834ad to start an election
I20260709 13:17:53.148311 15332 raft_consensus.cc:1081] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 2 LEADER]: Signalling peer 52976c323d7a4e97bf1415f69c8834ad to start an election
I20260709 13:17:53.149794 15202 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ee7c7dfa5424b0bb683bb62badf9658"
dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
 from {username='slave'} at 127.0.0.1:42670
I20260709 13:17:53.150275 15202 raft_consensus.cc:493] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:53.150530 15202 raft_consensus.cc:3060] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:53.155114 15202 raft_consensus.cc:515] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:53.156703 15202 leader_election.cc:290] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 3 election: Requested vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825)
I20260709 13:17:53.157819 15053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee7c7dfa5424b0bb683bb62badf9658" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
I20260709 13:17:53.157837 15127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee7c7dfa5424b0bb683bb62badf9658" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
I20260709 13:17:53.158653 15053 raft_consensus.cc:3055] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:53.158735 15127 raft_consensus.cc:3060] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:53.158969 15053 raft_consensus.cc:740] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Running, Role: LEADER
I20260709 13:17:53.159725 15053 consensus_queue.cc:260] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:53.160666 15053 raft_consensus.cc:3060] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:53.163509 15127 raft_consensus.cc:2468] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52976c323d7a4e97bf1415f69c8834ad in term 3.
I20260709 13:17:53.164549 15167 leader_election.cc:304] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 52976c323d7a4e97bf1415f69c8834ad; no voters: 
I20260709 13:17:53.165223 15289 raft_consensus.cc:2804] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:53.165612 15289 raft_consensus.cc:697] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 3 LEADER]: Becoming Leader. State: Replica: 52976c323d7a4e97bf1415f69c8834ad, State: Running, Role: LEADER
I20260709 13:17:53.165897 15053 raft_consensus.cc:2468] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52976c323d7a4e97bf1415f69c8834ad in term 3.
I20260709 13:17:53.166352 15289 consensus_queue.cc:237] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:53.173429 14951 catalog_manager.cc:5697] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad reported cstate change: term changed from 2 to 3, leader changed from 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65) to 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67). New cstate: current_term: 3 leader_uuid: "52976c323d7a4e97bf1415f69c8834ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:53.410943 15053 raft_consensus.cc:1275] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Refusing update from remote peer 52976c323d7a4e97bf1415f69c8834ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:53.412127 15289 consensus_queue.cc:1048] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:53.421886 15127 raft_consensus.cc:1275] T eb35caf64cd84f77a7807ec904c0e340 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Refusing update from remote peer 52976c323d7a4e97bf1415f69c8834ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:53.423790 15344 consensus_queue.cc:1048] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:53.634635 15127 raft_consensus.cc:1275] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 FOLLOWER]: Refusing update from remote peer 52976c323d7a4e97bf1415f69c8834ad: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:53.635877 15344 consensus_queue.cc:1048] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:53.644305 15053 raft_consensus.cc:1275] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 3 FOLLOWER]: Refusing update from remote peer 52976c323d7a4e97bf1415f69c8834ad: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:53.646059 15335 consensus_queue.cc:1048] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:53.790110 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:53.800233 15200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb35caf64cd84f77a7807ec904c0e340"
dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
mode: GRACEFUL
new_leader_uuid: "4d322b2e1af04b48b88b588e012740d7"
 from {username='slave'} at 127.0.0.1:42682
I20260709 13:17:53.800860 15200 raft_consensus.cc:606] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [term 2 LEADER]: Received request to transfer leadership to TS 4d322b2e1af04b48b88b588e012740d7
I20260709 13:17:53.808301 15200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ee7c7dfa5424b0bb683bb62badf9658"
dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
mode: GRACEFUL
new_leader_uuid: "4d322b2e1af04b48b88b588e012740d7"
 from {username='slave'} at 127.0.0.1:42682
I20260709 13:17:53.808910 15200 raft_consensus.cc:606] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 3 LEADER]: Received request to transfer leadership to TS 4d322b2e1af04b48b88b588e012740d7
I20260709 13:17:53.818303 15127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7809a2a83ca241a4ba6bc8015cc802c6"
dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
mode: GRACEFUL
new_leader_uuid: "4d322b2e1af04b48b88b588e012740d7"
 from {username='slave'} at 127.0.0.1:52616
I20260709 13:17:53.818813 15127 raft_consensus.cc:606] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 LEADER]: Received request to transfer leadership to TS 4d322b2e1af04b48b88b588e012740d7
I20260709 13:17:53.823983 15127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fba2aa7361fd4aad8cb86213305039be"
dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
mode: GRACEFUL
new_leader_uuid: "4d322b2e1af04b48b88b588e012740d7"
 from {username='slave'} at 127.0.0.1:52616
I20260709 13:17:53.824503 15127 raft_consensus.cc:606] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 LEADER]: Received request to transfer leadership to TS 4d322b2e1af04b48b88b588e012740d7
I20260709 13:17:53.829209 15127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0060b39cf81428da22c632f18e594b6"
dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
mode: GRACEFUL
new_leader_uuid: "4d322b2e1af04b48b88b588e012740d7"
 from {username='slave'} at 127.0.0.1:52616
I20260709 13:17:53.829704 15127 raft_consensus.cc:606] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 LEADER]: Received request to transfer leadership to TS 4d322b2e1af04b48b88b588e012740d7
I20260709 13:17:53.850668 15335 raft_consensus.cc:993] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad: : Instructing follower 4d322b2e1af04b48b88b588e012740d7 to start an election
I20260709 13:17:53.851155 15344 raft_consensus.cc:1081] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [term 2 LEADER]: Signalling peer 4d322b2e1af04b48b88b588e012740d7 to start an election
I20260709 13:17:53.852536 15053 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb35caf64cd84f77a7807ec904c0e340"
dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
 from {username='slave'} at 127.0.0.1:52514
I20260709 13:17:53.853053 15053 raft_consensus.cc:493] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:53.853348 15053 raft_consensus.cc:3060] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:53.857872 15351 raft_consensus.cc:993] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3: : Instructing follower 4d322b2e1af04b48b88b588e012740d7 to start an election
I20260709 13:17:53.858322 15285 raft_consensus.cc:1081] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 LEADER]: Signalling peer 4d322b2e1af04b48b88b588e012740d7 to start an election
I20260709 13:17:53.859505 15053 raft_consensus.cc:515] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:53.859933 15052 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0060b39cf81428da22c632f18e594b6"
dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
 from {username='slave'} at 127.0.0.1:52500
I20260709 13:17:53.860446 15052 raft_consensus.cc:493] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:53.860764 15052 raft_consensus.cc:3060] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:53.862020 15053 leader_election.cc:290] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 3 election: Requested vote from peers 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:53.862257 15200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb35caf64cd84f77a7807ec904c0e340" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
I20260709 13:17:53.862023 15127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb35caf64cd84f77a7807ec904c0e340" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
I20260709 13:17:53.862905 15200 raft_consensus.cc:3055] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:53.863195 15127 raft_consensus.cc:3060] T eb35caf64cd84f77a7807ec904c0e340 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:53.863363 15200 raft_consensus.cc:740] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52976c323d7a4e97bf1415f69c8834ad, State: Running, Role: LEADER
I20260709 13:17:53.864297 15200 consensus_queue.cc:260] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:53.865154 15200 raft_consensus.cc:3060] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:53.867774 15052 raft_consensus.cc:515] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:53.869665 15200 raft_consensus.cc:2468] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 3.
I20260709 13:17:53.869962 15127 raft_consensus.cc:2468] T eb35caf64cd84f77a7807ec904c0e340 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 3.
I20260709 13:17:53.870754 15128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0060b39cf81428da22c632f18e594b6" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
I20260709 13:17:53.871214 15128 raft_consensus.cc:3055] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:53.871173 15019 leader_election.cc:304] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d322b2e1af04b48b88b588e012740d7, 52976c323d7a4e97bf1415f69c8834ad; no voters: 
I20260709 13:17:53.871734 15128 raft_consensus.cc:740] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11b6d8a2d2034e7d8e53e9df74226ac3, State: Running, Role: LEADER
I20260709 13:17:53.872471 15332 raft_consensus.cc:2804] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:53.872968 15052 leader_election.cc:290] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 2 election: Requested vote from peers 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:53.872557 15128 consensus_queue.cc:260] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:53.873647 15128 raft_consensus.cc:3060] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:53.873620 15202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0060b39cf81428da22c632f18e594b6" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
I20260709 13:17:53.874396 15202 raft_consensus.cc:3060] T c0060b39cf81428da22c632f18e594b6 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:53.875320 15332 raft_consensus.cc:697] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 3 LEADER]: Becoming Leader. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Running, Role: LEADER
I20260709 13:17:53.876253 15332 consensus_queue.cc:237] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:53.880479 15128 raft_consensus.cc:2468] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 2.
I20260709 13:17:53.881121 15202 raft_consensus.cc:2468] T c0060b39cf81428da22c632f18e594b6 P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 2.
I20260709 13:17:53.881799 15018 leader_election.cc:304] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 4d322b2e1af04b48b88b588e012740d7; no voters: 
I20260709 13:17:53.882622 15332 raft_consensus.cc:2804] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:53.883059 15332 raft_consensus.cc:697] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [term 2 LEADER]: Becoming Leader. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Running, Role: LEADER
I20260709 13:17:53.883458 14950 catalog_manager.cc:5697] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 reported cstate change: term changed from 2 to 3, leader changed from 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67) to 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65). New cstate: current_term: 3 leader_uuid: "4d322b2e1af04b48b88b588e012740d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:53.883641 15332 consensus_queue.cc:237] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:54.084241 15349 raft_consensus.cc:993] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad: : Instructing follower 4d322b2e1af04b48b88b588e012740d7 to start an election
I20260709 13:17:54.084734 15349 raft_consensus.cc:1081] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 3 LEADER]: Signalling peer 4d322b2e1af04b48b88b588e012740d7 to start an election
I20260709 13:17:54.086410 15052 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ee7c7dfa5424b0bb683bb62badf9658"
dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
 from {username='slave'} at 127.0.0.1:52514
I20260709 13:17:54.086946 15052 raft_consensus.cc:493] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:54.087360 15052 raft_consensus.cc:3060] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:54.102427 15052 raft_consensus.cc:515] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:54.106412 15128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee7c7dfa5424b0bb683bb62badf9658" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
I20260709 13:17:54.107132 15128 raft_consensus.cc:3060] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:54.114295 15128 raft_consensus.cc:2468] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 4.
I20260709 13:17:54.115324 15052 leader_election.cc:290] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 4 election: Requested vote from peers 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:54.116537 15018 leader_election.cc:304] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 4d322b2e1af04b48b88b588e012740d7; no voters: 
I20260709 13:17:54.149614 15352 raft_consensus.cc:2804] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:54.150334 15352 raft_consensus.cc:697] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 4 LEADER]: Becoming Leader. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Running, Role: LEADER
I20260709 13:17:54.151270 15352 consensus_queue.cc:237] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:54.159567 15201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee7c7dfa5424b0bb683bb62badf9658" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
I20260709 13:17:54.160077 15201 raft_consensus.cc:3055] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:54.160315 15201 raft_consensus.cc:740] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 52976c323d7a4e97bf1415f69c8834ad, State: Running, Role: LEADER
I20260709 13:17:54.160970 15201 consensus_queue.cc:260] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:54.162017 15201 raft_consensus.cc:3060] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:54.166980 15201 raft_consensus.cc:2468] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 4.
I20260709 13:17:54.167304 15351 raft_consensus.cc:993] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3: : Instructing follower 4d322b2e1af04b48b88b588e012740d7 to start an election
I20260709 13:17:54.167587 14950 catalog_manager.cc:5697] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 reported cstate change: term changed from 1 to 2, leader changed from 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66) to 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65). New cstate: current_term: 2 leader_uuid: "4d322b2e1af04b48b88b588e012740d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:54.169310 15285 raft_consensus.cc:1081] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 LEADER]: Signalling peer 4d322b2e1af04b48b88b588e012740d7 to start an election
I20260709 13:17:54.172864 15052 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fba2aa7361fd4aad8cb86213305039be"
dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
 from {username='slave'} at 127.0.0.1:52500
I20260709 13:17:54.173480 15052 raft_consensus.cc:493] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:54.173868 15052 raft_consensus.cc:3060] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:54.179754 15052 raft_consensus.cc:515] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:54.181399 15052 leader_election.cc:290] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 3 election: Requested vote from peers 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:54.182226 15128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba2aa7361fd4aad8cb86213305039be" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
I20260709 13:17:54.182673 15201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba2aa7361fd4aad8cb86213305039be" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
I20260709 13:17:54.182925 15128 raft_consensus.cc:3055] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:54.182410 14950 catalog_manager.cc:5697] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 reported cstate change: term changed from 3 to 4, leader changed from 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67) to 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65). New cstate: current_term: 4 leader_uuid: "4d322b2e1af04b48b88b588e012740d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:54.183388 15128 raft_consensus.cc:740] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 11b6d8a2d2034e7d8e53e9df74226ac3, State: Running, Role: LEADER
I20260709 13:17:54.183403 15201 raft_consensus.cc:3060] T fba2aa7361fd4aad8cb86213305039be P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:54.184321 15128 consensus_queue.cc:260] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:54.185550 15128 raft_consensus.cc:3060] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:54.190476 15201 raft_consensus.cc:2468] T fba2aa7361fd4aad8cb86213305039be P 52976c323d7a4e97bf1415f69c8834ad [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 3.
I20260709 13:17:54.192119 15019 leader_election.cc:304] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d322b2e1af04b48b88b588e012740d7, 52976c323d7a4e97bf1415f69c8834ad; no voters: 
I20260709 13:17:54.192852 15352 raft_consensus.cc:2804] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:54.193256 15352 raft_consensus.cc:697] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 3 LEADER]: Becoming Leader. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Running, Role: LEADER
I20260709 13:17:54.194048 15352 consensus_queue.cc:237] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:54.194818 15128 raft_consensus.cc:2468] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 3.
I20260709 13:17:54.202495 14950 catalog_manager.cc:5697] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 reported cstate change: term changed from 2 to 3, leader changed from 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66) to 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65). New cstate: current_term: 3 leader_uuid: "4d322b2e1af04b48b88b588e012740d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:54.203835 15358 raft_consensus.cc:993] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3: : Instructing follower 4d322b2e1af04b48b88b588e012740d7 to start an election
I20260709 13:17:54.204257 15285 raft_consensus.cc:1081] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 LEADER]: Signalling peer 4d322b2e1af04b48b88b588e012740d7 to start an election
I20260709 13:17:54.205679 15052 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7809a2a83ca241a4ba6bc8015cc802c6"
dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
 from {username='slave'} at 127.0.0.1:52500
I20260709 13:17:54.206225 15052 raft_consensus.cc:493] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:54.206526 15052 raft_consensus.cc:3060] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:54.212020 15052 raft_consensus.cc:515] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:54.213594 15052 leader_election.cc:290] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 3 election: Requested vote from peers 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:54.214344 15128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7809a2a83ca241a4ba6bc8015cc802c6" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
I20260709 13:17:54.214619 15201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7809a2a83ca241a4ba6bc8015cc802c6" candidate_uuid: "4d322b2e1af04b48b88b588e012740d7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
I20260709 13:17:54.214926 15128 raft_consensus.cc:3055] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:54.215097 15201 raft_consensus.cc:3060] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:54.215252 15128 raft_consensus.cc:740] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 11b6d8a2d2034e7d8e53e9df74226ac3, State: Running, Role: LEADER
I20260709 13:17:54.215965 15128 consensus_queue.cc:260] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:54.216881 15128 raft_consensus.cc:3060] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:54.220018 15201 raft_consensus.cc:2468] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 3.
I20260709 13:17:54.221062 15019 leader_election.cc:304] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d322b2e1af04b48b88b588e012740d7, 52976c323d7a4e97bf1415f69c8834ad; no voters: 
I20260709 13:17:54.221168 15128 raft_consensus.cc:2468] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d322b2e1af04b48b88b588e012740d7 in term 3.
I20260709 13:17:54.221789 15352 raft_consensus.cc:2804] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:54.222293 15352 raft_consensus.cc:697] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [term 3 LEADER]: Becoming Leader. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Running, Role: LEADER
I20260709 13:17:54.223079 15352 consensus_queue.cc:237] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:54.230370 14950 catalog_manager.cc:5697] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 reported cstate change: term changed from 2 to 3, leader changed from 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66) to 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65). New cstate: current_term: 3 leader_uuid: "4d322b2e1af04b48b88b588e012740d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:54.266983 15201 raft_consensus.cc:1275] T c0060b39cf81428da22c632f18e594b6 P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Refusing update from remote peer 4d322b2e1af04b48b88b588e012740d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:54.268226 15352 consensus_queue.cc:1048] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:54.277493 15128 raft_consensus.cc:1275] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Refusing update from remote peer 4d322b2e1af04b48b88b588e012740d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:54.280467 15332 consensus_queue.cc:1048] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:54.306563 15201 raft_consensus.cc:1275] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [term 3 FOLLOWER]: Refusing update from remote peer 4d322b2e1af04b48b88b588e012740d7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:54.307819 15332 consensus_queue.cc:1048] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:54.315404 15128 raft_consensus.cc:1275] T eb35caf64cd84f77a7807ec904c0e340 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 FOLLOWER]: Refusing update from remote peer 4d322b2e1af04b48b88b588e012740d7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:54.317798 15366 consensus_queue.cc:1048] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:54.629184 15128 raft_consensus.cc:1275] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 FOLLOWER]: Refusing update from remote peer 4d322b2e1af04b48b88b588e012740d7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:54.630422 15366 consensus_queue.cc:1048] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:54.638360 15201 raft_consensus.cc:1275] T fba2aa7361fd4aad8cb86213305039be P 52976c323d7a4e97bf1415f69c8834ad [term 3 FOLLOWER]: Refusing update from remote peer 4d322b2e1af04b48b88b588e012740d7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:54.639905 15352 consensus_queue.cc:1048] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:54.645395 15128 raft_consensus.cc:1275] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 FOLLOWER]: Refusing update from remote peer 4d322b2e1af04b48b88b588e012740d7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:54.647161 15366 consensus_queue.cc:1048] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:54.657135 15201 raft_consensus.cc:1275] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 3 FOLLOWER]: Refusing update from remote peer 4d322b2e1af04b48b88b588e012740d7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:54.658569 15366 consensus_queue.cc:1048] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:54.695322 15201 raft_consensus.cc:1275] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 4 FOLLOWER]: Refusing update from remote peer 4d322b2e1af04b48b88b588e012740d7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:54.696620 15352 consensus_queue.cc:1048] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:54.704421 15128 raft_consensus.cc:1275] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 4 FOLLOWER]: Refusing update from remote peer 4d322b2e1af04b48b88b588e012740d7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:54.705730 15366 consensus_queue.cc:1048] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:17:55.104804 15149 debug-util.cc:398] Leaking SignalData structure 0x7b0800193160 after lost signal to thread 14432
W20260709 13:17:55.105639 15149 debug-util.cc:398] Leaking SignalData structure 0x7b08000f6160 after lost signal to thread 14985
W20260709 13:17:55.106496 15149 debug-util.cc:398] Leaking SignalData structure 0x7b08000ba2a0 after lost signal to thread 15077
W20260709 13:17:55.107211 15149 debug-util.cc:398] Leaking SignalData structure 0x7b08000def40 after lost signal to thread 15152
W20260709 13:17:55.107898 15149 debug-util.cc:398] Leaking SignalData structure 0x7b0800142060 after lost signal to thread 15226
I20260709 13:17:57.020515 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:57.028944 15052 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3b1769efa1d947e78a05c89546a0e917"
dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
mode: GRACEFUL
new_leader_uuid: "52976c323d7a4e97bf1415f69c8834ad"
 from {username='slave'} at 127.0.0.1:48290
I20260709 13:17:57.029747 15052 raft_consensus.cc:606] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [term 1 LEADER]: Received request to transfer leadership to TS 52976c323d7a4e97bf1415f69c8834ad
I20260709 13:17:57.032642 15052 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ee7c7dfa5424b0bb683bb62badf9658"
dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
mode: GRACEFUL
new_leader_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
 from {username='slave'} at 127.0.0.1:48290
I20260709 13:17:57.033118 15052 raft_consensus.cc:606] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 4 LEADER]: Received request to transfer leadership to TS 11b6d8a2d2034e7d8e53e9df74226ac3
I20260709 13:17:57.035686 15052 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0060b39cf81428da22c632f18e594b6"
dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
mode: GRACEFUL
new_leader_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
 from {username='slave'} at 127.0.0.1:48290
I20260709 13:17:57.036214 15052 raft_consensus.cc:606] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [term 2 LEADER]: Received request to transfer leadership to TS 11b6d8a2d2034e7d8e53e9df74226ac3
I20260709 13:17:57.038689 15052 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db57138f7b4e42ac852be5436039207e"
dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
mode: GRACEFUL
new_leader_uuid: "52976c323d7a4e97bf1415f69c8834ad"
 from {username='slave'} at 127.0.0.1:48290
I20260709 13:17:57.039135 15052 raft_consensus.cc:606] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [term 2 LEADER]: Received request to transfer leadership to TS 52976c323d7a4e97bf1415f69c8834ad
I20260709 13:17:57.040256 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:17:57.040871 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:57.117548 15376 raft_consensus.cc:993] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7: : Instructing follower 11b6d8a2d2034e7d8e53e9df74226ac3 to start an election
I20260709 13:17:57.118103 15375 raft_consensus.cc:1081] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 4 LEADER]: Signalling peer 11b6d8a2d2034e7d8e53e9df74226ac3 to start an election
I20260709 13:17:57.119851 15127 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ee7c7dfa5424b0bb683bb62badf9658"
dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
 from {username='slave'} at 127.0.0.1:52594
I20260709 13:17:57.120371 15127 raft_consensus.cc:493] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:57.120729 15127 raft_consensus.cc:3060] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:57.126318 15127 raft_consensus.cc:515] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:57.128341 15127 leader_election.cc:290] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 5 election: Requested vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:57.129204 15052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee7c7dfa5424b0bb683bb62badf9658" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
I20260709 13:17:57.129648 15201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ee7c7dfa5424b0bb683bb62badf9658" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
I20260709 13:17:57.130147 15052 raft_consensus.cc:3055] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:57.130270 15201 raft_consensus.cc:3060] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:57.130517 15052 raft_consensus.cc:740] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Running, Role: LEADER
I20260709 13:17:57.131517 15052 consensus_queue.cc:260] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:57.132964 15052 raft_consensus.cc:3060] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:57.135859 15201 raft_consensus.cc:2468] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11b6d8a2d2034e7d8e53e9df74226ac3 in term 5.
I20260709 13:17:57.137326 15094 leader_election.cc:304] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 52976c323d7a4e97bf1415f69c8834ad; no voters: 
I20260709 13:17:57.138468 15285 raft_consensus.cc:2804] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:57.140928 15052 raft_consensus.cc:2468] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11b6d8a2d2034e7d8e53e9df74226ac3 in term 5.
I20260709 13:17:57.141499 15285 raft_consensus.cc:697] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 5 LEADER]: Becoming Leader. State: Replica: 11b6d8a2d2034e7d8e53e9df74226ac3, State: Running, Role: LEADER
I20260709 13:17:57.143059 15285 consensus_queue.cc:237] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:57.151760 14950 catalog_manager.cc:5697] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 reported cstate change: term changed from 4 to 5, leader changed from 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65) to 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66). New cstate: current_term: 5 leader_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.239966 15375 raft_consensus.cc:993] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7: : Instructing follower 52976c323d7a4e97bf1415f69c8834ad to start an election
I20260709 13:17:57.240788 15375 raft_consensus.cc:1081] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [term 1 LEADER]: Signalling peer 52976c323d7a4e97bf1415f69c8834ad to start an election
I20260709 13:17:57.242856 15201 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3b1769efa1d947e78a05c89546a0e917"
dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
 from {username='slave'} at 127.0.0.1:42670
I20260709 13:17:57.244047 15201 raft_consensus.cc:493] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:57.244824 15201 raft_consensus.cc:3060] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.251688 15201 raft_consensus.cc:515] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:57.253393 15201 leader_election.cc:290] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 2 election: Requested vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825)
I20260709 13:17:57.254437 15052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b1769efa1d947e78a05c89546a0e917" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
I20260709 13:17:57.254639 15127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b1769efa1d947e78a05c89546a0e917" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
I20260709 13:17:57.255069 15052 raft_consensus.cc:3055] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:57.255167 15127 raft_consensus.cc:3060] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.255477 15052 raft_consensus.cc:740] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Running, Role: LEADER
I20260709 13:17:57.256363 15052 consensus_queue.cc:260] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:57.257637 15052 raft_consensus.cc:3060] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.259805 15127 raft_consensus.cc:2468] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52976c323d7a4e97bf1415f69c8834ad in term 2.
I20260709 13:17:57.261022 15167 leader_election.cc:304] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 52976c323d7a4e97bf1415f69c8834ad; no voters: 
I20260709 13:17:57.261849 15344 raft_consensus.cc:2804] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:57.263146 15344 raft_consensus.cc:697] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad [term 2 LEADER]: Becoming Leader. State: Replica: 52976c323d7a4e97bf1415f69c8834ad, State: Running, Role: LEADER
I20260709 13:17:57.264065 15344 consensus_queue.cc:237] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:57.264577 15052 raft_consensus.cc:2468] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52976c323d7a4e97bf1415f69c8834ad in term 2.
I20260709 13:17:57.272042 14950 catalog_manager.cc:5697] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad reported cstate change: term changed from 1 to 2, leader changed from 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65) to 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67). New cstate: current_term: 2 leader_uuid: "52976c323d7a4e97bf1415f69c8834ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:57.443432 15376 raft_consensus.cc:993] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7: : Instructing follower 52976c323d7a4e97bf1415f69c8834ad to start an election
I20260709 13:17:57.443912 15375 raft_consensus.cc:1081] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [term 2 LEADER]: Signalling peer 52976c323d7a4e97bf1415f69c8834ad to start an election
I20260709 13:17:57.445292 15201 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "db57138f7b4e42ac852be5436039207e"
dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
 from {username='slave'} at 127.0.0.1:42670
I20260709 13:17:57.445734 15201 raft_consensus.cc:493] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:57.446046 15201 raft_consensus.cc:3060] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:57.450711 15201 raft_consensus.cc:515] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:57.452320 15201 leader_election.cc:290] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 3 election: Requested vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825)
I20260709 13:17:57.453107 15052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db57138f7b4e42ac852be5436039207e" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
I20260709 13:17:57.453411 15127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db57138f7b4e42ac852be5436039207e" candidate_uuid: "52976c323d7a4e97bf1415f69c8834ad" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
I20260709 13:17:57.453680 15052 raft_consensus.cc:3055] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:57.453886 15127 raft_consensus.cc:3060] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:57.454020 15052 raft_consensus.cc:740] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Running, Role: LEADER
I20260709 13:17:57.454818 15052 consensus_queue.cc:260] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:57.455698 15052 raft_consensus.cc:3060] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:57.458127 15127 raft_consensus.cc:2468] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52976c323d7a4e97bf1415f69c8834ad in term 3.
I20260709 13:17:57.459088 15167 leader_election.cc:304] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 52976c323d7a4e97bf1415f69c8834ad; no voters: 
I20260709 13:17:57.459863 15344 raft_consensus.cc:2804] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:57.460302 15344 raft_consensus.cc:697] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad [term 3 LEADER]: Becoming Leader. State: Replica: 52976c323d7a4e97bf1415f69c8834ad, State: Running, Role: LEADER
I20260709 13:17:57.460705 15052 raft_consensus.cc:2468] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52976c323d7a4e97bf1415f69c8834ad in term 3.
I20260709 13:17:57.461023 15344 consensus_queue.cc:237] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:57.468217 14950 catalog_manager.cc:5697] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad reported cstate change: term changed from 2 to 3, leader changed from 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65) to 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67). New cstate: current_term: 3 leader_uuid: "52976c323d7a4e97bf1415f69c8834ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:57.508291 15376 raft_consensus.cc:993] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7: : Instructing follower 11b6d8a2d2034e7d8e53e9df74226ac3 to start an election
I20260709 13:17:57.508716 15375 raft_consensus.cc:1081] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [term 2 LEADER]: Signalling peer 11b6d8a2d2034e7d8e53e9df74226ac3 to start an election
I20260709 13:17:57.510155 15127 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0060b39cf81428da22c632f18e594b6"
dest_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3"
 from {username='slave'} at 127.0.0.1:52594
I20260709 13:17:57.510588 15127 raft_consensus.cc:493] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:57.510885 15127 raft_consensus.cc:3060] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:57.515061 15127 raft_consensus.cc:515] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:57.516444 15127 leader_election.cc:290] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 3 election: Requested vote from peers 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629), 52976c323d7a4e97bf1415f69c8834ad (127.14.23.67:36917)
I20260709 13:17:57.517321 15052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0060b39cf81428da22c632f18e594b6" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4d322b2e1af04b48b88b588e012740d7"
I20260709 13:17:57.517555 15201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0060b39cf81428da22c632f18e594b6" candidate_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "52976c323d7a4e97bf1415f69c8834ad"
I20260709 13:17:57.517802 15052 raft_consensus.cc:3055] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:57.518097 15201 raft_consensus.cc:3060] T c0060b39cf81428da22c632f18e594b6 P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:57.518095 15052 raft_consensus.cc:740] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4d322b2e1af04b48b88b588e012740d7, State: Running, Role: LEADER
I20260709 13:17:57.518940 15052 consensus_queue.cc:260] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:57.519778 15052 raft_consensus.cc:3060] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:57.523124 15201 raft_consensus.cc:2468] T c0060b39cf81428da22c632f18e594b6 P 52976c323d7a4e97bf1415f69c8834ad [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11b6d8a2d2034e7d8e53e9df74226ac3 in term 3.
I20260709 13:17:57.524156 15052 raft_consensus.cc:2468] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11b6d8a2d2034e7d8e53e9df74226ac3 in term 3.
I20260709 13:17:57.524159 15094 leader_election.cc:304] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6d8a2d2034e7d8e53e9df74226ac3, 52976c323d7a4e97bf1415f69c8834ad; no voters: 
I20260709 13:17:57.525117 15285 raft_consensus.cc:2804] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:57.525544 15285 raft_consensus.cc:697] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 LEADER]: Becoming Leader. State: Replica: 11b6d8a2d2034e7d8e53e9df74226ac3, State: Running, Role: LEADER
I20260709 13:17:57.526269 15285 consensus_queue.cc:237] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } }
I20260709 13:17:57.533484 14950 catalog_manager.cc:5697] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 reported cstate change: term changed from 2 to 3, leader changed from 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65) to 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66). New cstate: current_term: 3 leader_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.635560 15052 raft_consensus.cc:1275] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 5 FOLLOWER]: Refusing update from remote peer 11b6d8a2d2034e7d8e53e9df74226ac3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:17:57.636636 15285 consensus_queue.cc:1048] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:57.646788 15201 raft_consensus.cc:1275] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 5 FOLLOWER]: Refusing update from remote peer 11b6d8a2d2034e7d8e53e9df74226ac3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:17:57.648355 15285 consensus_queue.cc:1048] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:57.725117 15127 raft_consensus.cc:1275] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Refusing update from remote peer 52976c323d7a4e97bf1415f69c8834ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:57.726416 15344 consensus_queue.cc:1048] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:57.737112 15052 raft_consensus.cc:1275] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Refusing update from remote peer 52976c323d7a4e97bf1415f69c8834ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:57.738543 15393 consensus_queue.cc:1048] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:58.014124 15201 raft_consensus.cc:1275] T c0060b39cf81428da22c632f18e594b6 P 52976c323d7a4e97bf1415f69c8834ad [term 3 FOLLOWER]: Refusing update from remote peer 11b6d8a2d2034e7d8e53e9df74226ac3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:58.015348 15285 consensus_queue.cc:1048] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52976c323d7a4e97bf1415f69c8834ad" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 36917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:58.023964 15052 raft_consensus.cc:1275] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [term 3 FOLLOWER]: Refusing update from remote peer 11b6d8a2d2034e7d8e53e9df74226ac3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:58.025816 15396 consensus_queue.cc:1048] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d322b2e1af04b48b88b588e012740d7" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:58.041790 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:58.048025 14429 tablet_server.cc:179] TabletServer@127.14.23.65:0 shutting down...
I20260709 13:17:58.050087 15127 raft_consensus.cc:1275] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 FOLLOWER]: Refusing update from remote peer 52976c323d7a4e97bf1415f69c8834ad: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:58.051766 15393 consensus_queue.cc:1048] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "11b6d8a2d2034e7d8e53e9df74226ac3" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 42825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:17:58.070844 15168 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 4 similar messages]
W20260709 13:17:58.078022 15168 consensus_peers.cc:597] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad -> Peer 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629): Couldn't send request to peer 4d322b2e1af04b48b88b588e012740d7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:58.087123 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:58.087951 14429 tablet_replica.cc:333] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7: stopping tablet replica
I20260709 13:17:58.088599 14429 raft_consensus.cc:2243] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:58.089020 14429 raft_consensus.cc:2272] T c0060b39cf81428da22c632f18e594b6 P 4d322b2e1af04b48b88b588e012740d7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.091291 14429 tablet_replica.cc:333] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7: stopping tablet replica
I20260709 13:17:58.091821 14429 raft_consensus.cc:2243] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:58.092581 14429 raft_consensus.cc:2272] T 7809a2a83ca241a4ba6bc8015cc802c6 P 4d322b2e1af04b48b88b588e012740d7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.094383 14429 tablet_replica.cc:333] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7: stopping tablet replica
I20260709 13:17:58.094904 14429 raft_consensus.cc:2243] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:58.095281 14429 raft_consensus.cc:2272] T 3b1769efa1d947e78a05c89546a0e917 P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.097398 14429 tablet_replica.cc:333] T 49fac936bede4d7491681c3d913a1fd0 P 4d322b2e1af04b48b88b588e012740d7: stopping tablet replica
I20260709 13:17:58.097890 14429 raft_consensus.cc:2243] T 49fac936bede4d7491681c3d913a1fd0 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:58.098356 14429 raft_consensus.cc:2272] T 49fac936bede4d7491681c3d913a1fd0 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.099977 14429 tablet_replica.cc:333] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7: stopping tablet replica
I20260709 13:17:58.100589 14429 raft_consensus.cc:2243] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:58.101554 14429 raft_consensus.cc:2272] T 6baa6e039e11445488bb6c0086c79cf6 P 4d322b2e1af04b48b88b588e012740d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.103267 14429 tablet_replica.cc:333] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7: stopping tablet replica
I20260709 13:17:58.103736 14429 raft_consensus.cc:2243] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:58.104115 14429 raft_consensus.cc:2272] T 5ee7c7dfa5424b0bb683bb62badf9658 P 4d322b2e1af04b48b88b588e012740d7 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.106218 14429 tablet_replica.cc:333] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7: stopping tablet replica
I20260709 13:17:58.106709 14429 raft_consensus.cc:2243] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:58.107112 14429 raft_consensus.cc:2272] T f16bebdc1ec542578d9b1dfef1160556 P 4d322b2e1af04b48b88b588e012740d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.108948 14429 tablet_replica.cc:333] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7: stopping tablet replica
I20260709 13:17:58.109406 14429 raft_consensus.cc:2243] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:58.110227 14429 raft_consensus.cc:2272] T fba2aa7361fd4aad8cb86213305039be P 4d322b2e1af04b48b88b588e012740d7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.112056 14429 tablet_replica.cc:333] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7: stopping tablet replica
I20260709 13:17:58.112486 14429 raft_consensus.cc:2243] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:58.113339 14429 raft_consensus.cc:2272] T eb35caf64cd84f77a7807ec904c0e340 P 4d322b2e1af04b48b88b588e012740d7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.115733 14429 tablet_replica.cc:333] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7: stopping tablet replica
I20260709 13:17:58.116305 14429 raft_consensus.cc:2243] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:58.116670 14429 raft_consensus.cc:2272] T db57138f7b4e42ac852be5436039207e P 4d322b2e1af04b48b88b588e012740d7 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:58.136560 15168 consensus_peers.cc:597] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad -> Peer 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629): Couldn't send request to peer 4d322b2e1af04b48b88b588e012740d7. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43629: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:58.137290 14429 tablet_server.cc:196] TabletServer@127.14.23.65:0 shutdown complete.
I20260709 13:17:58.156903 14429 tablet_server.cc:179] TabletServer@127.14.23.66:0 shutting down...
W20260709 13:17:58.163694 15167 consensus_peers.cc:597] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad -> Peer 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825): Couldn't send request to peer 11b6d8a2d2034e7d8e53e9df74226ac3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:58.173033 15168 consensus_peers.cc:597] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad -> Peer 4d322b2e1af04b48b88b588e012740d7 (127.14.23.65:43629): Couldn't send request to peer 4d322b2e1af04b48b88b588e012740d7. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43629: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:58.184806 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:58.185642 14429 tablet_replica.cc:333] T 6baa6e039e11445488bb6c0086c79cf6 P 11b6d8a2d2034e7d8e53e9df74226ac3: stopping tablet replica
I20260709 13:17:58.186305 14429 raft_consensus.cc:2243] T 6baa6e039e11445488bb6c0086c79cf6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:58.186751 14429 raft_consensus.cc:2272] T 6baa6e039e11445488bb6c0086c79cf6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.188542 14429 tablet_replica.cc:333] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3: stopping tablet replica
I20260709 13:17:58.189051 14429 raft_consensus.cc:2243] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:58.189491 14429 raft_consensus.cc:2272] T db57138f7b4e42ac852be5436039207e P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.191591 14429 tablet_replica.cc:333] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3: stopping tablet replica
I20260709 13:17:58.192090 14429 raft_consensus.cc:2243] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:58.192816 14429 raft_consensus.cc:2272] T f16bebdc1ec542578d9b1dfef1160556 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.194520 14429 tablet_replica.cc:333] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3: stopping tablet replica
I20260709 13:17:58.195012 14429 raft_consensus.cc:2243] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:17:58.195676 14429 raft_consensus.cc:2272] T 5ee7c7dfa5424b0bb683bb62badf9658 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.197592 14429 tablet_replica.cc:333] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3: stopping tablet replica
I20260709 13:17:58.198127 14429 raft_consensus.cc:2243] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:58.198823 14429 raft_consensus.cc:2272] T c0060b39cf81428da22c632f18e594b6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.200887 14429 tablet_replica.cc:333] T 49fac936bede4d7491681c3d913a1fd0 P 11b6d8a2d2034e7d8e53e9df74226ac3: stopping tablet replica
I20260709 13:17:58.201385 14429 raft_consensus.cc:2243] T 49fac936bede4d7491681c3d913a1fd0 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:58.201771 14429 raft_consensus.cc:2272] T 49fac936bede4d7491681c3d913a1fd0 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.203495 14429 tablet_replica.cc:333] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3: stopping tablet replica
I20260709 13:17:58.203943 14429 raft_consensus.cc:2243] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:58.204327 14429 raft_consensus.cc:2272] T 7809a2a83ca241a4ba6bc8015cc802c6 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.205977 14429 tablet_replica.cc:333] T eb35caf64cd84f77a7807ec904c0e340 P 11b6d8a2d2034e7d8e53e9df74226ac3: stopping tablet replica
I20260709 13:17:58.206460 14429 raft_consensus.cc:2243] T eb35caf64cd84f77a7807ec904c0e340 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:58.206858 14429 raft_consensus.cc:2272] T eb35caf64cd84f77a7807ec904c0e340 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.208456 14429 tablet_replica.cc:333] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3: stopping tablet replica
I20260709 13:17:58.208901 14429 raft_consensus.cc:2243] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:58.209275 14429 raft_consensus.cc:2272] T fba2aa7361fd4aad8cb86213305039be P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.210979 14429 tablet_replica.cc:333] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3: stopping tablet replica
I20260709 13:17:58.211444 14429 raft_consensus.cc:2243] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:58.211833 14429 raft_consensus.cc:2272] T 3b1769efa1d947e78a05c89546a0e917 P 11b6d8a2d2034e7d8e53e9df74226ac3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.257483 14429 tablet_server.cc:196] TabletServer@127.14.23.66:0 shutdown complete.
I20260709 13:17:58.274786 14429 tablet_server.cc:179] TabletServer@127.14.23.67:0 shutting down...
W20260709 13:17:58.295451 15167 consensus_peers.cc:597] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad -> Peer 11b6d8a2d2034e7d8e53e9df74226ac3 (127.14.23.66:42825): Couldn't send request to peer 11b6d8a2d2034e7d8e53e9df74226ac3. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.66:42825: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:58.299396 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:58.300065 14429 tablet_replica.cc:333] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad: stopping tablet replica
I20260709 13:17:58.300642 14429 raft_consensus.cc:2243] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:58.301394 14429 raft_consensus.cc:2272] T 3b1769efa1d947e78a05c89546a0e917 P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.303369 14429 tablet_replica.cc:333] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad: stopping tablet replica
I20260709 13:17:58.303877 14429 raft_consensus.cc:2243] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:58.304576 14429 raft_consensus.cc:2272] T db57138f7b4e42ac852be5436039207e P 52976c323d7a4e97bf1415f69c8834ad [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.306507 14429 tablet_replica.cc:333] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad: stopping tablet replica
I20260709 13:17:58.307003 14429 raft_consensus.cc:2243] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:58.307426 14429 raft_consensus.cc:2272] T 7809a2a83ca241a4ba6bc8015cc802c6 P 52976c323d7a4e97bf1415f69c8834ad [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.309026 14429 tablet_replica.cc:333] T 6baa6e039e11445488bb6c0086c79cf6 P 52976c323d7a4e97bf1415f69c8834ad: stopping tablet replica
I20260709 13:17:58.309485 14429 raft_consensus.cc:2243] T 6baa6e039e11445488bb6c0086c79cf6 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:58.309888 14429 raft_consensus.cc:2272] T 6baa6e039e11445488bb6c0086c79cf6 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.311553 14429 tablet_replica.cc:333] T c0060b39cf81428da22c632f18e594b6 P 52976c323d7a4e97bf1415f69c8834ad: stopping tablet replica
I20260709 13:17:58.312013 14429 raft_consensus.cc:2243] T c0060b39cf81428da22c632f18e594b6 P 52976c323d7a4e97bf1415f69c8834ad [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:58.312390 14429 raft_consensus.cc:2272] T c0060b39cf81428da22c632f18e594b6 P 52976c323d7a4e97bf1415f69c8834ad [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.314354 14429 tablet_replica.cc:333] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad: stopping tablet replica
I20260709 13:17:58.314821 14429 raft_consensus.cc:2243] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:58.315217 14429 raft_consensus.cc:2272] T eb35caf64cd84f77a7807ec904c0e340 P 52976c323d7a4e97bf1415f69c8834ad [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.316768 14429 tablet_replica.cc:333] T fba2aa7361fd4aad8cb86213305039be P 52976c323d7a4e97bf1415f69c8834ad: stopping tablet replica
I20260709 13:17:58.317233 14429 raft_consensus.cc:2243] T fba2aa7361fd4aad8cb86213305039be P 52976c323d7a4e97bf1415f69c8834ad [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:58.317620 14429 raft_consensus.cc:2272] T fba2aa7361fd4aad8cb86213305039be P 52976c323d7a4e97bf1415f69c8834ad [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.319237 14429 tablet_replica.cc:333] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad: stopping tablet replica
I20260709 13:17:58.319676 14429 raft_consensus.cc:2243] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:58.320394 14429 raft_consensus.cc:2272] T 49fac936bede4d7491681c3d913a1fd0 P 52976c323d7a4e97bf1415f69c8834ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.322026 14429 tablet_replica.cc:333] T f16bebdc1ec542578d9b1dfef1160556 P 52976c323d7a4e97bf1415f69c8834ad: stopping tablet replica
I20260709 13:17:58.322465 14429 raft_consensus.cc:2243] T f16bebdc1ec542578d9b1dfef1160556 P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:58.322878 14429 raft_consensus.cc:2272] T f16bebdc1ec542578d9b1dfef1160556 P 52976c323d7a4e97bf1415f69c8834ad [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.324456 14429 tablet_replica.cc:333] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad: stopping tablet replica
I20260709 13:17:58.324954 14429 raft_consensus.cc:2243] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:58.325359 14429 raft_consensus.cc:2272] T 5ee7c7dfa5424b0bb683bb62badf9658 P 52976c323d7a4e97bf1415f69c8834ad [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.353350 14429 tablet_server.cc:196] TabletServer@127.14.23.67:0 shutdown complete.
I20260709 13:17:58.371881 14429 master.cc:562] Master@127.14.23.126:43765 shutting down...
I20260709 13:17:58.390884 14429 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:58.391388 14429 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:58.391717 14429 tablet_replica.cc:333] T 00000000000000000000000000000000 P 1b56247107e2429391d454af42a0f55f: stopping tablet replica
I20260709 13:17:58.410497 14429 master.cc:584] Master@127.14.23.126:43765 shutdown complete.
11b6d8a2d2034e7d8e53e9df74226ac3  4
4d322b2e1af04b48b88b588e012740d7  5
52976c323d7a4e97bf1415f69c8834ad  1
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10445 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:17:58.446982 14429 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.23.126:39251
I20260709 13:17:58.448101 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:58.454074 15402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:58.454104 15401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:58.454764 15404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:58.455183 14429 server_base.cc:1061] running on GCE node
I20260709 13:17:58.455932 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:58.456108 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:58.456228 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603078456216 us; error 0 us; skew 500 ppm
I20260709 13:17:58.456681 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:58.459074 14429 webserver.cc:533] Webserver started at http://127.14.23.126:40755/ using document root <none> and password file <none>
I20260709 13:17:58.459524 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:58.459684 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:58.459897 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:58.460965 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/master-0-root/instance:
uuid: "88b9585660f045bda650d7ffa0d7a0b7"
format_stamp: "Formatted at 2026-07-09 13:17:58 on dist-test-slave-twc1"
I20260709 13:17:58.465323 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:17:58.468698 15409 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:58.469416 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:17:58.469666 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/master-0-root
uuid: "88b9585660f045bda650d7ffa0d7a0b7"
format_stamp: "Formatted at 2026-07-09 13:17:58 on dist-test-slave-twc1"
I20260709 13:17:58.469941 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:58.482654 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:58.483815 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:58.521256 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.126:39251
I20260709 13:17:58.521355 15460 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.126:39251 every 8 connection(s)
I20260709 13:17:58.525050 15461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.534452 15461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7: Bootstrap starting.
I20260709 13:17:58.538897 15461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:58.543045 15461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7: No bootstrap required, opened a new log
I20260709 13:17:58.545112 15461 raft_consensus.cc:359] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88b9585660f045bda650d7ffa0d7a0b7" member_type: VOTER }
I20260709 13:17:58.545487 15461 raft_consensus.cc:385] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:58.545660 15461 raft_consensus.cc:740] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88b9585660f045bda650d7ffa0d7a0b7, State: Initialized, Role: FOLLOWER
I20260709 13:17:58.546160 15461 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88b9585660f045bda650d7ffa0d7a0b7" member_type: VOTER }
I20260709 13:17:58.546562 15461 raft_consensus.cc:399] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:17:58.546759 15461 raft_consensus.cc:493] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:17:58.546974 15461 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:58.551033 15461 raft_consensus.cc:515] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88b9585660f045bda650d7ffa0d7a0b7" member_type: VOTER }
I20260709 13:17:58.551510 15461 leader_election.cc:304] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88b9585660f045bda650d7ffa0d7a0b7; no voters: 
I20260709 13:17:58.552599 15461 leader_election.cc:290] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:17:58.552980 15464 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:58.554243 15464 raft_consensus.cc:697] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7 [term 1 LEADER]: Becoming Leader. State: Replica: 88b9585660f045bda650d7ffa0d7a0b7, State: Running, Role: LEADER
I20260709 13:17:58.554916 15464 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88b9585660f045bda650d7ffa0d7a0b7" member_type: VOTER }
I20260709 13:17:58.555382 15461 sys_catalog.cc:565] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:17:58.558967 15465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88b9585660f045bda650d7ffa0d7a0b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88b9585660f045bda650d7ffa0d7a0b7" member_type: VOTER } }
I20260709 13:17:58.559158 15466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88b9585660f045bda650d7ffa0d7a0b7. Latest consensus state: current_term: 1 leader_uuid: "88b9585660f045bda650d7ffa0d7a0b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88b9585660f045bda650d7ffa0d7a0b7" member_type: VOTER } }
I20260709 13:17:58.559556 15465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7 [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:58.559612 15466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7 [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:58.561703 15470 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:17:58.566619 15470 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:17:58.571130 14429 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:17:58.575199 15470 catalog_manager.cc:1383] Generated new cluster ID: 35ce996e75464afdad9222bf72bfc541
I20260709 13:17:58.575479 15470 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:17:58.594884 15470 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:17:58.596086 15470 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:17:58.607761 15470 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7: Generated new TSK 0
I20260709 13:17:58.608322 15470 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:17:58.638854 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:58.644757 15482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:58.645622 15483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:58.648059 14429 server_base.cc:1061] running on GCE node
W20260709 13:17:58.648034 15485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:58.649106 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:58.649307 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:58.649456 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603078649438 us; error 0 us; skew 500 ppm
I20260709 13:17:58.649998 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:58.652235 14429 webserver.cc:533] Webserver started at http://127.14.23.65:42385/ using document root <none> and password file <none>
I20260709 13:17:58.652700 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:58.652879 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:58.653131 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:58.654248 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-0-root/instance:
uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
format_stamp: "Formatted at 2026-07-09 13:17:58 on dist-test-slave-twc1"
I20260709 13:17:58.658681 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:17:58.661861 15490 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:58.662690 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:17:58.662962 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-0-root
uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
format_stamp: "Formatted at 2026-07-09 13:17:58 on dist-test-slave-twc1"
I20260709 13:17:58.663240 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:58.678187 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:58.679416 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:58.680892 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:58.683234 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:58.683434 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:58.683655 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:58.683823 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:58.723003 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.65:42481
I20260709 13:17:58.723098 15552 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.65:42481 every 8 connection(s)
I20260709 13:17:58.727732 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:58.734093 15557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:58.735888 15558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:58.737713 15560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:58.738049 14429 server_base.cc:1061] running on GCE node
I20260709 13:17:58.738934 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:58.739116 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:58.739243 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603078739232 us; error 0 us; skew 500 ppm
I20260709 13:17:58.739787 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:58.742468 14429 webserver.cc:533] Webserver started at http://127.14.23.66:42377/ using document root <none> and password file <none>
I20260709 13:17:58.742949 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:58.743032 15553 heartbeater.cc:344] Connected to a master server at 127.14.23.126:39251
I20260709 13:17:58.743188 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:58.743419 15553 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:58.743527 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:58.744297 15553 heartbeater.cc:507] Master 127.14.23.126:39251 requested a full tablet report, sending...
I20260709 13:17:58.745082 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-1-root/instance:
uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
format_stamp: "Formatted at 2026-07-09 13:17:58 on dist-test-slave-twc1"
I20260709 13:17:58.746582 15426 ts_manager.cc:194] Registered new tserver with Master: d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:17:58.748272 15426 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49132
I20260709 13:17:58.749866 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.003s
I20260709 13:17:58.753365 15565 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:58.754089 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:17:58.754346 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-1-root
uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
format_stamp: "Formatted at 2026-07-09 13:17:58 on dist-test-slave-twc1"
I20260709 13:17:58.754585 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:58.773030 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:58.774243 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:58.775616 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:58.777806 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:58.778012 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:58.778275 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:58.778414 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:58.816151 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.66:41593
I20260709 13:17:58.816246 15627 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.66:41593 every 8 connection(s)
I20260709 13:17:58.820770 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:58.827634 15631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:58.829289 15632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:58.832362 14429 server_base.cc:1061] running on GCE node
W20260709 13:17:58.832870 15634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:58.833320 15628 heartbeater.cc:344] Connected to a master server at 127.14.23.126:39251
I20260709 13:17:58.833673 15628 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:58.833811 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:58.834100 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:58.834266 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603078834250 us; error 0 us; skew 500 ppm
I20260709 13:17:58.834487 15628 heartbeater.cc:507] Master 127.14.23.126:39251 requested a full tablet report, sending...
I20260709 13:17:58.834865 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:58.836551 15426 ts_manager.cc:194] Registered new tserver with Master: 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:17:58.837396 14429 webserver.cc:533] Webserver started at http://127.14.23.67:44151/ using document root <none> and password file <none>
I20260709 13:17:58.837875 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:58.837903 15426 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49140
I20260709 13:17:58.838158 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:58.838481 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:58.839691 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-2-root/instance:
uuid: "3ef186ec12554a1d87814af0a0ace9a4"
format_stamp: "Formatted at 2026-07-09 13:17:58 on dist-test-slave-twc1"
I20260709 13:17:58.844095 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:17:58.847415 15639 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:58.848137 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:17:58.848421 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-2-root
uuid: "3ef186ec12554a1d87814af0a0ace9a4"
format_stamp: "Formatted at 2026-07-09 13:17:58 on dist-test-slave-twc1"
I20260709 13:17:58.848696 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:58.882253 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:58.883467 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:58.884927 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:58.887179 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:58.887382 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:58.887602 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:58.887758 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:58.925879 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.67:32779
I20260709 13:17:58.926040 15701 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.67:32779 every 8 connection(s)
I20260709 13:17:58.939258 15702 heartbeater.cc:344] Connected to a master server at 127.14.23.126:39251
I20260709 13:17:58.939610 15702 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:58.940317 15702 heartbeater.cc:507] Master 127.14.23.126:39251 requested a full tablet report, sending...
I20260709 13:17:58.942291 15426 ts_manager.cc:194] Registered new tserver with Master: 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:17:58.942417 14429 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013528982s
I20260709 13:17:58.943634 15426 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49144
I20260709 13:17:58.968627 15426 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49148:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:58.975386 15426 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:59.166296 15518 tablet_service.cc:1511] Processing CreateTablet for tablet 12b56c6369484d92b328d276d4fca9e3 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:17:59.167765 15518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12b56c6369484d92b328d276d4fca9e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.171010 15517 tablet_service.cc:1511] Processing CreateTablet for tablet b9a4c0b115544577a855e9c7383d9a48 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:17:59.172420 15517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9a4c0b115544577a855e9c7383d9a48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.173902 15662 tablet_service.cc:1511] Processing CreateTablet for tablet 40f71e7bfe1a4575846bcddbb0b9fd55 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:17:59.175527 15662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40f71e7bfe1a4575846bcddbb0b9fd55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.175926 15664 tablet_service.cc:1511] Processing CreateTablet for tablet 271b5d375f0145b78b28a0c13328ce7f (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:17:59.177311 15664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 271b5d375f0145b78b28a0c13328ce7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.178779 15661 tablet_service.cc:1511] Processing CreateTablet for tablet 4344539a12c94917991c2f39d3ff5858 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:17:59.174026 15663 tablet_service.cc:1511] Processing CreateTablet for tablet 4b73eb6a0096455b87f6109b059cf0ee (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:17:59.179212 15665 tablet_service.cc:1511] Processing CreateTablet for tablet 974a060267d24f00888c0942dfbca77e (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:17:59.180215 15661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4344539a12c94917991c2f39d3ff5858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.181130 15666 tablet_service.cc:1511] Processing CreateTablet for tablet b9a4c0b115544577a855e9c7383d9a48 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:17:59.182655 15665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 974a060267d24f00888c0942dfbca77e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.183156 15667 tablet_service.cc:1511] Processing CreateTablet for tablet 12b56c6369484d92b328d276d4fca9e3 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:17:59.184815 15660 tablet_service.cc:1511] Processing CreateTablet for tablet bd36c2cfd99c45cea9e48096d54bed1a (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:17:59.186432 15515 tablet_service.cc:1511] Processing CreateTablet for tablet 271b5d375f0145b78b28a0c13328ce7f (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:17:59.187865 15515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 271b5d375f0145b78b28a0c13328ce7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.191184 15516 tablet_service.cc:1511] Processing CreateTablet for tablet 974a060267d24f00888c0942dfbca77e (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:17:59.192592 15516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 974a060267d24f00888c0942dfbca77e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.193122 15659 tablet_service.cc:1511] Processing CreateTablet for tablet 68dd26481a1b4442b702cb5976b1fb41 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:17:59.198164 15658 tablet_service.cc:1511] Processing CreateTablet for tablet 1c436bd48e6b4ad9a1bb596e2ecb15fc (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:17:59.199774 15513 tablet_service.cc:1511] Processing CreateTablet for tablet 40f71e7bfe1a4575846bcddbb0b9fd55 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:17:59.201226 15513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40f71e7bfe1a4575846bcddbb0b9fd55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.201457 15514 tablet_service.cc:1511] Processing CreateTablet for tablet 4b73eb6a0096455b87f6109b059cf0ee (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:17:59.203033 15514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b73eb6a0096455b87f6109b059cf0ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.204317 15717 tablet_bootstrap.cc:492] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:17:59.205463 15510 tablet_service.cc:1511] Processing CreateTablet for tablet 68dd26481a1b4442b702cb5976b1fb41 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:17:59.203042 15512 tablet_service.cc:1511] Processing CreateTablet for tablet 4344539a12c94917991c2f39d3ff5858 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:17:59.206959 15510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68dd26481a1b4442b702cb5976b1fb41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.207582 15512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4344539a12c94917991c2f39d3ff5858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.204262 15511 tablet_service.cc:1511] Processing CreateTablet for tablet bd36c2cfd99c45cea9e48096d54bed1a (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:17:59.208985 15509 tablet_service.cc:1511] Processing CreateTablet for tablet 1c436bd48e6b4ad9a1bb596e2ecb15fc (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:17:59.184757 15667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12b56c6369484d92b328d276d4fca9e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.209415 15511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd36c2cfd99c45cea9e48096d54bed1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.210556 15663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b73eb6a0096455b87f6109b059cf0ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.212263 15509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c436bd48e6b4ad9a1bb596e2ecb15fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.212806 15659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68dd26481a1b4442b702cb5976b1fb41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.214254 15666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9a4c0b115544577a855e9c7383d9a48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.217582 15660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd36c2cfd99c45cea9e48096d54bed1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.222287 15658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c436bd48e6b4ad9a1bb596e2ecb15fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.238714 15593 tablet_service.cc:1511] Processing CreateTablet for tablet 12b56c6369484d92b328d276d4fca9e3 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:17:59.240159 15589 tablet_service.cc:1511] Processing CreateTablet for tablet 4b73eb6a0096455b87f6109b059cf0ee (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:17:59.240206 15593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12b56c6369484d92b328d276d4fca9e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.242657 15591 tablet_service.cc:1511] Processing CreateTablet for tablet 974a060267d24f00888c0942dfbca77e (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:17:59.244056 15589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b73eb6a0096455b87f6109b059cf0ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.245023 15590 tablet_service.cc:1511] Processing CreateTablet for tablet 271b5d375f0145b78b28a0c13328ce7f (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:17:59.246511 15590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 271b5d375f0145b78b28a0c13328ce7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.247677 15588 tablet_service.cc:1511] Processing CreateTablet for tablet 40f71e7bfe1a4575846bcddbb0b9fd55 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:17:59.249075 15588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40f71e7bfe1a4575846bcddbb0b9fd55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.241443 15592 tablet_service.cc:1511] Processing CreateTablet for tablet b9a4c0b115544577a855e9c7383d9a48 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:17:59.251847 15592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9a4c0b115544577a855e9c7383d9a48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.251700 15587 tablet_service.cc:1511] Processing CreateTablet for tablet 4344539a12c94917991c2f39d3ff5858 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:17:59.254596 15586 tablet_service.cc:1511] Processing CreateTablet for tablet bd36c2cfd99c45cea9e48096d54bed1a (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:17:59.256889 15662 tablet_service.cc:1511] Processing CreateTablet for tablet b2c57bed49da41d795c8b21ed4461f1f (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:17:59.257153 15585 tablet_service.cc:1511] Processing CreateTablet for tablet 68dd26481a1b4442b702cb5976b1fb41 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:17:59.258203 15662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2c57bed49da41d795c8b21ed4461f1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.258818 15587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4344539a12c94917991c2f39d3ff5858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.262284 15585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68dd26481a1b4442b702cb5976b1fb41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.264061 15586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd36c2cfd99c45cea9e48096d54bed1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.268904 15591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 974a060267d24f00888c0942dfbca77e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.286099 15664 tablet_service.cc:1511] Processing CreateTablet for tablet b87d5c4641304e03a8a3fd4782b043e5 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:17:59.287559 15664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b87d5c4641304e03a8a3fd4782b043e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.291359 15584 tablet_service.cc:1511] Processing CreateTablet for tablet 1c436bd48e6b4ad9a1bb596e2ecb15fc (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:17:59.292819 15584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c436bd48e6b4ad9a1bb596e2ecb15fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.293015 15515 tablet_service.cc:1511] Processing CreateTablet for tablet b2c57bed49da41d795c8b21ed4461f1f (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:17:59.294322 15515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2c57bed49da41d795c8b21ed4461f1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.305114 15516 tablet_service.cc:1511] Processing CreateTablet for tablet b87d5c4641304e03a8a3fd4782b043e5 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:17:59.306492 15516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b87d5c4641304e03a8a3fd4782b043e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.316980 15513 tablet_service.cc:1511] Processing CreateTablet for tablet d481434f60dc4276831163b535fb8159 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:17:59.318296 15513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d481434f60dc4276831163b535fb8159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.323747 15662 tablet_service.cc:1511] Processing CreateTablet for tablet d481434f60dc4276831163b535fb8159 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:17:59.324544 15717 tablet_bootstrap.cc:654] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.325101 15662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d481434f60dc4276831163b535fb8159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.331072 15718 tablet_bootstrap.cc:492] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:17:59.336140 15518 tablet_service.cc:1511] Processing CreateTablet for tablet ced58b89726545d3af99de36a1aa1f30 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:17:59.337492 15518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ced58b89726545d3af99de36a1aa1f30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.367403 15661 tablet_service.cc:1511] Processing CreateTablet for tablet ced58b89726545d3af99de36a1aa1f30 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:17:59.368775 15661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ced58b89726545d3af99de36a1aa1f30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.367430 15509 tablet_service.cc:1511] Processing CreateTablet for tablet f25e63ed82fa4e008109b74fe8df71ba (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:17:59.369854 15509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f25e63ed82fa4e008109b74fe8df71ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.372974 15718 tablet_bootstrap.cc:654] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.383194 15585 tablet_service.cc:1511] Processing CreateTablet for tablet b2c57bed49da41d795c8b21ed4461f1f (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:17:59.387418 15585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2c57bed49da41d795c8b21ed4461f1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.387233 15514 tablet_service.cc:1511] Processing CreateTablet for tablet 6db63c1e295d42d9938e0ef858365436 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:17:59.387593 15588 tablet_service.cc:1511] Processing CreateTablet for tablet b87d5c4641304e03a8a3fd4782b043e5 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:17:59.388698 15514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6db63c1e295d42d9938e0ef858365436. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.388880 15588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b87d5c4641304e03a8a3fd4782b043e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.387262 15590 tablet_service.cc:1511] Processing CreateTablet for tablet d481434f60dc4276831163b535fb8159 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:17:59.393963 15590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d481434f60dc4276831163b535fb8159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.395411 15717 tablet_bootstrap.cc:492] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:17:59.395983 15717 ts_tablet_manager.cc:1403] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.192s	user 0.006s	sys 0.006s
I20260709 13:17:59.399242 15717 raft_consensus.cc:359] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:17:59.400074 15717 raft_consensus.cc:385] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.400359 15717 raft_consensus.cc:740] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.401289 15717 consensus_queue.cc:260] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:17:59.404554 15584 tablet_service.cc:1511] Processing CreateTablet for tablet f25e63ed82fa4e008109b74fe8df71ba (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:17:59.406308 15584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f25e63ed82fa4e008109b74fe8df71ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.406790 15658 tablet_service.cc:1511] Processing CreateTablet for tablet f25e63ed82fa4e008109b74fe8df71ba (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:17:59.408270 15658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f25e63ed82fa4e008109b74fe8df71ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.411813 15586 tablet_service.cc:1511] Processing CreateTablet for tablet 6db63c1e295d42d9938e0ef858365436 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:17:59.413228 15586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6db63c1e295d42d9938e0ef858365436. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.413587 15667 tablet_service.cc:1511] Processing CreateTablet for tablet 6db63c1e295d42d9938e0ef858365436 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:17:59.419577 15720 tablet_bootstrap.cc:492] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:17:59.422648 15593 tablet_service.cc:1511] Processing CreateTablet for tablet ced58b89726545d3af99de36a1aa1f30 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:17:59.423997 15593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ced58b89726545d3af99de36a1aa1f30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.436757 15666 tablet_service.cc:1511] Processing CreateTablet for tablet d7b7542687ad43acb128a39bb1f40829 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:17:59.421711 15667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6db63c1e295d42d9938e0ef858365436. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.441365 15515 tablet_service.cc:1511] Processing CreateTablet for tablet d7b7542687ad43acb128a39bb1f40829 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:17:59.442713 15718 tablet_bootstrap.cc:492] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:17:59.455129 15718 ts_tablet_manager.cc:1403] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.124s	user 0.010s	sys 0.025s
I20260709 13:17:59.458170 15718 raft_consensus.cc:359] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:17:59.458904 15718 raft_consensus.cc:385] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.466430 15666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7b7542687ad43acb128a39bb1f40829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.479911 15513 tablet_service.cc:1511] Processing CreateTablet for tablet a3d461474623405ab8ee3ca7fdff1722 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:17:59.481240 15589 tablet_service.cc:1511] Processing CreateTablet for tablet a3d461474623405ab8ee3ca7fdff1722 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:17:59.482548 15589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3d461474623405ab8ee3ca7fdff1722. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.483669 15518 tablet_service.cc:1511] Processing CreateTablet for tablet 2afa8e112a744f218227526d861b98c3 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:17:59.486449 15515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7b7542687ad43acb128a39bb1f40829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.487142 15518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2afa8e112a744f218227526d861b98c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.459223 15718 raft_consensus.cc:740] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.488797 15513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3d461474623405ab8ee3ca7fdff1722. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.488353 15718 consensus_queue.cc:260] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:17:59.496922 15702 heartbeater.cc:499] Master 127.14.23.126:39251 was elected leader, sending a full tablet report...
I20260709 13:17:59.506376 15717 ts_tablet_manager.cc:1434] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.110s	user 0.033s	sys 0.037s
I20260709 13:17:59.507463 15717 tablet_bootstrap.cc:492] T 271b5d375f0145b78b28a0c13328ce7f P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:17:59.504292 15723 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.509232 15723 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:17:59.510095 15516 tablet_service.cc:1511] Processing CreateTablet for tablet 1fa44270fe124591b1e8c82c82b89efe (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:17:59.531967 15516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fa44270fe124591b1e8c82c82b89efe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.532022 15665 tablet_service.cc:1511] Processing CreateTablet for tablet 1fa44270fe124591b1e8c82c82b89efe (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:17:59.533315 15665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fa44270fe124591b1e8c82c82b89efe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.533427 15723 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:17:59.534667 15660 tablet_service.cc:1511] Processing CreateTablet for tablet 2afa8e112a744f218227526d861b98c3 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:17:59.535943 15660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2afa8e112a744f218227526d861b98c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.469318 15720 tablet_bootstrap.cc:654] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.515844 15585 tablet_service.cc:1511] Processing CreateTablet for tablet 2afa8e112a744f218227526d861b98c3 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:17:59.541334 15585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2afa8e112a744f218227526d861b98c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.531461 15664 tablet_service.cc:1511] Processing CreateTablet for tablet a3d461474623405ab8ee3ca7fdff1722 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:17:59.545591 15664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3d461474623405ab8ee3ca7fdff1722. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.526081 15593 tablet_service.cc:1511] Processing CreateTablet for tablet 1fa44270fe124591b1e8c82c82b89efe (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:17:59.555543 15591 tablet_service.cc:1511] Processing CreateTablet for tablet d7b7542687ad43acb128a39bb1f40829 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:17:59.559934 15509 tablet_service.cc:1511] Processing CreateTablet for tablet 83022a40fade44a2988f3de3b6b36aec (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:17:59.561321 15509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83022a40fade44a2988f3de3b6b36aec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.555457 15593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fa44270fe124591b1e8c82c82b89efe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.577507 15591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7b7542687ad43acb128a39bb1f40829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.579674 15717 tablet_bootstrap.cc:654] T 271b5d375f0145b78b28a0c13328ce7f P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:59.584816 15474 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:59.614399 15586 tablet_service.cc:1511] Processing CreateTablet for tablet 340e065cf2744fde9d65230a8ea91e14 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:17:59.614773 15590 tablet_service.cc:1511] Processing CreateTablet for tablet 83022a40fade44a2988f3de3b6b36aec (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:17:59.614905 15518 tablet_service.cc:1511] Processing CreateTablet for tablet 340e065cf2744fde9d65230a8ea91e14 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:17:59.615860 15586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 340e065cf2744fde9d65230a8ea91e14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.616328 15518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 340e065cf2744fde9d65230a8ea91e14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.617892 15590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83022a40fade44a2988f3de3b6b36aec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.645608 15513 tablet_service.cc:1511] Processing CreateTablet for tablet 4da1202d21704a61a455cc91fa7c4022 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:17:59.646989 15513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4da1202d21704a61a455cc91fa7c4022. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.682253 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:17:59.684171 15641 leader_election.cc:343] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:59.698534 15720 tablet_bootstrap.cc:492] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:17:59.702239 15553 heartbeater.cc:499] Master 127.14.23.126:39251 was elected leader, sending a full tablet report...
I20260709 13:17:59.714596 15665 tablet_service.cc:1511] Processing CreateTablet for tablet b99eaeae24b44aa78cfa8c90caca288e (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:17:59.715066 15720 ts_tablet_manager.cc:1403] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.296s	user 0.002s	sys 0.021s
I20260709 13:17:59.715889 15665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b99eaeae24b44aa78cfa8c90caca288e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.716832 15717 tablet_bootstrap.cc:492] T 271b5d375f0145b78b28a0c13328ce7f P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:17:59.702517 15516 tablet_service.cc:1511] Processing CreateTablet for tablet 38190179f6494b5591027e1c41f83176 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:17:59.703578 15658 tablet_service.cc:1511] Processing CreateTablet for tablet 83022a40fade44a2988f3de3b6b36aec (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:17:59.717311 15717 ts_tablet_manager.cc:1403] T 271b5d375f0145b78b28a0c13328ce7f P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.210s	user 0.014s	sys 0.005s
I20260709 13:17:59.718113 15658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83022a40fade44a2988f3de3b6b36aec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.718539 15720 raft_consensus.cc:359] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:17:59.719307 15720 raft_consensus.cc:385] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.719658 15720 raft_consensus.cc:740] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.720482 15720 consensus_queue.cc:260] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:17:59.720151 15717 raft_consensus.cc:359] T 271b5d375f0145b78b28a0c13328ce7f P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:17:59.723225 15717 raft_consensus.cc:385] T 271b5d375f0145b78b28a0c13328ce7f P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.723532 15717 raft_consensus.cc:740] T 271b5d375f0145b78b28a0c13328ce7f P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.723737 15516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38190179f6494b5591027e1c41f83176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.703965 15585 tablet_service.cc:1511] Processing CreateTablet for tablet 4da1202d21704a61a455cc91fa7c4022 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:17:59.726697 15585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4da1202d21704a61a455cc91fa7c4022. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.727008 15593 tablet_service.cc:1511] Processing CreateTablet for tablet 01ad9d7b8dba45be8fdf625f27e8d273 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:17:59.728282 15593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01ad9d7b8dba45be8fdf625f27e8d273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.705197 15718 ts_tablet_manager.cc:1434] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.250s	user 0.008s	sys 0.008s
I20260709 13:17:59.730849 15718 tablet_bootstrap.cc:492] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:17:59.706489 15509 tablet_service.cc:1511] Processing CreateTablet for tablet b99eaeae24b44aa78cfa8c90caca288e (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:17:59.702527 15586 tablet_service.cc:1511] Processing CreateTablet for tablet 38190179f6494b5591027e1c41f83176 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:17:59.734745 15586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38190179f6494b5591027e1c41f83176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.707430 15666 tablet_service.cc:1511] Processing CreateTablet for tablet 38190179f6494b5591027e1c41f83176 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:17:59.708269 15664 tablet_service.cc:1511] Processing CreateTablet for tablet 340e065cf2744fde9d65230a8ea91e14 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:17:59.709124 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:17:59.734745 15509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b99eaeae24b44aa78cfa8c90caca288e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.706489 15660 tablet_service.cc:1511] Processing CreateTablet for tablet 4da1202d21704a61a455cc91fa7c4022 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:17:59.712927 15591 tablet_service.cc:1511] Processing CreateTablet for tablet b99eaeae24b44aa78cfa8c90caca288e (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:17:59.737759 15718 tablet_bootstrap.cc:654] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.738410 15591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b99eaeae24b44aa78cfa8c90caca288e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:17:59.738541 15642 leader_election.cc:343] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:59.739001 15642 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:17:59.742254 15513 tablet_service.cc:1511] Processing CreateTablet for tablet 01ad9d7b8dba45be8fdf625f27e8d273 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:17:59.742851 15722 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:59.743537 15513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01ad9d7b8dba45be8fdf625f27e8d273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.743683 15724 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.744109 15724 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:17:59.746289 15666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38190179f6494b5591027e1c41f83176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.746686 15724 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:17:59.748988 15660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4da1202d21704a61a455cc91fa7c4022. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.750584 15664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 340e065cf2744fde9d65230a8ea91e14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.769480 15518 tablet_service.cc:1511] Processing CreateTablet for tablet 7a5315557f994ec689bb9a79498bb2d5 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:17:59.770960 15518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a5315557f994ec689bb9a79498bb2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.771569 15584 tablet_service.cc:1511] Processing CreateTablet for tablet 7a5315557f994ec689bb9a79498bb2d5 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:17:59.772871 15584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a5315557f994ec689bb9a79498bb2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.795791 15720 ts_tablet_manager.cc:1434] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.080s	user 0.013s	sys 0.015s
I20260709 13:17:59.793869 15717 consensus_queue.cc:260] T 271b5d375f0145b78b28a0c13328ce7f P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:17:59.798138 15717 ts_tablet_manager.cc:1434] T 271b5d375f0145b78b28a0c13328ce7f P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.080s	user 0.002s	sys 0.008s
I20260709 13:17:59.799113 15717 tablet_bootstrap.cc:492] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:17:59.792754 15628 heartbeater.cc:499] Master 127.14.23.126:39251 was elected leader, sending a full tablet report...
I20260709 13:17:59.811925 15720 tablet_bootstrap.cc:492] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:17:59.820327 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:17:59.822618 15493 leader_election.cc:343] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:59.824584 15720 tablet_bootstrap.cc:654] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.830375 15717 tablet_bootstrap.cc:654] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.831724 15658 tablet_service.cc:1511] Processing CreateTablet for tablet 01ad9d7b8dba45be8fdf625f27e8d273 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:17:59.833532 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:17:59.833719 15666 tablet_service.cc:1511] Processing CreateTablet for tablet 7a5315557f994ec689bb9a79498bb2d5 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:17:59.834723 15658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01ad9d7b8dba45be8fdf625f27e8d273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.835390 15666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a5315557f994ec689bb9a79498bb2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.843433 15718 tablet_bootstrap.cc:492] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:17:59.843951 15718 ts_tablet_manager.cc:1403] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.113s	user 0.006s	sys 0.011s
I20260709 13:17:59.847790 15585 tablet_service.cc:1511] Processing CreateTablet for tablet 3502bb7169aa41bb9ebdaeddf5068af9 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:17:59.849684 15585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3502bb7169aa41bb9ebdaeddf5068af9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:17:59.850515 15491 leader_election.cc:343] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:59.850368 15516 tablet_service.cc:1511] Processing CreateTablet for tablet 5243f722506441269fac66aa989ce36d (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:17:59.851002 15491 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:17:59.851681 15516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5243f722506441269fac66aa989ce36d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.851867 15724 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:59.854694 15718 raft_consensus.cc:359] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:17:59.855429 15718 raft_consensus.cc:385] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.855708 15718 raft_consensus.cc:740] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.858217 15509 tablet_service.cc:1511] Processing CreateTablet for tablet 3502bb7169aa41bb9ebdaeddf5068af9 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:17:59.858590 15718 consensus_queue.cc:260] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:17:59.859742 15509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3502bb7169aa41bb9ebdaeddf5068af9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.862165 15718 ts_tablet_manager.cc:1434] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.002s
I20260709 13:17:59.863129 15718 tablet_bootstrap.cc:492] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:17:59.871464 15513 tablet_service.cc:1511] Processing CreateTablet for tablet 8a81ba00c9bd4a45ae8d2379570072fe (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:17:59.873099 15513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a81ba00c9bd4a45ae8d2379570072fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.878690 15717 tablet_bootstrap.cc:492] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:17:59.879228 15717 ts_tablet_manager.cc:1403] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.080s	user 0.012s	sys 0.012s
I20260709 13:17:59.882093 15717 raft_consensus.cc:359] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:17:59.882797 15717 raft_consensus.cc:385] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.883121 15717 raft_consensus.cc:740] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.883795 15717 consensus_queue.cc:260] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:17:59.885412 15717 ts_tablet_manager.cc:1434] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:17:59.885887 15718 tablet_bootstrap.cc:654] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.886544 15591 tablet_service.cc:1511] Processing CreateTablet for tablet 5243f722506441269fac66aa989ce36d (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:17:59.888027 15591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5243f722506441269fac66aa989ce36d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.887678 15665 tablet_service.cc:1511] Processing CreateTablet for tablet 5243f722506441269fac66aa989ce36d (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:17:59.890609 15665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5243f722506441269fac66aa989ce36d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.882738 15664 tablet_service.cc:1511] Processing CreateTablet for tablet 3502bb7169aa41bb9ebdaeddf5068af9 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:17:59.892819 15664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3502bb7169aa41bb9ebdaeddf5068af9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.902386 15717 tablet_bootstrap.cc:492] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:17:59.902788 15658 tablet_service.cc:1511] Processing CreateTablet for tablet 8a81ba00c9bd4a45ae8d2379570072fe (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:17:59.904063 15658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a81ba00c9bd4a45ae8d2379570072fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.907015 15720 tablet_bootstrap.cc:492] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:17:59.907560 15720 ts_tablet_manager.cc:1403] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.096s	user 0.016s	sys 0.000s
I20260709 13:17:59.909051 15585 tablet_service.cc:1511] Processing CreateTablet for tablet 8a81ba00c9bd4a45ae8d2379570072fe (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:17:59.911566 15717 tablet_bootstrap.cc:654] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.912040 15585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a81ba00c9bd4a45ae8d2379570072fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.919785 15591 tablet_service.cc:1511] Processing CreateTablet for tablet c1e80bd01b0c4006852da32ca0eeb0ae (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:17:59.921128 15591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1e80bd01b0c4006852da32ca0eeb0ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.921685 15720 raft_consensus.cc:359] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:17:59.922468 15720 raft_consensus.cc:385] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.922770 15720 raft_consensus.cc:740] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.928460 15720 consensus_queue.cc:260] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:17:59.930891 15720 ts_tablet_manager.cc:1434] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:17:59.931926 15720 tablet_bootstrap.cc:492] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:17:59.938886 15720 tablet_bootstrap.cc:654] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.940222 15516 tablet_service.cc:1511] Processing CreateTablet for tablet c1e80bd01b0c4006852da32ca0eeb0ae (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:17:59.944309 15516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1e80bd01b0c4006852da32ca0eeb0ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.948385 15658 tablet_service.cc:1511] Processing CreateTablet for tablet c1e80bd01b0c4006852da32ca0eeb0ae (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:17:59.949995 15658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1e80bd01b0c4006852da32ca0eeb0ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.957208 15718 tablet_bootstrap.cc:492] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:17:59.956794 15513 tablet_service.cc:1511] Processing CreateTablet for tablet 582343936346420b87ecf2eebd00dc25 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:17:59.957732 15718 ts_tablet_manager.cc:1403] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.095s	user 0.015s	sys 0.008s
I20260709 13:17:59.958386 15513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 582343936346420b87ecf2eebd00dc25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.960827 15718 raft_consensus.cc:359] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:17:59.961575 15718 raft_consensus.cc:385] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.963580 15729 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.963460 15593 tablet_service.cc:1511] Processing CreateTablet for tablet 582343936346420b87ecf2eebd00dc25 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:17:59.964283 15664 tablet_service.cc:1511] Processing CreateTablet for tablet 582343936346420b87ecf2eebd00dc25 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:17:59.964089 15729 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:17:59.965576 15664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 582343936346420b87ecf2eebd00dc25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.967101 15729 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:17:59.964757 15593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 582343936346420b87ecf2eebd00dc25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.971750 15509 tablet_service.cc:1511] Processing CreateTablet for tablet 98787e13cd7a49528aeeb59c72559e46 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:17:59.973155 15509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98787e13cd7a49528aeeb59c72559e46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.977784 15717 tablet_bootstrap.cc:492] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:17:59.978343 15717 ts_tablet_manager.cc:1403] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.076s	user 0.020s	sys 0.004s
I20260709 13:17:59.984505 15584 tablet_service.cc:1511] Processing CreateTablet for tablet 98787e13cd7a49528aeeb59c72559e46 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:17:59.985885 15584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98787e13cd7a49528aeeb59c72559e46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.961889 15718 raft_consensus.cc:740] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.992513 15718 consensus_queue.cc:260] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:17:59.994733 15718 ts_tablet_manager.cc:1434] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.000s
I20260709 13:17:59.995738 15718 tablet_bootstrap.cc:492] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:00.006283 15717 raft_consensus.cc:359] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.007170 15717 raft_consensus.cc:385] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:00.007468 15717 raft_consensus.cc:740] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:00.010700 15664 tablet_service.cc:1511] Processing CreateTablet for tablet 98787e13cd7a49528aeeb59c72559e46 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:18:00.012035 15664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98787e13cd7a49528aeeb59c72559e46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.018420 15518 tablet_service.cc:1511] Processing CreateTablet for tablet 4c48036212bb4277acf36aac79d8b35e (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:18:00.019755 15518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c48036212bb4277acf36aac79d8b35e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.020328 15720 tablet_bootstrap.cc:492] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:00.020788 15720 ts_tablet_manager.cc:1403] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.089s	user 0.017s	sys 0.026s
I20260709 13:18:00.023185 15717 consensus_queue.cc:260] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.023847 15720 raft_consensus.cc:359] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.024578 15720 raft_consensus.cc:385] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:00.024904 15720 raft_consensus.cc:740] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:00.029022 15717 ts_tablet_manager.cc:1434] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.000s
I20260709 13:18:00.030076 15717 tablet_bootstrap.cc:492] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:00.031163 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:00.032188 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:00.034487 15567 leader_election.cc:343] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:18:00.036101 15566 leader_election.cc:343] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:00.042419 15724 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.042799 15566 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:00.042945 15724 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.043745 15717 tablet_bootstrap.cc:654] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:00.043992 15729 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:00.045042 15724 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:00.045804 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:00.035795 15665 tablet_service.cc:1511] Processing CreateTablet for tablet 4c48036212bb4277acf36aac79d8b35e (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:18:00.047340 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:00.047988 15677 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:00.049247 15491 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:18:00.050220 15493 leader_election.cc:343] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:00.050442 15720 consensus_queue.cc:260] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.051129 15724 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:00.051494 15724 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.042451 15591 tablet_service.cc:1511] Processing CreateTablet for tablet 4c48036212bb4277acf36aac79d8b35e (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:18:00.055791 15724 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:00.054714 15593 tablet_service.cc:1511] Processing CreateTablet for tablet 50726e0d396948b79be43aece292acb5 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:18:00.051383 15665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c48036212bb4277acf36aac79d8b35e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.057029 15718 tablet_bootstrap.cc:654] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:00.059618 15658 tablet_service.cc:1511] Processing CreateTablet for tablet 50726e0d396948b79be43aece292acb5 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:18:00.061501 15658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50726e0d396948b79be43aece292acb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.063407 15593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50726e0d396948b79be43aece292acb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.068420 15720 ts_tablet_manager.cc:1434] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.047s	user 0.006s	sys 0.002s
I20260709 13:18:00.075083 15585 tablet_service.cc:1511] Processing CreateTablet for tablet 2a5f6755312b4dc09781ecf8a9468c17 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:18:00.054898 15509 tablet_service.cc:1511] Processing CreateTablet for tablet 50726e0d396948b79be43aece292acb5 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:18:00.076541 15585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a5f6755312b4dc09781ecf8a9468c17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.076751 15509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50726e0d396948b79be43aece292acb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.074461 15724 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.077306 15720 tablet_bootstrap.cc:492] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:00.084059 15666 tablet_service.cc:1511] Processing CreateTablet for tablet 2a5f6755312b4dc09781ecf8a9468c17 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:18:00.085387 15516 tablet_service.cc:1511] Processing CreateTablet for tablet ccb13c2addbf41569d0c2c23750bd4c9 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:18:00.087312 15516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccb13c2addbf41569d0c2c23750bd4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.085829 15666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a5f6755312b4dc09781ecf8a9468c17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.090912 15513 tablet_service.cc:1511] Processing CreateTablet for tablet 2a5f6755312b4dc09781ecf8a9468c17 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:18:00.091279 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:00.091717 15584 tablet_service.cc:1511] Processing CreateTablet for tablet ccb13c2addbf41569d0c2c23750bd4c9 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:18:00.092163 15513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a5f6755312b4dc09781ecf8a9468c17. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:18:00.093307 15493 leader_election.cc:343] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:00.090072 15591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c48036212bb4277acf36aac79d8b35e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.090123 15660 tablet_service.cc:1511] Processing CreateTablet for tablet ccb13c2addbf41569d0c2c23750bd4c9 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:18:00.094645 15584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccb13c2addbf41569d0c2c23750bd4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.094285 15724 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:00.094969 15660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccb13c2addbf41569d0c2c23750bd4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.085752 15717 tablet_bootstrap.cc:492] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:00.097415 15717 ts_tablet_manager.cc:1403] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.068s	user 0.025s	sys 0.015s
I20260709 13:18:00.098606 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:00.106237 15677 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:00.108741 15515 tablet_service.cc:1511] Processing CreateTablet for tablet 4b9f476ac2ad44cd8124fde4f1aadc78 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:18:00.109347 15720 tablet_bootstrap.cc:654] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:00.116904 15717 raft_consensus.cc:359] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.117878 15724 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.118371 15724 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.118669 15718 tablet_bootstrap.cc:492] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:00.121241 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:18:00.122617 15493 leader_election.cc:343] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:00.124882 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:00.126375 15491 leader_election.cc:343] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:00.126456 15724 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:00.127166 15491 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:00.127923 15724 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:00.126313 15718 ts_tablet_manager.cc:1403] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.131s	user 0.008s	sys 0.008s
I20260709 13:18:00.133559 15515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b9f476ac2ad44cd8124fde4f1aadc78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.128800 15677 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:00.139446 15491 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:00.140666 15724 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:00.117681 15717 raft_consensus.cc:385] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:00.153003 15718 raft_consensus.cc:359] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.146231 15717 raft_consensus.cc:740] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:00.154438 15717 consensus_queue.cc:260] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.156213 15585 tablet_service.cc:1511] Processing CreateTablet for tablet 4b9f476ac2ad44cd8124fde4f1aadc78 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:18:00.157537 15585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b9f476ac2ad44cd8124fde4f1aadc78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.158982 15717 ts_tablet_manager.cc:1434] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.053s	user 0.006s	sys 0.000s
I20260709 13:18:00.160972 15717 tablet_bootstrap.cc:492] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:00.168924 15665 tablet_service.cc:1511] Processing CreateTablet for tablet 4b9f476ac2ad44cd8124fde4f1aadc78 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:18:00.170398 15665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b9f476ac2ad44cd8124fde4f1aadc78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.171765 15591 tablet_service.cc:1511] Processing CreateTablet for tablet 2c0b73c09a0849518f4bc6e3886a348e (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:18:00.173324 15591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c0b73c09a0849518f4bc6e3886a348e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.174198 15509 tablet_service.cc:1511] Processing CreateTablet for tablet 2c0b73c09a0849518f4bc6e3886a348e (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:18:00.175562 15509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c0b73c09a0849518f4bc6e3886a348e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.179116 15720 tablet_bootstrap.cc:492] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:00.179657 15720 ts_tablet_manager.cc:1403] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.103s	user 0.006s	sys 0.005s
I20260709 13:18:00.182715 15724 raft_consensus.cc:697] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:00.183629 15724 consensus_queue.cc:237] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.188297 15717 tablet_bootstrap.cc:654] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:00.153818 15718 raft_consensus.cc:385] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:00.194094 15718 raft_consensus.cc:740] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:00.194921 15718 consensus_queue.cc:260] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.197118 15718 ts_tablet_manager.cc:1434] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.067s	user 0.005s	sys 0.001s
I20260709 13:18:00.202338 15509 tablet_service.cc:1511] Processing CreateTablet for tablet db313a4cd18a47a2b72587d04ed6e747 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:18:00.202672 15665 tablet_service.cc:1511] Processing CreateTablet for tablet 2c0b73c09a0849518f4bc6e3886a348e (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:18:00.203238 15717 tablet_bootstrap.cc:492] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:00.203788 15509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db313a4cd18a47a2b72587d04ed6e747. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.203771 15717 ts_tablet_manager.cc:1403] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.043s	user 0.010s	sys 0.002s
I20260709 13:18:00.205039 15591 tablet_service.cc:1511] Processing CreateTablet for tablet db313a4cd18a47a2b72587d04ed6e747 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:18:00.203953 15665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c0b73c09a0849518f4bc6e3886a348e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.206933 15720 raft_consensus.cc:359] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.207671 15720 raft_consensus.cc:385] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:00.207365 15717 raft_consensus.cc:359] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.207999 15720 raft_consensus.cc:740] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:00.208156 15717 raft_consensus.cc:385] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:00.208505 15717 raft_consensus.cc:740] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:00.208820 15720 consensus_queue.cc:260] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.209553 15717 consensus_queue.cc:260] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.210384 15591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db313a4cd18a47a2b72587d04ed6e747. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.212122 15718 tablet_bootstrap.cc:492] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:00.212028 15424 catalog_manager.cc:5697] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:00.215473 15717 ts_tablet_manager.cc:1434] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.003s
I20260709 13:18:00.216593 15717 tablet_bootstrap.cc:492] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:00.220023 15724 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.221859 15724 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.222257 15513 tablet_service.cc:1511] Processing CreateTablet for tablet 88a66a9fb8df432fb0d1d7ba3756235e (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:18:00.225041 15513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88a66a9fb8df432fb0d1d7ba3756235e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.225616 15585 tablet_service.cc:1511] Processing CreateTablet for tablet 88a66a9fb8df432fb0d1d7ba3756235e (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:18:00.226713 15585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88a66a9fb8df432fb0d1d7ba3756235e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.232601 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:00.233263 15717 tablet_bootstrap.cc:654] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
W20260709 13:18:00.234205 15493 leader_election.cc:343] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:00.223664 15660 tablet_service.cc:1511] Processing CreateTablet for tablet db313a4cd18a47a2b72587d04ed6e747 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:18:00.239073 15660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db313a4cd18a47a2b72587d04ed6e747. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.241163 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:00.250351 15729 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.250869 15729 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:18:00.251220 15591 tablet_service.cc:1511] Processing CreateTablet for tablet 15e5bd79113c48c5a0b0f265bdd1d7b2 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:18:00.252429 15591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15e5bd79113c48c5a0b0f265bdd1d7b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.252714 15518 tablet_service.cc:1511] Processing CreateTablet for tablet 15e5bd79113c48c5a0b0f265bdd1d7b2 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:18:00.253221 15729 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:18:00.254489 15722 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.254871 15518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15e5bd79113c48c5a0b0f265bdd1d7b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.254966 15722 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.257413 15722 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:00.260000 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:00.261617 15566 leader_election.cc:343] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:18:00.262638 15491 leader_election.cc:343] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:00.263104 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:00.263496 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:00.263634 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:00.263123 15491 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:18:00.265740 15641 leader_election.cc:343] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:00.266952 15744 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:00.244207 15724 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:00.245040 15720 ts_tablet_manager.cc:1434] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.065s	user 0.008s	sys 0.000s
I20260709 13:18:00.264309 15527 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:00.268745 15720 tablet_bootstrap.cc:492] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:00.268993 15567 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:00.270354 15729 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:00.270725 15729 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.271067 15729 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:00.272743 15642 leader_election.cc:343] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:00.272984 15515 tablet_service.cc:1511] Processing CreateTablet for tablet 3088d5241ec64ddc919925d5dccfeda0 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:18:00.274856 15665 tablet_service.cc:1511] Processing CreateTablet for tablet 15e5bd79113c48c5a0b0f265bdd1d7b2 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:18:00.275523 15585 tablet_service.cc:1511] Processing CreateTablet for tablet 3088d5241ec64ddc919925d5dccfeda0 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:18:00.276118 15665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15e5bd79113c48c5a0b0f265bdd1d7b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.276708 15585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3088d5241ec64ddc919925d5dccfeda0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.276388 15516 tablet_service.cc:1511] Processing CreateTablet for tablet d6670c20b1954b36a4920b44d1a8209d (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:18:00.275518 15515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3088d5241ec64ddc919925d5dccfeda0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.278635 15516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6670c20b1954b36a4920b44d1a8209d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.282065 15642 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:00.282899 15722 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:00.283820 15720 tablet_bootstrap.cc:654] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:00.285266 15666 tablet_service.cc:1511] Processing CreateTablet for tablet 3088d5241ec64ddc919925d5dccfeda0 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:18:00.243317 15717 tablet_bootstrap.cc:492] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:00.252741 15718 tablet_bootstrap.cc:654] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:00.293591 15666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3088d5241ec64ddc919925d5dccfeda0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.299865 15658 tablet_service.cc:1511] Processing CreateTablet for tablet 88a66a9fb8df432fb0d1d7ba3756235e (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:18:00.300714 15722 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.301162 15658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88a66a9fb8df432fb0d1d7ba3756235e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.301182 15722 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.307407 15664 tablet_service.cc:1511] Processing CreateTablet for tablet d6670c20b1954b36a4920b44d1a8209d (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:18:00.307981 15722 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:00.309690 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:00.310369 15603 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:00.312127 15642 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:00.313073 15722 raft_consensus.cc:2804] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:00.313421 15722 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.302984 15717 ts_tablet_manager.cc:1403] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.087s	user 0.011s	sys 0.001s
I20260709 13:18:00.316689 15664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6670c20b1954b36a4920b44d1a8209d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.315883 15513 tablet_service.cc:1511] Processing CreateTablet for tablet e93d4a9ac6e844da87dcd8c6bfa7549b (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:18:00.319732 15513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e93d4a9ac6e844da87dcd8c6bfa7549b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.320372 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:00.321884 15591 tablet_service.cc:1511] Processing CreateTablet for tablet d6670c20b1954b36a4920b44d1a8209d (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:18:00.313736 15722 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:00.313741 15729 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:18:00.324625 15591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6670c20b1954b36a4920b44d1a8209d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.334184 15729 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:18:00.334779 15729 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.334947 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:00.335242 15729 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.336103 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:00.336670 15527 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:00.337286 15729 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:00.338020 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:00.340062 15665 tablet_service.cc:1511] Processing CreateTablet for tablet e93d4a9ac6e844da87dcd8c6bfa7549b (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:18:00.341279 15665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e93d4a9ac6e844da87dcd8c6bfa7549b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:18:00.342962 15566 leader_election.cc:343] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:00.344228 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:00.344094 15509 tablet_service.cc:1511] Processing CreateTablet for tablet ef08c91ca56042df89e729f13df83ed9 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:18:00.347281 15509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef08c91ca56042df89e729f13df83ed9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.347527 15585 tablet_service.cc:1511] Processing CreateTablet for tablet e93d4a9ac6e844da87dcd8c6bfa7549b (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:18:00.348613 15585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e93d4a9ac6e844da87dcd8c6bfa7549b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:18:00.346397 15567 leader_election.cc:343] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:00.354722 15753 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.355212 15753 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.358500 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:00.358770 15584 tablet_service.cc:1511] Processing CreateTablet for tablet ef08c91ca56042df89e729f13df83ed9 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:18:00.359153 15603 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:00.360033 15584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef08c91ca56042df89e729f13df83ed9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.361248 15642 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:00.361855 15720 tablet_bootstrap.cc:492] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:00.362392 15720 ts_tablet_manager.cc:1403] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.094s	user 0.006s	sys 0.006s
W20260709 13:18:00.350622 15641 leader_election.cc:343] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:00.365319 15720 raft_consensus.cc:359] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.366129 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:18:00.367740 15641 leader_election.cc:343] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:00.370412 15720 raft_consensus.cc:385] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:00.367574 15717 raft_consensus.cc:359] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.374508 15753 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:00.374677 15527 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:00.396297 15720 raft_consensus.cc:740] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:00.400312 15720 consensus_queue.cc:260] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.401417 15567 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:00.397148 15717 raft_consensus.cc:385] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:00.402707 15720 ts_tablet_manager.cc:1434] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.000s
I20260709 13:18:00.403000 15729 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:00.403461 15729 raft_consensus.cc:697] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:00.403759 15720 tablet_bootstrap.cc:492] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:00.404240 15729 consensus_queue.cc:237] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:18:00.402622 15717 raft_consensus.cc:740] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:00.406714 15717 consensus_queue.cc:260] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.401417 15722 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.408942 15717 ts_tablet_manager.cc:1434] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.090s	user 0.005s	sys 0.001s
I20260709 13:18:00.407138 15753 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:00.409366 15677 raft_consensus.cc:2393] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 1: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:18:00.409617 15753 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.410053 15753 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:00.412714 15566 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:00.413632 15751 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:00.416357 15584 tablet_service.cc:1511] Processing CreateTablet for tablet 061ed80a3fca4096b7c13dc0e2256ef0 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:18:00.417725 15584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 061ed80a3fca4096b7c13dc0e2256ef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.419373 15515 tablet_service.cc:1511] Processing CreateTablet for tablet f9328896aa5a4fdc8fa7c908efdae695 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:18:00.420779 15515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9328896aa5a4fdc8fa7c908efdae695. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.424053 15718 tablet_bootstrap.cc:492] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:00.430292 15718 ts_tablet_manager.cc:1403] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.232s	user 0.023s	sys 0.010s
I20260709 13:18:00.424901 15513 tablet_service.cc:1511] Processing CreateTablet for tablet 7b8ed23a60014da39806e750cee98fa6 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:18:00.431363 15513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b8ed23a60014da39806e750cee98fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.431751 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:00.432376 15664 tablet_service.cc:1511] Processing CreateTablet for tablet 061ed80a3fca4096b7c13dc0e2256ef0 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:18:00.433113 15751 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:00.433495 15664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 061ed80a3fca4096b7c13dc0e2256ef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.426584 15660 tablet_service.cc:1511] Processing CreateTablet for tablet ef08c91ca56042df89e729f13df83ed9 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:18:00.437736 15753 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.438751 15660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef08c91ca56042df89e729f13df83ed9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.443600 15720 tablet_bootstrap.cc:654] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:00.445011 15585 tablet_service.cc:1511] Processing CreateTablet for tablet 7b8ed23a60014da39806e750cee98fa6 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:18:00.446210 15585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b8ed23a60014da39806e750cee98fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.428975 15717 tablet_bootstrap.cc:492] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:00.451289 15518 tablet_service.cc:1511] Processing CreateTablet for tablet e67cef93bbdb4ff88a2ee2ac558ba090 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:18:00.452610 15518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e67cef93bbdb4ff88a2ee2ac558ba090. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.453436 15603 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:00.455025 15593 tablet_service.cc:1511] Processing CreateTablet for tablet e67cef93bbdb4ff88a2ee2ac558ba090 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:18:00.456439 15593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e67cef93bbdb4ff88a2ee2ac558ba090. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.457366 15591 tablet_service.cc:1511] Processing CreateTablet for tablet f9328896aa5a4fdc8fa7c908efdae695 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:18:00.424327 15509 tablet_service.cc:1511] Processing CreateTablet for tablet 061ed80a3fca4096b7c13dc0e2256ef0 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:18:00.458629 15509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 061ed80a3fca4096b7c13dc0e2256ef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.460214 15642 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:00.430150 15666 tablet_service.cc:1511] Processing CreateTablet for tablet f9328896aa5a4fdc8fa7c908efdae695 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:18:00.425359 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:00.429693 15722 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:00.461133 15755 raft_consensus.cc:2804] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:00.466460 15755 raft_consensus.cc:697] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:00.441732 15718 raft_consensus.cc:359] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.466740 15665 tablet_service.cc:1511] Processing CreateTablet for tablet 7b8ed23a60014da39806e750cee98fa6 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
W20260709 13:18:00.467047 15641 leader_election.cc:343] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:00.467866 15665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b8ed23a60014da39806e750cee98fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.468465 15666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9328896aa5a4fdc8fa7c908efdae695. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.468034 15755 consensus_queue.cc:237] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.465564 15424 catalog_manager.cc:5697] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:00.465512 15751 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.471698 15591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9328896aa5a4fdc8fa7c908efdae695. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.471375 15751 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.473441 15751 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:00.475667 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:00.475778 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:00.476250 15717 tablet_bootstrap.cc:654] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:00.476397 15677 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:00.476583 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:00.477708 15566 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:18:00.477968 15641 leader_election.cc:343] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:00.478929 15753 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:00.480600 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:00.481190 15603 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:00.486027 15718 raft_consensus.cc:385] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:00.486474 15718 raft_consensus.cc:740] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:00.487365 15718 consensus_queue.cc:260] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.488372 15528 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:00.493834 15729 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:00.494266 15729 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.494593 15729 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:00.499353 15658 tablet_service.cc:1511] Processing CreateTablet for tablet e67cef93bbdb4ff88a2ee2ac558ba090 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:18:00.500509 15658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e67cef93bbdb4ff88a2ee2ac558ba090. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.507283 15718 ts_tablet_manager.cc:1434] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.068s	user 0.005s	sys 0.001s
I20260709 13:18:00.508334 15718 tablet_bootstrap.cc:492] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:00.520584 15603 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:00.521972 15642 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:00.528358 15509 tablet_service.cc:1511] Processing CreateTablet for tablet 8153c7c094654fb492f5c47863ad170d (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:18:00.529558 15509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8153c7c094654fb492f5c47863ad170d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.530098 15753 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:00.530695 15753 raft_consensus.cc:697] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:00.531701 15718 tablet_bootstrap.cc:654] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:00.531541 15753 consensus_queue.cc:237] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.550699 15591 tablet_service.cc:1511] Processing CreateTablet for tablet 8153c7c094654fb492f5c47863ad170d (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:18:00.550948 15424 catalog_manager.cc:5697] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:00.552093 15591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8153c7c094654fb492f5c47863ad170d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.552795 15729 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.556130 15665 tablet_service.cc:1511] Processing CreateTablet for tablet 8153c7c094654fb492f5c47863ad170d (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:18:00.557694 15665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8153c7c094654fb492f5c47863ad170d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.560525 15593 tablet_service.cc:1511] Processing CreateTablet for tablet 7562b5f342134e39a5e6934d5ca16e46 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:18:00.561800 15593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7562b5f342134e39a5e6934d5ca16e46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.560206 15717 tablet_bootstrap.cc:492] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:00.562903 15515 tablet_service.cc:1511] Processing CreateTablet for tablet 7562b5f342134e39a5e6934d5ca16e46 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:18:00.563699 15717 ts_tablet_manager.cc:1403] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.135s	user 0.015s	sys 0.010s
I20260709 13:18:00.566736 15720 tablet_bootstrap.cc:492] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:00.567261 15720 ts_tablet_manager.cc:1403] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.164s	user 0.014s	sys 0.000s
I20260709 13:18:00.564193 15515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7562b5f342134e39a5e6934d5ca16e46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.573115 15717 raft_consensus.cc:359] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.573855 15717 raft_consensus.cc:385] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:00.574235 15717 raft_consensus.cc:740] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:00.574446 15666 tablet_service.cc:1511] Processing CreateTablet for tablet 7562b5f342134e39a5e6934d5ca16e46 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:18:00.575821 15666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7562b5f342134e39a5e6934d5ca16e46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.574980 15717 consensus_queue.cc:260] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.576431 15718 tablet_bootstrap.cc:492] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:00.576683 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:00.578801 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:00.583376 15718 ts_tablet_manager.cc:1403] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.075s	user 0.022s	sys 0.019s
I20260709 13:18:00.583601 15677 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:00.586089 15718 raft_consensus.cc:359] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.586813 15718 raft_consensus.cc:385] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:00.587090 15718 raft_consensus.cc:740] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:00.587772 15718 consensus_queue.cc:260] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.583377 15528 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:00.589952 15718 ts_tablet_manager.cc:1434] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:18:00.581426 15509 tablet_service.cc:1511] Processing CreateTablet for tablet 7dbabc834ad54c16ae25f79064932f42 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
W20260709 13:18:00.588198 15474 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:00.591840 15717 ts_tablet_manager.cc:1434] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.000s
I20260709 13:18:00.592396 15509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dbabc834ad54c16ae25f79064932f42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.593087 15718 tablet_bootstrap.cc:492] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:00.596963 15591 tablet_service.cc:1511] Processing CreateTablet for tablet 7dbabc834ad54c16ae25f79064932f42 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:18:00.598284 15591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dbabc834ad54c16ae25f79064932f42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.598773 15718 tablet_bootstrap.cc:654] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:00.599957 15677 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:00.601281 15658 tablet_service.cc:1511] Processing CreateTablet for tablet 7dbabc834ad54c16ae25f79064932f42 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:18:00.627103 15658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dbabc834ad54c16ae25f79064932f42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.627383 15566 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:00.602236 15528 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:00.630524 15751 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:00.632169 15589 tablet_service.cc:1511] Processing CreateTablet for tablet ab68091b31d248a6ac99ca234a252f77 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:18:00.632786 15666 tablet_service.cc:1511] Processing CreateTablet for tablet ab68091b31d248a6ac99ca234a252f77 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:18:00.633447 15589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab68091b31d248a6ac99ca234a252f77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.579267 15729 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:00.634008 15666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab68091b31d248a6ac99ca234a252f77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.602694 15755 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.634886 15755 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.636945 15755 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:00.603529 15518 tablet_service.cc:1511] Processing CreateTablet for tablet 0af5649725244ae599dfc6ed7aeac5f1 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:18:00.638283 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:00.638554 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:00.605506 15584 tablet_service.cc:1511] Processing CreateTablet for tablet 0af5649725244ae599dfc6ed7aeac5f1 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:18:00.639231 15528 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:00.606205 15662 tablet_service.cc:1511] Processing CreateTablet for tablet 0af5649725244ae599dfc6ed7aeac5f1 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:18:00.639946 15584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0af5649725244ae599dfc6ed7aeac5f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.640622 15662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0af5649725244ae599dfc6ed7aeac5f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.607633 15585 tablet_service.cc:1511] Processing CreateTablet for tablet ecb5853a05664246a26d3fc7ecf92777 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:18:00.642578 15585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecb5853a05664246a26d3fc7ecf92777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.608064 15717 tablet_bootstrap.cc:492] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:00.607946 15513 tablet_service.cc:1511] Processing CreateTablet for tablet ecb5853a05664246a26d3fc7ecf92777 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:18:00.582719 15720 raft_consensus.cc:359] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.646160 15720 raft_consensus.cc:385] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:00.646522 15720 raft_consensus.cc:740] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:00.647296 15720 consensus_queue.cc:260] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.648259 15603 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:00.649541 15720 ts_tablet_manager.cc:1434] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.082s	user 0.006s	sys 0.000s
I20260709 13:18:00.611285 15516 tablet_service.cc:1511] Processing CreateTablet for tablet 0f8365fc4e2f4decb3d554a43e7d4d52 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:18:00.650408 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:00.650882 15720 tablet_bootstrap.cc:492] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:00.651446 15755 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:00.651854 15755 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.638578 15518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0af5649725244ae599dfc6ed7aeac5f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.652218 15755 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:00.654239 15516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f8365fc4e2f4decb3d554a43e7d4d52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.658236 15513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecb5853a05664246a26d3fc7ecf92777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.660607 15755 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.615936 15425 catalog_manager.cc:5697] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:00.618438 15593 tablet_service.cc:1511] Processing CreateTablet for tablet d4fd164d74054954b750fe9958e66778 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:18:00.664381 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:00.664845 15593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4fd164d74054954b750fe9958e66778. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.665088 15603 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:00.618811 15514 tablet_service.cc:1511] Processing CreateTablet for tablet d4fd164d74054954b750fe9958e66778 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:18:00.666750 15514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4fd164d74054954b750fe9958e66778. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.667219 15720 tablet_bootstrap.cc:654] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:00.622241 15665 tablet_service.cc:1511] Processing CreateTablet for tablet 0f8365fc4e2f4decb3d554a43e7d4d52 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:18:00.670266 15509 tablet_service.cc:1511] Processing CreateTablet for tablet ab68091b31d248a6ac99ca234a252f77 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:18:00.670727 15665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f8365fc4e2f4decb3d554a43e7d4d52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.672561 15509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab68091b31d248a6ac99ca234a252f77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.673096 15724 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:00.624096 15512 tablet_service.cc:1511] Processing CreateTablet for tablet 6139553a47b64d798a29413921f18396 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:18:00.624402 15590 tablet_service.cc:1511] Processing CreateTablet for tablet 6139553a47b64d798a29413921f18396 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:18:00.674755 15512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6139553a47b64d798a29413921f18396. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.675195 15590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6139553a47b64d798a29413921f18396. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.622332 15660 tablet_service.cc:1511] Processing CreateTablet for tablet d4fd164d74054954b750fe9958e66778 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:18:00.677403 15660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4fd164d74054954b750fe9958e66778. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.680570 15603 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:00.625468 15667 tablet_service.cc:1511] Processing CreateTablet for tablet 6139553a47b64d798a29413921f18396 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:18:00.681849 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:00.682713 15667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6139553a47b64d798a29413921f18396. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.682730 15753 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:00.683315 15753 raft_consensus.cc:697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:00.684136 15753 consensus_queue.cc:237] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.692886 15755 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:00.625711 15664 tablet_service.cc:1511] Processing CreateTablet for tablet ecb5853a05664246a26d3fc7ecf92777 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:18:00.610579 15586 tablet_service.cc:1511] Processing CreateTablet for tablet 0f8365fc4e2f4decb3d554a43e7d4d52 (DEFAULT_TABLE table=test-workload [id=8efcedb1864d41909abf9d2535dfa264]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:18:00.694609 15586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f8365fc4e2f4decb3d554a43e7d4d52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.694715 15717 tablet_bootstrap.cc:654] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:00.698145 15664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecb5853a05664246a26d3fc7ecf92777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:00.710489 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:00.711109 15528 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:00.724912 15720 tablet_bootstrap.cc:492] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:00.725427 15720 ts_tablet_manager.cc:1403] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.075s	user 0.009s	sys 0.000s
I20260709 13:18:00.730095 15722 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.738201 15722 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.740375 15722 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:00.741272 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:18:00.743516 15641 leader_election.cc:343] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:00.745406 15603 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:00.718377 15755 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.747706 15755 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.751116 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:00.751852 15527 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:00.753293 15641 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:00.754281 15722 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:00.754621 15722 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.754930 15722 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:00.764657 15755 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:00.773033 15720 raft_consensus.cc:359] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.773777 15720 raft_consensus.cc:385] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:00.774075 15720 raft_consensus.cc:740] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:00.774791 15720 consensus_queue.cc:260] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.775748 15528 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:00.776011 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:00.765107 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:18:00.777632 15642 leader_election.cc:343] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:18:00.778978 15642 leader_election.cc:343] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:00.779415 15642 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:00.782271 15722 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.782717 15755 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:00.784883 15722 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:00.786971 15724 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:00.786098 15751 raft_consensus.cc:697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:00.789402 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:00.790030 15528 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:00.789836 15751 consensus_queue.cc:237] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.791216 15720 ts_tablet_manager.cc:1434] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.065s	user 0.008s	sys 0.000s
I20260709 13:18:00.792261 15720 tablet_bootstrap.cc:492] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:00.794346 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:18:00.796042 15642 leader_election.cc:343] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:00.792589 15425 catalog_manager.cc:5697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:00.814654 15528 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:00.815994 15641 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:00.817250 15722 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:00.821412 15717 tablet_bootstrap.cc:492] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:00.821893 15717 ts_tablet_manager.cc:1403] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.214s	user 0.010s	sys 0.011s
I20260709 13:18:00.817955 15722 raft_consensus.cc:697] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:00.824597 15717 raft_consensus.cc:359] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.825338 15717 raft_consensus.cc:385] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:00.825649 15717 raft_consensus.cc:740] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:00.825570 15722 consensus_queue.cc:237] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.826666 15717 consensus_queue.cc:260] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.828890 15717 ts_tablet_manager.cc:1434] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 13:18:00.829842 15717 tablet_bootstrap.cc:492] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:00.824536 15718 tablet_bootstrap.cc:492] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:00.833218 15718 ts_tablet_manager.cc:1403] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.240s	user 0.015s	sys 0.045s
I20260709 13:18:00.838449 15720 tablet_bootstrap.cc:654] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:00.846937 15718 raft_consensus.cc:359] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.847853 15718 raft_consensus.cc:385] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:00.848207 15718 raft_consensus.cc:740] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:00.846833 15424 catalog_manager.cc:5697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:00.862190 15717 tablet_bootstrap.cc:654] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:00.867928 15718 consensus_queue.cc:260] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.876428 15718 ts_tablet_manager.cc:1434] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.040s	user 0.005s	sys 0.001s
I20260709 13:18:00.877344 15718 tablet_bootstrap.cc:492] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
W20260709 13:18:00.885775 15566 consensus_peers.cc:597] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Couldn't send request to peer 3ef186ec12554a1d87814af0a0ace9a4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:00.886735 15718 tablet_bootstrap.cc:654] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:00.893591 15720 tablet_bootstrap.cc:492] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:00.894147 15720 ts_tablet_manager.cc:1403] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.102s	user 0.016s	sys 0.015s
I20260709 13:18:00.896247 15425 catalog_manager.cc:5697] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:00.896924 15720 raft_consensus.cc:359] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.897683 15720 raft_consensus.cc:385] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:00.900373 15720 raft_consensus.cc:740] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:00.901081 15720 consensus_queue.cc:260] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.902801 15751 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.903141 15720 ts_tablet_manager.cc:1434] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:18:00.904242 15720 tablet_bootstrap.cc:492] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:00.903275 15751 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.906225 15751 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:00.906972 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:00.907624 15528 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:00.908951 15567 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:00.909780 15718 tablet_bootstrap.cc:492] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:00.909868 15751 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:00.910511 15751 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.910925 15751 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:00.910713 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:00.917328 15566 leader_election.cc:343] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:00.917146 15751 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.911273 15718 ts_tablet_manager.cc:1403] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.034s	user 0.009s	sys 0.016s
I20260709 13:18:00.920239 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:00.920825 15528 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:00.920920 15718 raft_consensus.cc:359] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.921572 15718 raft_consensus.cc:385] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:00.921846 15718 raft_consensus.cc:740] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:00.922453 15722 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:00.922590 15718 consensus_queue.cc:260] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.927349 15751 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:00.927541 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:00.928191 15717 tablet_bootstrap.cc:492] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:00.928654 15717 ts_tablet_manager.cc:1403] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.099s	user 0.015s	sys 0.004s
I20260709 13:18:00.928509 15528 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
W20260709 13:18:00.928994 15566 leader_election.cc:343] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:00.929850 15567 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:00.930975 15751 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:00.931231 15717 raft_consensus.cc:359] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:18:00.931685 15751 raft_consensus.cc:697] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:00.931912 15729 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:00.932605 15720 tablet_bootstrap.cc:654] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:00.932569 15751 consensus_queue.cc:237] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.936784 15718 ts_tablet_manager.cc:1434] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 13:18:00.937933 15718 tablet_bootstrap.cc:492] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:00.944474 15720 tablet_bootstrap.cc:492] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:00.944993 15720 ts_tablet_manager.cc:1403] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.041s	user 0.016s	sys 0.000s
I20260709 13:18:00.947736 15720 raft_consensus.cc:359] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.948421 15720 raft_consensus.cc:385] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:00.948689 15720 raft_consensus.cc:740] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:00.949317 15720 consensus_queue.cc:260] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.931926 15717 raft_consensus.cc:385] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:00.951926 15720 ts_tablet_manager.cc:1434] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:18:00.951813 15717 raft_consensus.cc:740] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:00.952869 15717 consensus_queue.cc:260] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:18:00.953234 15720 tablet_bootstrap.cc:492] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:00.957870 15717 ts_tablet_manager.cc:1434] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.001s
I20260709 13:18:00.959487 15720 tablet_bootstrap.cc:654] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:00.962296 15718 tablet_bootstrap.cc:654] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:00.966264 15425 catalog_manager.cc:5697] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:00.968719 15641 consensus_peers.cc:597] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:00.977757 15718 tablet_bootstrap.cc:492] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:00.978392 15718 ts_tablet_manager.cc:1403] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.041s	user 0.016s	sys 0.005s
I20260709 13:18:00.978579 15720 tablet_bootstrap.cc:492] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:00.979008 15720 ts_tablet_manager.cc:1403] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.003s
I20260709 13:18:00.980957 15718 raft_consensus.cc:359] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.981130 15720 raft_consensus.cc:359] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.981729 15718 raft_consensus.cc:385] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:00.981814 15720 raft_consensus.cc:385] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:00.982101 15718 raft_consensus.cc:740] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:00.982890 15718 consensus_queue.cc:260] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.984777 15718 ts_tablet_manager.cc:1434] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:18:00.982209 15720 raft_consensus.cc:740] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:00.985543 15720 consensus_queue.cc:260] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:00.981491 15717 tablet_bootstrap.cc:492] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:00.987474 15720 ts_tablet_manager.cc:1434] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:18:00.988694 15720 tablet_bootstrap.cc:492] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:00.990746 15717 tablet_bootstrap.cc:654] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:00.992734 15720 tablet_bootstrap.cc:654] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:00.994187 15718 tablet_bootstrap.cc:492] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:00.999833 15753 consensus_queue.cc:1048] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:01.000744 15718 tablet_bootstrap.cc:654] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.007462 15717 tablet_bootstrap.cc:492] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:01.008134 15717 ts_tablet_manager.cc:1403] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.030s	user 0.007s	sys 0.003s
I20260709 13:18:01.016299 15753 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.016860 15753 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:18:01.020174 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:01.026553 15527 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.040189 15718 tablet_bootstrap.cc:492] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:01.040652 15718 ts_tablet_manager.cc:1403] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.047s	user 0.022s	sys 0.023s
I20260709 13:18:01.041177 15720 tablet_bootstrap.cc:492] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:01.041642 15720 ts_tablet_manager.cc:1403] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.053s	user 0.028s	sys 0.022s
I20260709 13:18:01.042698 15718 raft_consensus.cc:359] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.043184 15718 raft_consensus.cc:385] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.043390 15718 raft_consensus.cc:740] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.043970 15718 consensus_queue.cc:260] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.044153 15720 raft_consensus.cc:359] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.044837 15720 raft_consensus.cc:385] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.045130 15720 raft_consensus.cc:740] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.045774 15718 ts_tablet_manager.cc:1434] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.001s
I20260709 13:18:01.045768 15720 consensus_queue.cc:260] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.046994 15718 tablet_bootstrap.cc:492] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:01.047719 15753 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:01.047624 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:01.047964 15720 ts_tablet_manager.cc:1434] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:18:01.048928 15720 tablet_bootstrap.cc:492] T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:01.049382 15642 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:01.052410 15718 tablet_bootstrap.cc:654] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.054132 15720 tablet_bootstrap.cc:654] T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.056469 15753 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:01.063411 15718 tablet_bootstrap.cc:492] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:01.063829 15718 ts_tablet_manager.cc:1403] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 13:18:01.065979 15718 raft_consensus.cc:359] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.066677 15718 raft_consensus.cc:385] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.067035 15718 raft_consensus.cc:740] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.067775 15718 consensus_queue.cc:260] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.067946 15753 consensus_queue.cc:1048] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:01.066629 15717 raft_consensus.cc:359] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.068786 15717 raft_consensus.cc:385] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.069113 15717 raft_consensus.cc:740] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.070128 15718 ts_tablet_manager.cc:1434] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:18:01.070019 15717 consensus_queue.cc:260] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.071007 15718 tablet_bootstrap.cc:492] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:01.074554 15717 ts_tablet_manager.cc:1434] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.066s	user 0.004s	sys 0.003s
I20260709 13:18:01.075594 15717 tablet_bootstrap.cc:492] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:01.077028 15718 tablet_bootstrap.cc:654] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.086253 15717 tablet_bootstrap.cc:654] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.091220 15720 tablet_bootstrap.cc:492] T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:01.091739 15720 ts_tablet_manager.cc:1403] T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.043s	user 0.012s	sys 0.028s
I20260709 13:18:01.093328 15724 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.093950 15724 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.094852 15720 raft_consensus.cc:359] T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.095502 15720 raft_consensus.cc:385] T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.095644 15718 tablet_bootstrap.cc:492] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:01.095804 15720 raft_consensus.cc:740] T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.096042 15724 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:01.096169 15718 ts_tablet_manager.cc:1403] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.007s
I20260709 13:18:01.097638 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:01.098356 15603 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:01.098821 15718 raft_consensus.cc:359] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.097637 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:01.099516 15718 raft_consensus.cc:385] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.099709 15677 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:01.099840 15718 raft_consensus.cc:740] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.096678 15720 consensus_queue.cc:260] T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.100807 15718 consensus_queue.cc:260] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.101130 15491 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:01.102831 15724 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:01.103261 15724 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:01.103587 15724 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.107333 15720 ts_tablet_manager.cc:1434] T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.015s	user 0.000s	sys 0.007s
I20260709 13:18:01.108404 15720 tablet_bootstrap.cc:492] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:01.110415 15717 tablet_bootstrap.cc:492] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:01.110881 15717 ts_tablet_manager.cc:1403] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.036s	user 0.007s	sys 0.009s
I20260709 13:18:01.113458 15717 raft_consensus.cc:359] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.114017 15720 tablet_bootstrap.cc:654] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.114202 15717 raft_consensus.cc:385] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.114491 15717 raft_consensus.cc:740] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.115402 15717 consensus_queue.cc:260] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.116009 15718 ts_tablet_manager.cc:1434] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.001s
I20260709 13:18:01.116891 15718 tablet_bootstrap.cc:492] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:01.120709 15717 ts_tablet_manager.cc:1434] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:18:01.121829 15717 tablet_bootstrap.cc:492] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:01.124043 15724 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.125294 15720 tablet_bootstrap.cc:492] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:01.125774 15720 ts_tablet_manager.cc:1403] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.003s
I20260709 13:18:01.126912 15718 tablet_bootstrap.cc:654] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.127010 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:01.127322 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:01.127629 15603 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.127893 15724 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:01.132558 15724 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.133025 15724 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.127832 15677 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.135622 15724 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:01.135735 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:01.136399 15602 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:01.136950 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:01.137530 15676 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:01.137604 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:01.138481 15724 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:01.138832 15724 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:01.139146 15724 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.139195 15720 raft_consensus.cc:359] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.139961 15720 raft_consensus.cc:385] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.140229 15720 raft_consensus.cc:740] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.140895 15720 consensus_queue.cc:260] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.134687 15603 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:01.145552 15493 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:01.146605 15771 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:01.147387 15720 ts_tablet_manager.cc:1434] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:18:01.147260 15677 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:01.147161 15724 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.148387 15720 tablet_bootstrap.cc:492] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:01.149885 15724 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:01.154458 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:01.155090 15603 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.159096 15717 tablet_bootstrap.cc:654] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.162308 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:01.162930 15677 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.163302 15720 tablet_bootstrap.cc:654] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.167652 15753 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:01.174733 15717 tablet_bootstrap.cc:492] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:01.175513 15717 ts_tablet_manager.cc:1403] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.054s	user 0.010s	sys 0.003s
I20260709 13:18:01.178488 15717 raft_consensus.cc:359] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.179219 15717 raft_consensus.cc:385] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.179265 15603 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:01.179553 15717 raft_consensus.cc:740] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.180507 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:01.180338 15717 consensus_queue.cc:260] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.181341 15724 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:01.181854 15724 raft_consensus.cc:697] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:01.182777 15724 consensus_queue.cc:237] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.182771 15717 ts_tablet_manager.cc:1434] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:18:01.185161 15717 tablet_bootstrap.cc:492] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:01.190637 15717 tablet_bootstrap.cc:654] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.205423 15753 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.205991 15753 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.208120 15753 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:01.209841 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:18:01.211347 15642 leader_election.cc:343] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:01.211601 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:01.212392 15527 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:01.215302 15641 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:01.217141 15722 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:01.218721 15722 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:01.219107 15722 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:01.219446 15722 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.213208 15771 raft_consensus.cc:697] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:01.253695 15729 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:01.254196 15718 tablet_bootstrap.cc:492] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:01.254772 15718 ts_tablet_manager.cc:1403] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.138s	user 0.030s	sys 0.019s
I20260709 13:18:01.258368 15718 raft_consensus.cc:359] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.259130 15718 raft_consensus.cc:385] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.259423 15718 raft_consensus.cc:740] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.260674 15718 consensus_queue.cc:260] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.266577 15677 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:01.262348 15771 consensus_queue.cc:237] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.276364 15722 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.278666 15718 ts_tablet_manager.cc:1434] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:18:01.279855 15718 tablet_bootstrap.cc:492] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:01.285735 15722 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:01.287070 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:01.290218 15720 tablet_bootstrap.cc:492] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:01.290694 15720 ts_tablet_manager.cc:1403] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.142s	user 0.029s	sys 0.032s
I20260709 13:18:01.290819 15718 tablet_bootstrap.cc:654] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.294106 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:01.294756 15528 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:01.294124 15642 leader_election.cc:343] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:01.300856 15751 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:01.304811 15720 raft_consensus.cc:359] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.305584 15720 raft_consensus.cc:385] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.306367 15720 raft_consensus.cc:740] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.307101 15720 consensus_queue.cc:260] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.307972 15602 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.318456 15802 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:01.319664 15426 catalog_manager.cc:5697] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:01.324837 15720 ts_tablet_manager.cc:1434] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.023s	user 0.004s	sys 0.001s
I20260709 13:18:01.325011 15676 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.333182 15717 tablet_bootstrap.cc:492] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:01.333701 15717 ts_tablet_manager.cc:1403] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.149s	user 0.012s	sys 0.007s
I20260709 13:18:01.336354 15717 raft_consensus.cc:359] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.337030 15717 raft_consensus.cc:385] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.309847 15528 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:01.337298 15717 raft_consensus.cc:740] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.339103 15755 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:01.338406 15720 tablet_bootstrap.cc:492] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:01.358573 15641 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:01.360031 15755 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:01.359582 15424 catalog_manager.cc:5697] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:01.360878 15755 raft_consensus.cc:697] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:01.363605 15718 tablet_bootstrap.cc:492] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:01.367343 15718 ts_tablet_manager.cc:1403] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.088s	user 0.007s	sys 0.007s
I20260709 13:18:01.368723 15771 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.361779 15755 consensus_queue.cc:237] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.369230 15771 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.370293 15717 consensus_queue.cc:260] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.372398 15802 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.372871 15802 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.373515 15771 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:01.374786 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:01.375413 15676 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:01.375643 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:01.376261 15603 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:01.376581 15491 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:01.376906 15717 ts_tablet_manager.cc:1434] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.043s	user 0.004s	sys 0.001s
I20260709 13:18:01.377861 15717 tablet_bootstrap.cc:492] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:01.377987 15771 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:01.378949 15771 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:01.379345 15771 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.379624 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:01.380280 15603 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:01.380651 15802 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:01.381680 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:18:01.383073 15641 leader_election.cc:343] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:01.383821 15642 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:01.385686 15718 raft_consensus.cc:359] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.386385 15718 raft_consensus.cc:385] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.386636 15718 raft_consensus.cc:740] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.387274 15718 consensus_queue.cc:260] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.388679 15802 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:01.389060 15802 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:01.389381 15802 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.390108 15718 ts_tablet_manager.cc:1434] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:18:01.387976 15771 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.397532 15751 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.398181 15751 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.400918 15771 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:01.402285 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:01.403049 15526 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:01.403908 15718 tablet_bootstrap.cc:492] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:01.404961 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:01.405191 15751 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:01.405553 15603 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.405993 15567 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:01.406877 15720 tablet_bootstrap.cc:654] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.407013 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:01.407486 15753 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.411330 15753 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.411047 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:01.413813 15753 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:01.414243 15677 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.414664 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:01.415426 15526 raft_consensus.cc:2393] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 1: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:01.415429 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:01.415820 15603 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:01.416834 15493 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:01.417558 15771 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:01.417902 15771 raft_consensus.cc:697] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:01.417832 15641 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:01.418740 15755 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.418736 15771 consensus_queue.cc:237] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.419359 15802 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.421173 15802 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:01.409691 15751 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:01.421808 15751 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:01.422034 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:01.410633 15676 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:01.422254 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:18:01.423619 15641 leader_election.cc:343] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:01.422978 15603 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.428653 15755 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:01.430883 15717 tablet_bootstrap.cc:654] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.431226 15603 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:01.422185 15751 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.434432 15642 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:01.436137 15755 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:01.436689 15755 raft_consensus.cc:697] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:01.438212 15755 consensus_queue.cc:237] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.439791 15677 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:01.444900 15718 tablet_bootstrap.cc:654] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.457590 15426 catalog_manager.cc:5697] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:01.447602 15771 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.479290 15771 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.464298 15783 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 13:18:01.489837 15720 tablet_bootstrap.cc:492] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:01.490044 15526 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.490362 15720 ts_tablet_manager.cc:1403] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.152s	user 0.018s	sys 0.011s
I20260709 13:18:01.494796 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:01.497151 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:01.452464 15425 catalog_manager.cc:5697] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:01.461803 15751 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.499948 15493 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:01.500456 15720 raft_consensus.cc:359] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.502869 15720 raft_consensus.cc:385] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.503153 15720 raft_consensus.cc:740] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.503738 15720 consensus_queue.cc:260] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.506248 15720 ts_tablet_manager.cc:1434] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 13:18:01.502746 15771 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:01.511179 15720 tablet_bootstrap.cc:492] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:01.511051 15729 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:01.512322 15426 catalog_manager.cc:5697] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:01.515069 15718 tablet_bootstrap.cc:492] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:01.515535 15718 ts_tablet_manager.cc:1403] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.112s	user 0.018s	sys 0.006s
I20260709 13:18:01.515687 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:01.516458 15528 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.520066 15720 tablet_bootstrap.cc:654] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.528690 15528 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:01.530164 15567 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:01.531646 15783 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:01.532429 15783 raft_consensus.cc:697] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:01.533496 15718 raft_consensus.cc:359] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.533329 15783 consensus_queue.cc:237] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.534158 15718 raft_consensus.cc:385] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.534458 15718 raft_consensus.cc:740] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.535096 15718 consensus_queue.cc:260] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.537066 15718 ts_tablet_manager.cc:1434] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.000s
I20260709 13:18:01.538087 15718 tablet_bootstrap.cc:492] T f25e63ed82fa4e008109b74fe8df71ba P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:01.545341 15771 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.546003 15771 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.547134 15751 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:01.551077 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:01.551419 15720 tablet_bootstrap.cc:492] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:01.551738 15676 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.551429 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:01.552039 15720 ts_tablet_manager.cc:1403] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.041s	user 0.009s	sys 0.004s
I20260709 13:18:01.552861 15718 tablet_bootstrap.cc:654] T f25e63ed82fa4e008109b74fe8df71ba P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
W20260709 13:18:01.553241 15493 leader_election.cc:343] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:01.555626 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:01.557418 15491 leader_election.cc:343] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:01.557906 15491 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:01.558974 15797 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:01.560475 15771 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:01.561585 15717 tablet_bootstrap.cc:492] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:01.561537 15720 raft_consensus.cc:359] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.562443 15720 raft_consensus.cc:385] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.562605 15717 ts_tablet_manager.cc:1403] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.185s	user 0.019s	sys 0.014s
I20260709 13:18:01.562768 15720 raft_consensus.cc:740] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.563532 15720 consensus_queue.cc:260] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.565769 15720 ts_tablet_manager.cc:1434] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.006s
I20260709 13:18:01.566736 15720 tablet_bootstrap.cc:492] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:01.569968 15717 raft_consensus.cc:359] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.571010 15717 raft_consensus.cc:385] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.571746 15717 raft_consensus.cc:740] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.572889 15720 tablet_bootstrap.cc:654] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.575042 15717 consensus_queue.cc:260] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.577602 15717 ts_tablet_manager.cc:1434] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:18:01.578733 15764 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.580030 15676 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:01.588467 15764 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.590794 15764 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:01.594296 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:01.594601 15718 tablet_bootstrap.cc:492] T f25e63ed82fa4e008109b74fe8df71ba P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:01.595522 15718 ts_tablet_manager.cc:1403] T f25e63ed82fa4e008109b74fe8df71ba P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.058s	user 0.010s	sys 0.005s
W20260709 13:18:01.595909 15566 leader_election.cc:343] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:01.598240 15717 tablet_bootstrap.cc:492] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:01.599782 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:18:01.601583 15567 leader_election.cc:343] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:01.602098 15567 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:01.603425 15751 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:01.608412 15426 catalog_manager.cc:5697] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:01.610234 15803 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:01.613171 15717 tablet_bootstrap.cc:654] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.623489 15718 raft_consensus.cc:359] T f25e63ed82fa4e008109b74fe8df71ba P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.624212 15718 raft_consensus.cc:385] T f25e63ed82fa4e008109b74fe8df71ba P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.624490 15718 raft_consensus.cc:740] T f25e63ed82fa4e008109b74fe8df71ba P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.625171 15718 consensus_queue.cc:260] T f25e63ed82fa4e008109b74fe8df71ba P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.627300 15718 ts_tablet_manager.cc:1434] T f25e63ed82fa4e008109b74fe8df71ba P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 13:18:01.628273 15718 tablet_bootstrap.cc:492] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:01.629181 15720 tablet_bootstrap.cc:492] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:01.629420 15717 tablet_bootstrap.cc:492] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:01.629632 15720 ts_tablet_manager.cc:1403] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.063s	user 0.025s	sys 0.029s
I20260709 13:18:01.629868 15717 ts_tablet_manager.cc:1403] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.034s	user 0.005s	sys 0.009s
W20260709 13:18:01.631556 15474 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:01.632454 15717 raft_consensus.cc:359] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.633553 15717 raft_consensus.cc:385] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.633848 15717 raft_consensus.cc:740] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.634665 15720 raft_consensus.cc:359] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.635340 15720 raft_consensus.cc:385] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.635604 15720 raft_consensus.cc:740] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.635491 15717 consensus_queue.cc:260] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.636337 15720 consensus_queue.cc:260] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.637549 15717 ts_tablet_manager.cc:1434] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 13:18:01.638309 15720 ts_tablet_manager.cc:1434] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:18:01.638476 15717 tablet_bootstrap.cc:492] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:01.639191 15720 tablet_bootstrap.cc:492] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:01.640128 15803 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:01.633817 15718 tablet_bootstrap.cc:654] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.645010 15717 tablet_bootstrap.cc:654] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.651540 15720 tablet_bootstrap.cc:654] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.667310 15717 tablet_bootstrap.cc:492] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:01.666850 15724 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:01.667768 15717 ts_tablet_manager.cc:1403] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.012s
I20260709 13:18:01.670454 15717 raft_consensus.cc:359] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.671126 15717 raft_consensus.cc:385] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.671384 15717 raft_consensus.cc:740] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.672070 15718 tablet_bootstrap.cc:492] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:01.672649 15718 ts_tablet_manager.cc:1403] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.045s	user 0.017s	sys 0.000s
I20260709 13:18:01.672148 15717 consensus_queue.cc:260] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.674793 15717 ts_tablet_manager.cc:1434] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:18:01.675366 15718 raft_consensus.cc:359] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.676077 15718 raft_consensus.cc:385] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.676374 15718 raft_consensus.cc:740] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.677058 15718 consensus_queue.cc:260] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.679873 15717 tablet_bootstrap.cc:492] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:01.680136 15718 ts_tablet_manager.cc:1434] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:18:01.681120 15718 tablet_bootstrap.cc:492] T d7b7542687ad43acb128a39bb1f40829 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:01.686484 15751 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.687058 15751 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.689266 15751 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:01.690889 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:01.691668 15526 raft_consensus.cc:2393] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 1: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:18:01.691967 15717 tablet_bootstrap.cc:654] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.691655 15720 tablet_bootstrap.cc:492] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:01.692631 15720 ts_tablet_manager.cc:1403] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.054s	user 0.015s	sys 0.002s
I20260709 13:18:01.691618 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:01.695305 15720 raft_consensus.cc:359] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.695986 15720 raft_consensus.cc:385] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.696244 15720 raft_consensus.cc:740] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.696938 15720 consensus_queue.cc:260] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.698487 15718 tablet_bootstrap.cc:654] T d7b7542687ad43acb128a39bb1f40829 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.700634 15566 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:01.702107 15751 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.702631 15717 tablet_bootstrap.cc:492] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:01.703569 15717 ts_tablet_manager.cc:1403] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.024s	user 0.005s	sys 0.009s
I20260709 13:18:01.706171 15717 raft_consensus.cc:359] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.706837 15717 raft_consensus.cc:385] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.707118 15717 raft_consensus.cc:740] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.707969 15755 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.707893 15717 consensus_queue.cc:260] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.708743 15751 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:01.708519 15755 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.702900 15720 ts_tablet_manager.cc:1434] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:18:01.710412 15717 ts_tablet_manager.cc:1434] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:18:01.710819 15755 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:01.711333 15717 tablet_bootstrap.cc:492] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:01.710757 15720 tablet_bootstrap.cc:492] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:01.712453 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:01.712962 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:01.713075 15526 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:01.713581 15603 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:01.714417 15641 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:01.715348 15724 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.715451 15755 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:01.715848 15755 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:01.716276 15755 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.715890 15724 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.717621 15717 tablet_bootstrap.cc:654] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.718230 15724 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:01.718989 15720 tablet_bootstrap.cc:654] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.720091 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:01.720825 15603 raft_consensus.cc:2393] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 1: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:18:01.728003 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:01.727201 15755 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.730391 15720 tablet_bootstrap.cc:492] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:01.733258 15720 ts_tablet_manager.cc:1403] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.002s
I20260709 13:18:01.732604 15491 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:01.733748 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:01.734320 15526 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.734683 15724 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.740947 15526 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:01.741226 15724 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:01.742396 15641 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:01.742728 15720 raft_consensus.cc:359] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.743285 15802 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:01.743544 15720 raft_consensus.cc:385] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.743978 15720 raft_consensus.cc:740] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.744120 15802 raft_consensus.cc:697] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:01.744851 15720 consensus_queue.cc:260] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.745056 15802 consensus_queue.cc:237] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.748135 15720 ts_tablet_manager.cc:1434] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.014s	user 0.001s	sys 0.007s
I20260709 13:18:01.730834 15718 tablet_bootstrap.cc:492] T d7b7542687ad43acb128a39bb1f40829 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:01.749158 15720 tablet_bootstrap.cc:492] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:01.749595 15718 ts_tablet_manager.cc:1403] T d7b7542687ad43acb128a39bb1f40829 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.069s	user 0.019s	sys 0.000s
I20260709 13:18:01.751147 15751 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.751590 15751 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.752250 15718 raft_consensus.cc:359] T d7b7542687ad43acb128a39bb1f40829 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.752895 15718 raft_consensus.cc:385] T d7b7542687ad43acb128a39bb1f40829 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.753136 15718 raft_consensus.cc:740] T d7b7542687ad43acb128a39bb1f40829 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.753604 15751 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:01.754464 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:01.755378 15720 tablet_bootstrap.cc:654] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.755224 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:01.756045 15525 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:01.757326 15567 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:18:01.757903 15566 leader_election.cc:343] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:01.758147 15751 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:01.758502 15751 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:01.758834 15751 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.758767 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:01.759392 15603 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.759601 15755 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:01.761821 15718 consensus_queue.cc:260] T d7b7542687ad43acb128a39bb1f40829 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.772132 15720 tablet_bootstrap.cc:492] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:01.771919 15751 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.772614 15720 ts_tablet_manager.cc:1403] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.000s
I20260709 13:18:01.773793 15797 consensus_queue.cc:1048] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:01.775856 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:01.776449 15525 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.776961 15751 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:01.776961 15603 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:01.775801 15720 raft_consensus.cc:359] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.779034 15720 raft_consensus.cc:385] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.779430 15720 raft_consensus.cc:740] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.780346 15720 consensus_queue.cc:260] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.778990 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:01.786417 15720 ts_tablet_manager.cc:1434] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.002s
I20260709 13:18:01.787557 15720 tablet_bootstrap.cc:492] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:01.792119 15718 ts_tablet_manager.cc:1434] T d7b7542687ad43acb128a39bb1f40829 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.000s
I20260709 13:18:01.793012 15718 tablet_bootstrap.cc:492] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
W20260709 13:18:01.795181 15566 leader_election.cc:343] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:01.800096 15426 catalog_manager.cc:5697] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:01.816730 15525 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:01.818584 15567 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:01.819705 15751 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:01.820168 15751 raft_consensus.cc:697] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:01.820945 15751 consensus_queue.cc:237] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.822199 15720 tablet_bootstrap.cc:654] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.826601 15783 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.827131 15783 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.828922 15717 tablet_bootstrap.cc:492] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:01.829149 15783 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:01.829412 15717 ts_tablet_manager.cc:1403] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.118s	user 0.021s	sys 0.044s
I20260709 13:18:01.832381 15803 consensus_queue.cc:1048] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:01.834784 15755 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:01.837568 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:01.838276 15525 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:01.839725 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:01.840137 15567 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:01.847440 15783 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:01.850795 15783 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:01.851204 15783 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.852453 15717 raft_consensus.cc:359] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.853298 15717 raft_consensus.cc:385] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.853586 15717 raft_consensus.cc:740] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.858358 15717 consensus_queue.cc:260] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.860579 15717 ts_tablet_manager.cc:1434] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.002s
I20260709 13:18:01.861567 15717 tablet_bootstrap.cc:492] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:01.861629 15424 catalog_manager.cc:5697] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:01.843755 15675 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:01.874086 15802 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:01.891803 15718 tablet_bootstrap.cc:654] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
W20260709 13:18:01.895442 15839 log.cc:927] Time spent T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:18:01.895584 15803 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:01.907232 15803 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:01.913220 15722 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.913702 15722 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.912976 15783 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.924737 15717 tablet_bootstrap.cc:654] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:01.927145 15722 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:01.930817 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:01.930764 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:01.937798 15526 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:01.938028 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:01.938737 15525 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.939586 15783 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:01.940169 15802 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.940665 15802 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.943137 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:01.936004 15600 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:01.943674 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:01.943770 15677 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:01.945448 15641 leader_election.cc:343] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:01.945670 15802 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:01.948210 15718 tablet_bootstrap.cc:492] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:01.948659 15718 ts_tablet_manager.cc:1403] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.156s	user 0.011s	sys 0.003s
I20260709 13:18:01.947952 15641 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:01.952808 15802 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:01.953052 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:01.953274 15802 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:01.953747 15802 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.955253 15720 tablet_bootstrap.cc:492] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:01.955720 15720 ts_tablet_manager.cc:1403] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.168s	user 0.012s	sys 0.040s
I20260709 13:18:01.953764 15600 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:01.962682 15525 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:01.965539 15567 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:01.967051 15642 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:01.967401 15783 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:01.968029 15718 raft_consensus.cc:359] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.968969 15718 raft_consensus.cc:385] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.969287 15718 raft_consensus.cc:740] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.970049 15718 consensus_queue.cc:260] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.972788 15783 raft_consensus.cc:697] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:01.973731 15718 ts_tablet_manager.cc:1434] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:18:01.973680 15783 consensus_queue.cc:237] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.975004 15718 tablet_bootstrap.cc:492] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:01.976560 15755 raft_consensus.cc:2804] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:01.976974 15755 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:01.977672 15720 raft_consensus.cc:359] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.978371 15720 raft_consensus.cc:385] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:01.978638 15720 raft_consensus.cc:740] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:01.979256 15720 consensus_queue.cc:260] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.981056 15755 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:01.985570 15720 ts_tablet_manager.cc:1434] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 13:18:01.986624 15720 tablet_bootstrap.cc:492] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:01.994916 15751 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.997900 15755 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:01.999680 15751 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.002465 15677 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:02.001909 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:02.004106 15755 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:02.003976 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:02.006040 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:02.006274 15526 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:02.006228 15722 consensus_queue.cc:1048] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:02.006815 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:02.007961 15567 leader_election.cc:343] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:02.009146 15600 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:02.011438 15566 leader_election.cc:343] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:02.011861 15566 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:02.012626 15783 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:02.002306 15751 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:02.019029 15717 tablet_bootstrap.cc:492] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:02.019451 15720 tablet_bootstrap.cc:654] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.020339 15717 ts_tablet_manager.cc:1403] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.159s	user 0.016s	sys 0.006s
I20260709 13:18:02.022518 15600 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
W20260709 13:18:02.024013 15845 log.cc:927] Time spent T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:18:02.027525 15718 tablet_bootstrap.cc:654] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.022383 15424 catalog_manager.cc:5697] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:02.030822 15642 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:02.031735 15755 raft_consensus.cc:2804] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:02.032199 15755 raft_consensus.cc:697] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:02.033007 15755 consensus_queue.cc:237] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.036563 15802 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.039696 15526 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:02.041394 15717 raft_consensus.cc:359] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.042487 15802 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:02.045964 15717 raft_consensus.cc:385] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.046341 15717 raft_consensus.cc:740] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:02.047537 15717 consensus_queue.cc:260] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.048919 15751 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.049410 15751 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.050015 15720 tablet_bootstrap.cc:492] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:02.050271 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:02.050478 15720 ts_tablet_manager.cc:1403] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.064s	user 0.012s	sys 0.008s
I20260709 13:18:02.050889 15527 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:02.051426 15751 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:02.053344 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:02.053869 15720 raft_consensus.cc:359] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.054551 15720 raft_consensus.cc:385] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.054438 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:02.054844 15720 raft_consensus.cc:740] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:02.055013 15601 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:02.055552 15720 consensus_queue.cc:260] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.057545 15720 ts_tablet_manager.cc:1434] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:18:02.058471 15720 tablet_bootstrap.cc:492] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:02.058981 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:02.062754 15425 catalog_manager.cc:5697] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:02.064631 15720 tablet_bootstrap.cc:654] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.067546 15753 consensus_queue.cc:1048] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:02.069172 15527 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
W20260709 13:18:02.054864 15566 leader_election.cc:343] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:02.071732 15718 tablet_bootstrap.cc:492] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:02.072441 15718 ts_tablet_manager.cc:1403] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.098s	user 0.012s	sys 0.000s
I20260709 13:18:02.075079 15641 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:18:02.073040 15567 leader_election.cc:343] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:02.075965 15567 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:02.077718 15764 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:02.078156 15802 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:02.075335 15718 raft_consensus.cc:359] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.078706 15802 raft_consensus.cc:697] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:02.078550 15751 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:02.078667 15718 raft_consensus.cc:385] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.079408 15718 raft_consensus.cc:740] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:02.079568 15802 consensus_queue.cc:237] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.080020 15718 consensus_queue.cc:260] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.082335 15718 ts_tablet_manager.cc:1434] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:18:02.084502 15718 tablet_bootstrap.cc:492] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:02.088672 15601 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:02.049664 15717 ts_tablet_manager.cc:1434] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:18:02.093681 15717 tablet_bootstrap.cc:492] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:02.099908 15718 tablet_bootstrap.cc:654] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.100703 15717 tablet_bootstrap.cc:654] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.106591 15424 catalog_manager.cc:5697] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:02.107363 15718 tablet_bootstrap.cc:492] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:02.109519 15717 tablet_bootstrap.cc:492] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:02.112404 15717 ts_tablet_manager.cc:1403] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.005s
I20260709 13:18:02.115453 15717 raft_consensus.cc:359] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.115953 15717 raft_consensus.cc:385] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.116173 15717 raft_consensus.cc:740] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:02.116766 15717 consensus_queue.cc:260] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.110960 15718 ts_tablet_manager.cc:1403] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.002s
I20260709 13:18:02.118685 15717 ts_tablet_manager.cc:1434] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:18:02.111536 15720 tablet_bootstrap.cc:492] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:02.119249 15720 ts_tablet_manager.cc:1403] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.061s	user 0.008s	sys 0.011s
I20260709 13:18:02.119666 15717 tablet_bootstrap.cc:492] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:02.122030 15718 raft_consensus.cc:359] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.125417 15717 tablet_bootstrap.cc:654] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.124758 15720 raft_consensus.cc:359] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.125938 15720 raft_consensus.cc:385] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.126346 15720 raft_consensus.cc:740] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:02.125607 15751 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:02.127179 15720 consensus_queue.cc:260] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.129447 15720 ts_tablet_manager.cc:1434] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:18:02.125543 15718 raft_consensus.cc:385] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.130146 15718 raft_consensus.cc:740] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:02.130967 15718 consensus_queue.cc:260] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.133309 15718 ts_tablet_manager.cc:1434] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:18:02.134467 15718 tablet_bootstrap.cc:492] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:02.138796 15717 tablet_bootstrap.cc:492] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:02.135545 15720 tablet_bootstrap.cc:492] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:02.139294 15717 ts_tablet_manager.cc:1403] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.004s
I20260709 13:18:02.144027 15717 raft_consensus.cc:359] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.144742 15717 raft_consensus.cc:385] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.145048 15717 raft_consensus.cc:740] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:02.145712 15717 consensus_queue.cc:260] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.147761 15717 ts_tablet_manager.cc:1434] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:18:02.148669 15717 tablet_bootstrap.cc:492] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:02.150285 15720 tablet_bootstrap.cc:654] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.157344 15718 tablet_bootstrap.cc:654] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.157344 15717 tablet_bootstrap.cc:654] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.163657 15720 tablet_bootstrap.cc:492] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:02.164088 15720 ts_tablet_manager.cc:1403] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.029s	user 0.007s	sys 0.006s
I20260709 13:18:02.166543 15720 raft_consensus.cc:359] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.167063 15720 raft_consensus.cc:385] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.167268 15720 raft_consensus.cc:740] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:02.167798 15720 consensus_queue.cc:260] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.170184 15720 ts_tablet_manager.cc:1434] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:18:02.170974 15771 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.171437 15771 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.173475 15771 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:02.174892 15720 tablet_bootstrap.cc:492] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:02.177235 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:02.178022 15718 tablet_bootstrap.cc:492] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:02.178150 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:02.178505 15718 ts_tablet_manager.cc:1403] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.044s	user 0.012s	sys 0.000s
I20260709 13:18:02.178746 15601 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:02.178680 15753 consensus_queue.cc:1048] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:02.178172 15677 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:02.180572 15493 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:02.181730 15718 raft_consensus.cc:359] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.182432 15718 raft_consensus.cc:385] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.182682 15718 raft_consensus.cc:740] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:02.183426 15718 consensus_queue.cc:260] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.183821 15797 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:02.184291 15797 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:02.184628 15797 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:02.204698 15718 ts_tablet_manager.cc:1434] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:18:02.205893 15718 tablet_bootstrap.cc:492] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:02.200814 15797 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.215216 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:02.215817 15601 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:02.218457 15717 tablet_bootstrap.cc:492] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:02.218956 15717 ts_tablet_manager.cc:1403] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.070s	user 0.016s	sys 0.035s
I20260709 13:18:02.220358 15718 tablet_bootstrap.cc:654] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.220563 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:02.221189 15675 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:02.221537 15717 raft_consensus.cc:359] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.222297 15717 raft_consensus.cc:385] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.222568 15717 raft_consensus.cc:740] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:02.223156 15601 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:02.224454 15493 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:02.223397 15717 consensus_queue.cc:260] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.225342 15771 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:02.225812 15771 raft_consensus.cc:697] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:02.226675 15771 consensus_queue.cc:237] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.227248 15717 ts_tablet_manager.cc:1434] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:18:02.228354 15717 tablet_bootstrap.cc:492] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:02.232426 15675 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:02.234252 15797 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:02.237411 15717 tablet_bootstrap.cc:654] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.238515 15720 tablet_bootstrap.cc:654] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.238744 15783 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:02.250888 15720 tablet_bootstrap.cc:492] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:02.251624 15720 ts_tablet_manager.cc:1403] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.077s	user 0.010s	sys 0.008s
I20260709 13:18:02.255532 15720 raft_consensus.cc:359] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.256465 15720 raft_consensus.cc:385] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.256736 15720 raft_consensus.cc:740] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:02.257404 15720 consensus_queue.cc:260] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.258313 15722 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.258773 15722 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.261797 15722 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:02.263427 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:18:02.264926 15641 leader_election.cc:343] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:02.266861 15717 tablet_bootstrap.cc:492] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:02.267416 15717 ts_tablet_manager.cc:1403] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.039s	user 0.022s	sys 0.004s
I20260709 13:18:02.266942 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:02.269789 15603 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:02.270540 15717 raft_consensus.cc:359] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.271235 15717 raft_consensus.cc:385] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.271570 15717 raft_consensus.cc:740] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:02.272226 15717 consensus_queue.cc:260] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.271152 15642 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:02.273748 15720 ts_tablet_manager.cc:1434] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.001s
I20260709 13:18:02.273999 15755 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:02.274374 15755 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:02.274828 15755 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:02.282342 15755 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.284612 15755 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:02.285075 15720 tablet_bootstrap.cc:492] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:02.274314 15717 ts_tablet_manager.cc:1434] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:18:02.287137 15717 tablet_bootstrap.cc:492] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:02.289690 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:02.292692 15720 tablet_bootstrap.cc:654] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.294880 15675 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:02.294973 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:02.295706 15603 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:02.300439 15641 leader_election.cc:343] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:02.308956 15764 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:02.311363 15603 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:02.313894 15802 consensus_queue.cc:1048] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:02.322891 15717 tablet_bootstrap.cc:654] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.326557 15642 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:02.327622 15802 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:02.328178 15802 raft_consensus.cc:697] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:02.329103 15802 consensus_queue.cc:237] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.341662 15720 tablet_bootstrap.cc:492] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:02.342180 15720 ts_tablet_manager.cc:1403] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.057s	user 0.034s	sys 0.014s
I20260709 13:18:02.344946 15720 raft_consensus.cc:359] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.345672 15720 raft_consensus.cc:385] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.345985 15720 raft_consensus.cc:740] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:02.346777 15720 consensus_queue.cc:260] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.349097 15720 ts_tablet_manager.cc:1434] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:18:02.356427 15718 tablet_bootstrap.cc:492] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:02.356916 15718 ts_tablet_manager.cc:1403] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.151s	user 0.045s	sys 0.035s
I20260709 13:18:02.356892 15720 tablet_bootstrap.cc:492] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:02.362608 15720 tablet_bootstrap.cc:654] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.363793 15718 raft_consensus.cc:359] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.364454 15718 raft_consensus.cc:385] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.364818 15718 raft_consensus.cc:740] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:02.365729 15718 consensus_queue.cc:260] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.361197 15425 catalog_manager.cc:5697] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:02.368597 15718 ts_tablet_manager.cc:1434] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:18:02.369504 15718 tablet_bootstrap.cc:492] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:02.369894 15717 tablet_bootstrap.cc:492] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:02.370425 15717 ts_tablet_manager.cc:1403] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.084s	user 0.018s	sys 0.003s
I20260709 13:18:02.371044 15771 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.371547 15771 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.373203 15717 raft_consensus.cc:359] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.374027 15717 raft_consensus.cc:385] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.374379 15717 raft_consensus.cc:740] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:02.375797 15718 tablet_bootstrap.cc:654] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.375592 15717 consensus_queue.cc:260] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.377064 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:02.378274 15717 ts_tablet_manager.cc:1434] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:18:02.379400 15717 tablet_bootstrap.cc:492] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:02.381330 15720 tablet_bootstrap.cc:492] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:02.381825 15720 ts_tablet_manager.cc:1403] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.032s	user 0.012s	sys 0.002s
I20260709 13:18:02.377807 15677 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:02.384298 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:02.384779 15491 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:02.385659 15724 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:02.386088 15724 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:02.373581 15771 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:02.386425 15724 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:02.364843 15426 catalog_manager.cc:5697] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:02.389645 15720 raft_consensus.cc:359] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.390364 15720 raft_consensus.cc:385] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.390622 15720 raft_consensus.cc:740] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:02.391287 15720 consensus_queue.cc:260] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.393959 15601 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:02.397228 15724 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.399931 15720 ts_tablet_manager.cc:1434] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 13:18:02.400908 15720 tablet_bootstrap.cc:492] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:02.401079 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:02.401659 15601 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:02.403787 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:02.403990 15771 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.404394 15677 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:02.404502 15771 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.406427 15720 tablet_bootstrap.cc:654] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.410225 15601 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:02.410960 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:02.411705 15675 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:02.413159 15491 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:02.414130 15803 raft_consensus.cc:2804] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:02.414536 15803 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:02.414858 15803 raft_consensus.cc:3060] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:02.419085 15493 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:02.419350 15797 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.419931 15797 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.421002 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:02.421627 15601 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:02.427003 15797 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:02.427822 15797 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:02.428397 15797 raft_consensus.cc:697] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:02.429348 15797 consensus_queue.cc:237] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.430922 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:02.431680 15675 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:02.433264 15491 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:02.436813 15764 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:02.437736 15677 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:02.438287 15751 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.439133 15751 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.441288 15751 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:02.446244 15717 tablet_bootstrap.cc:654] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.446484 15771 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:02.449234 15771 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:02.449666 15771 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:02.450054 15771 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:02.451668 15718 tablet_bootstrap.cc:492] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:02.452171 15718 ts_tablet_manager.cc:1403] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.083s	user 0.037s	sys 0.021s
I20260709 13:18:02.455551 15718 raft_consensus.cc:359] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.456480 15718 raft_consensus.cc:385] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.456771 15718 raft_consensus.cc:740] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:02.457906 15718 consensus_queue.cc:260] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.458518 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:02.459561 15675 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:02.460207 15718 ts_tablet_manager.cc:1434] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:18:02.460934 15566 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:02.461306 15718 tablet_bootstrap.cc:492] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:02.461949 15751 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:02.462376 15751 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:02.462713 15751 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:02.467985 15718 tablet_bootstrap.cc:654] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
W20260709 13:18:02.473333 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 54370.5us
I20260709 13:18:02.475612 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:02.476509 15720 tablet_bootstrap.cc:492] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:02.477020 15720 ts_tablet_manager.cc:1403] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.076s	user 0.035s	sys 0.035s
I20260709 13:18:02.480116 15720 raft_consensus.cc:359] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.480767 15720 raft_consensus.cc:385] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.481036 15720 raft_consensus.cc:740] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:02.482474 15783 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.482946 15783 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.483187 15720 consensus_queue.cc:260] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.484406 15718 tablet_bootstrap.cc:492] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:02.484959 15718 ts_tablet_manager.cc:1403] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.024s	user 0.019s	sys 0.003s
I20260709 13:18:02.484993 15783 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:02.486985 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:02.487954 15718 raft_consensus.cc:359] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:02.488629 15566 leader_election.cc:343] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:02.488732 15718 raft_consensus.cc:385] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.489257 15718 raft_consensus.cc:740] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:02.490052 15718 consensus_queue.cc:260] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.492334 15718 ts_tablet_manager.cc:1434] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:18:02.492648 15720 ts_tablet_manager.cc:1434] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.002s
I20260709 13:18:02.493531 15718 tablet_bootstrap.cc:492] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:02.495721 15803 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.499148 15751 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.500463 15601 raft_consensus.cc:2393] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 1: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:02.501080 15718 tablet_bootstrap.cc:654] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.502856 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:02.503535 15675 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:02.506368 15751 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:02.506978 15878 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.507542 15878 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.510215 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:02.510459 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:02.511796 15755 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.512596 15718 tablet_bootstrap.cc:492] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:02.512424 15755 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.513367 15718 ts_tablet_manager.cc:1403] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.020s	user 0.007s	sys 0.011s
I20260709 13:18:02.513325 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:02.515791 15802 consensus_queue.cc:1048] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:02.516397 15718 raft_consensus.cc:359] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.517071 15718 raft_consensus.cc:385] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.517045 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:02.517488 15718 raft_consensus.cc:740] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:02.517728 15601 raft_consensus.cc:3060] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:02.518163 15718 consensus_queue.cc:260] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.520336 15718 ts_tablet_manager.cc:1434] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:18:02.524072 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:02.524506 15601 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:02.524856 15603 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:02.526590 15642 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:02.527529 15755 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:02.528945 15783 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.529508 15783 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.446682 15724 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:02.530210 15720 tablet_bootstrap.cc:492] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:02.530211 15718 tablet_bootstrap.cc:492] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:02.532182 15783 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:02.536128 15753 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:02.538766 15720 tablet_bootstrap.cc:654] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.543076 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:02.543840 15677 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
W20260709 13:18:02.543946 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 110808us
I20260709 13:18:02.542923 15771 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.545506 15527 raft_consensus.cc:2393] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 1: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:18:02.545688 15566 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:02.546720 15751 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:02.536223 15718 tablet_bootstrap.cc:654] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.547072 15751 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:02.547549 15751 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:02.550060 15803 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:02.552757 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:02.553687 15601 raft_consensus.cc:2393] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 1: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:18:02.559068 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:02.559736 15528 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:02.560441 15720 tablet_bootstrap.cc:492] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:02.560921 15720 ts_tablet_manager.cc:1403] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.031s	user 0.006s	sys 0.004s
I20260709 13:18:02.563513 15720 raft_consensus.cc:359] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.564172 15720 raft_consensus.cc:385] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.564932 15720 raft_consensus.cc:740] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:02.565062 15675 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:02.566113 15718 tablet_bootstrap.cc:492] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:02.565649 15720 consensus_queue.cc:260] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.566581 15718 ts_tablet_manager.cc:1403] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.045s	user 0.007s	sys 0.011s
I20260709 13:18:02.567337 15878 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:02.568106 15720 ts_tablet_manager.cc:1434] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:18:02.568256 15717 tablet_bootstrap.cc:492] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:02.568526 15771 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:02.568809 15717 ts_tablet_manager.cc:1403] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.190s	user 0.049s	sys 0.047s
I20260709 13:18:02.569088 15720 tablet_bootstrap.cc:492] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
W20260709 13:18:02.569502 15567 leader_election.cc:343] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:02.570104 15567 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:02.569310 15718 raft_consensus.cc:359] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.570989 15718 raft_consensus.cc:385] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.571259 15718 raft_consensus.cc:740] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
W20260709 13:18:02.571866 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 45397.8us
I20260709 13:18:02.571486 15764 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:02.572345 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:02.572970 15527 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:02.574378 15493 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:02.564169 15751 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.575291 15718 consensus_queue.cc:260] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.579396 15764 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:18:02.579358 15797 raft_consensus.cc:2804] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:02.582183 15797 raft_consensus.cc:697] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:02.582733 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:02.574990 15566 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:02.582928 15753 consensus_queue.cc:1048] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:02.584017 15764 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:02.584316 15718 ts_tablet_manager.cc:1434] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.002s
W20260709 13:18:02.571671 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 48492.1us
I20260709 13:18:02.584506 15764 raft_consensus.cc:697] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:02.585354 15764 consensus_queue.cc:237] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.583009 15797 consensus_queue.cc:237] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.589742 15717 raft_consensus.cc:359] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.590510 15717 raft_consensus.cc:385] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.590775 15717 raft_consensus.cc:740] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:02.591586 15717 consensus_queue.cc:260] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.578984 15755 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:02.595651 15717 ts_tablet_manager.cc:1434] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.001s
I20260709 13:18:02.596638 15717 tablet_bootstrap.cc:492] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:02.590888 15751 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:02.597105 15720 tablet_bootstrap.cc:654] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.599156 15755 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:02.626688 15755 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:02.615307 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:02.778376 15751 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.786803 15751 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.788910 15751 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:02.775992 15755 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.811517 15675 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 1: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:18:02.811601 15720 tablet_bootstrap.cc:492] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:02.812354 15425 catalog_manager.cc:5697] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:02.812274 15720 ts_tablet_manager.cc:1403] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.243s	user 0.007s	sys 0.007s
I20260709 13:18:02.818220 15717 tablet_bootstrap.cc:654] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.823314 15897 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.823874 15897 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.817054 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:02.831349 15675 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
W20260709 13:18:02.832762 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 250594us
I20260709 13:18:02.836577 15755 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:02.838726 15771 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.839286 15771 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.828218 15720 raft_consensus.cc:359] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.840509 15720 raft_consensus.cc:385] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:02.840827 15720 raft_consensus.cc:740] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:02.841495 15720 consensus_queue.cc:260] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.846120 15718 tablet_bootstrap.cc:492] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:02.850149 15897 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:02.851311 15720 ts_tablet_manager.cc:1434] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.001s
I20260709 13:18:02.853302 15720 tablet_bootstrap.cc:492] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:02.853433 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:02.854091 15528 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:02.866817 15474 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:02.869262 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:02.870500 15675 raft_consensus.cc:3060] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:02.863142 15566 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:02.874752 15803 consensus_queue.cc:1048] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:02.875243 15751 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:02.879634 15751 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:02.880017 15751 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:02.887022 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:02.899477 15600 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 1: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:02.902139 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:02.884701 15528 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:02.898481 15717 tablet_bootstrap.cc:492] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:02.890619 15753 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:18:02.911480 15717 ts_tablet_manager.cc:1403] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.315s	user 0.022s	sys 0.014s
I20260709 13:18:02.909479 15771 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:02.924252 15720 tablet_bootstrap.cc:654] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.906479 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:02.925627 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:02.925879 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:02.925885 15526 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:02.914757 15675 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:02.932914 15567 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:02.935178 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:02.936120 15600 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:02.936790 15896 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.937335 15783 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:18:02.937335 15896 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:02.939954 15567 leader_election.cc:343] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:02.940343 15896 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:02.946355 15425 catalog_manager.cc:5697] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:02.948441 15425 catalog_manager.cc:5697] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:02.947525 15641 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:02.947440 15526 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:02.953533 15718 tablet_bootstrap.cc:654] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:02.958604 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:02.960198 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:02.960932 15676 raft_consensus.cc:2393] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 1: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:02.962493 15797 consensus_queue.cc:1048] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:02.962666 15493 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:02.966521 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:02.969297 15566 leader_election.cc:343] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:02.969816 15566 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:02.969851 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:02.970654 15764 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:02.963505 15753 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:02.972390 15751 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.975986 15797 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:02.976425 15797 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:02.976797 15797 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:02.972350 15753 raft_consensus.cc:697] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:02.978142 15751 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:02.977954 15753 consensus_queue.cc:237] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.981467 15720 tablet_bootstrap.cc:492] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:02.982409 15720 ts_tablet_manager.cc:1403] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.130s	user 0.013s	sys 0.013s
I20260709 13:18:02.976815 15491 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:18:02.985385 15566 leader_election.cc:343] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:02.987176 15491 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:02.987923 15803 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:02.988530 15803 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:18:02.989069 15803 consensus_queue.cc:1048] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:02.989562 15797 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:02.992677 15722 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:02.994346 15797 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:03.005432 15717 raft_consensus.cc:359] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.009588 15717 raft_consensus.cc:385] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.009882 15717 raft_consensus.cc:740] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.010778 15717 consensus_queue.cc:260] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.013054 15720 raft_consensus.cc:359] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.013723 15720 raft_consensus.cc:385] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.014050 15720 raft_consensus.cc:740] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.017683 15677 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:03.021850 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:03.008741 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:03.020257 15720 consensus_queue.cc:260] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.036375 15720 ts_tablet_manager.cc:1434] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.053s	user 0.006s	sys 0.001s
I20260709 13:18:03.037456 15720 tablet_bootstrap.cc:492] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:03.039347 15717 ts_tablet_manager.cc:1434] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.123s	user 0.008s	sys 0.000s
I20260709 13:18:03.040263 15717 tablet_bootstrap.cc:492] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:03.040386 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:03.029524 15675 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:03.033219 15674 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:03.041046 15524 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:03.035385 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:03.047675 15895 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:03.049727 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:03.051602 15718 tablet_bootstrap.cc:492] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:03.051981 15718 ts_tablet_manager.cc:1403] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.206s	user 0.013s	sys 0.005s
I20260709 13:18:03.050410 15602 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:03.086031 15567 leader_election.cc:343] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:03.085883 15879 consensus_queue.cc:1048] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:03.088274 15718 raft_consensus.cc:359] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.089567 15718 raft_consensus.cc:385] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.091259 15896 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:03.093703 15718 raft_consensus.cc:740] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.094481 15718 consensus_queue.cc:260] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.079394 15771 consensus_queue.cc:1048] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:18:03.097121 15567 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:03.099462 15896 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:03.096894 15602 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:03.108315 15675 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:03.113524 15566 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:03.114524 15896 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:03.115056 15896 raft_consensus.cc:697] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:03.115906 15896 consensus_queue.cc:237] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.101753 15524 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:03.147984 15718 ts_tablet_manager.cc:1434] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.096s	user 0.007s	sys 0.000s
I20260709 13:18:03.148952 15718 tablet_bootstrap.cc:492] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:03.151906 15720 tablet_bootstrap.cc:654] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.157104 15717 tablet_bootstrap.cc:654] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.156028 15493 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:03.162050 15751 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.162642 15751 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.167655 15803 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:03.179836 15803 raft_consensus.cc:697] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:03.180701 15803 consensus_queue.cc:237] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.181700 15674 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:03.241142 15751 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:03.387831 15751 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.388305 15751 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.282290 15426 catalog_manager.cc:5697] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
W20260709 13:18:03.344165 15911 log.cc:927] Time spent T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 13:18:03.378950 15908 log.cc:927] Time spent T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.178s	user 0.018s	sys 0.018s
I20260709 13:18:03.375404 15879 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.245090 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:03.403761 15879 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:03.417734 15912 log.cc:927] Time spent T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 13:18:03.421255 15720 tablet_bootstrap.cc:492] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:03.421752 15720 ts_tablet_manager.cc:1403] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.385s	user 0.020s	sys 0.012s
I20260709 13:18:03.422878 15751 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:03.426445 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:03.430899 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:03.436065 15879 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:03.426672 15425 catalog_manager.cc:5697] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:03.446789 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:03.452881 15718 tablet_bootstrap.cc:654] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
W20260709 13:18:03.454712 15567 leader_election.cc:343] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:18:03.455729 15567 leader_election.cc:343] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:03.456990 15764 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:03.490513 15424 catalog_manager.cc:5697] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:03.506623 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:03.506835 15895 consensus_queue.cc:1048] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:18:03.508157 15491 leader_election.cc:343] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:03.508713 15717 tablet_bootstrap.cc:492] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:03.510674 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:03.511369 15600 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:03.512696 15720 raft_consensus.cc:359] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.513279 15720 raft_consensus.cc:385] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.513540 15720 raft_consensus.cc:740] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.514170 15720 consensus_queue.cc:260] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:03.515822 15566 leader_election.cc:343] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:03.516287 15566 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:03.523773 15764 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:03.518630 15724 consensus_queue.cc:1048] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:18:03.529959 15566 leader_election.cc:343] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:03.530431 15566 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:03.531983 15751 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:03.536116 15718 tablet_bootstrap.cc:492] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:03.536545 15718 ts_tablet_manager.cc:1403] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.388s	user 0.010s	sys 0.001s
I20260709 13:18:03.519462 15493 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:03.537535 15724 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:03.537895 15724 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:03.538379 15724 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:03.539065 15718 raft_consensus.cc:359] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.539733 15718 raft_consensus.cc:385] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.539991 15718 raft_consensus.cc:740] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.540591 15718 consensus_queue.cc:260] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.542294 15718 ts_tablet_manager.cc:1434] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:18:03.543097 15718 tablet_bootstrap.cc:492] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:03.522207 15720 ts_tablet_manager.cc:1434] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.100s	user 0.011s	sys 0.002s
I20260709 13:18:03.522534 15717 ts_tablet_manager.cc:1403] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.482s	user 0.012s	sys 0.027s
I20260709 13:18:03.545549 15720 tablet_bootstrap.cc:492] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:03.552094 15717 raft_consensus.cc:359] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.553632 15717 raft_consensus.cc:385] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.554083 15717 raft_consensus.cc:740] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.554659 15724 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.555444 15718 tablet_bootstrap.cc:654] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.557551 15717 consensus_queue.cc:260] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.561805 15724 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:03.564230 15717 ts_tablet_manager.cc:1434] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:18:03.564894 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:03.565160 15717 tablet_bootstrap.cc:492] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
W20260709 13:18:03.573532 15491 leader_election.cc:343] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:03.580149 15717 tablet_bootstrap.cc:654] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.583145 15718 tablet_bootstrap.cc:492] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:03.583711 15718 ts_tablet_manager.cc:1403] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.041s	user 0.012s	sys 0.000s
I20260709 13:18:03.588212 15718 raft_consensus.cc:359] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.588919 15718 raft_consensus.cc:385] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.589184 15718 raft_consensus.cc:740] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.592849 15718 consensus_queue.cc:260] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.598271 15718 ts_tablet_manager.cc:1434] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.002s
I20260709 13:18:03.599277 15718 tablet_bootstrap.cc:492] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:03.604961 15718 tablet_bootstrap.cc:654] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.612032 15720 tablet_bootstrap.cc:654] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.618882 15920 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:03.623544 15895 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.624089 15895 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.626782 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:03.627430 15601 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:03.628122 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:03.629726 15600 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:03.629599 15892 consensus_queue.cc:1048] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:03.631208 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:03.632402 15890 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:03.632872 15890 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:03.633325 15890 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:03.636857 15724 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.637358 15724 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.638350 15895 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:03.639119 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:03.639765 15526 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:03.640677 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:03.639413 15724 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:03.646711 15601 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:03.650434 15493 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:03.652240 15724 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:03.652755 15724 raft_consensus.cc:697] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:03.653566 15724 consensus_queue.cc:237] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.658994 15720 tablet_bootstrap.cc:492] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:03.659487 15720 ts_tablet_manager.cc:1403] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.114s	user 0.009s	sys 0.000s
I20260709 13:18:03.660104 15718 tablet_bootstrap.cc:492] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:03.660538 15718 ts_tablet_manager.cc:1403] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.061s	user 0.008s	sys 0.007s
I20260709 13:18:03.664834 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:03.665627 15601 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:03.662123 15720 raft_consensus.cc:359] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.667714 15720 raft_consensus.cc:385] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.668009 15720 raft_consensus.cc:740] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.668735 15720 consensus_queue.cc:260] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.670843 15720 ts_tablet_manager.cc:1434] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:18:03.669312 15717 tablet_bootstrap.cc:492] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:03.665652 15718 raft_consensus.cc:359] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.671782 15718 raft_consensus.cc:385] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.671877 15717 ts_tablet_manager.cc:1403] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.107s	user 0.027s	sys 0.004s
I20260709 13:18:03.672078 15718 raft_consensus.cc:740] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.672760 15718 consensus_queue.cc:260] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.664099 15890 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.675554 15673 raft_consensus.cc:2393] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 1: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:18:03.676265 15890 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:03.686879 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:03.687352 15720 tablet_bootstrap.cc:492] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:03.688364 15601 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:03.678608 15896 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:18:03.680732 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:03.695698 15525 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:03.680617 15493 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:03.702873 15601 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:03.675578 15717 raft_consensus.cc:359] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.704953 15717 raft_consensus.cc:385] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.705292 15717 raft_consensus.cc:740] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.704800 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:03.706264 15890 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:03.706037 15717 consensus_queue.cc:260] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.706784 15890 raft_consensus.cc:697] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:03.708388 15717 ts_tablet_manager.cc:1434] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.001s
I20260709 13:18:03.707597 15890 consensus_queue.cc:237] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.709368 15717 tablet_bootstrap.cc:492] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:03.714409 15720 tablet_bootstrap.cc:654] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.677634 15718 ts_tablet_manager.cc:1434] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.003s
I20260709 13:18:03.718031 15718 tablet_bootstrap.cc:492] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:03.715291 15717 tablet_bootstrap.cc:654] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.731221 15426 catalog_manager.cc:5697] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:03.715399 15525 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:03.733817 15797 raft_consensus.cc:2764] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:18:03.736956 15718 tablet_bootstrap.cc:654] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.745415 15896 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.745982 15896 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.747972 15896 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:03.748735 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:18:03.750025 15567 leader_election.cc:343] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:03.753997 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:03.755316 15566 leader_election.cc:343] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:03.755581 15720 tablet_bootstrap.cc:492] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:03.755770 15566 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:03.756152 15720 ts_tablet_manager.cc:1403] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.069s	user 0.013s	sys 0.016s
I20260709 13:18:03.757689 15897 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:03.758973 15720 raft_consensus.cc:359] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.759698 15720 raft_consensus.cc:385] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.760010 15720 raft_consensus.cc:740] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.760493 15718 tablet_bootstrap.cc:492] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:03.760946 15718 ts_tablet_manager.cc:1403] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.043s	user 0.010s	sys 0.000s
I20260709 13:18:03.760869 15720 consensus_queue.cc:260] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.763742 15717 tablet_bootstrap.cc:492] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:03.763043 15424 catalog_manager.cc:5697] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:03.764221 15717 ts_tablet_manager.cc:1403] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.055s	user 0.008s	sys 0.003s
I20260709 13:18:03.766132 15896 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.766598 15896 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.766924 15717 raft_consensus.cc:359] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.767531 15717 raft_consensus.cc:385] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.767800 15717 raft_consensus.cc:740] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.768445 15717 consensus_queue.cc:260] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.769949 15896 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:03.771648 15720 ts_tablet_manager.cc:1434] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:18:03.771636 15718 raft_consensus.cc:359] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.772400 15718 raft_consensus.cc:385] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.772666 15718 raft_consensus.cc:740] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.772889 15720 tablet_bootstrap.cc:492] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:03.773312 15718 consensus_queue.cc:260] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.774406 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:03.777799 15717 ts_tablet_manager.cc:1434] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:18:03.778790 15717 tablet_bootstrap.cc:492] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:03.780288 15720 tablet_bootstrap.cc:654] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
W20260709 13:18:03.780457 15566 leader_election.cc:343] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:03.784451 15717 tablet_bootstrap.cc:654] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.790263 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:18:03.791738 15567 leader_election.cc:343] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:03.792268 15718 ts_tablet_manager.cc:1434] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 13:18:03.792177 15567 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:03.793273 15896 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:03.793449 15724 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.793964 15724 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.796802 15896 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.797292 15896 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.798266 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:03.798879 15601 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:03.801153 15493 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:03.802256 15896 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:03.803663 15879 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:03.804096 15879 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:03.804420 15879 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:03.808800 15918 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:03.808969 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:18:03.811462 15567 leader_election.cc:343] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:03.812371 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:03.813736 15566 leader_election.cc:343] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:03.816355 15566 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:03.817118 15896 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:03.819625 15724 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:03.793236 15718 tablet_bootstrap.cc:492] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:03.822288 15879 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.819895 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:03.829829 15718 tablet_bootstrap.cc:654] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.830224 15673 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:03.831045 15720 tablet_bootstrap.cc:492] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:03.831511 15720 ts_tablet_manager.cc:1403] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.059s	user 0.023s	sys 0.017s
I20260709 13:18:03.834117 15720 raft_consensus.cc:359] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.834795 15720 raft_consensus.cc:385] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.835054 15720 raft_consensus.cc:740] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.835800 15720 consensus_queue.cc:260] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.838459 15720 ts_tablet_manager.cc:1434] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:18:03.839376 15720 tablet_bootstrap.cc:492] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:03.845053 15720 tablet_bootstrap.cc:654] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.852460 15718 tablet_bootstrap.cc:492] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:03.852226 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:03.852962 15718 ts_tablet_manager.cc:1403] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.060s	user 0.009s	sys 0.008s
I20260709 13:18:03.853014 15602 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:03.853471 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:03.854080 15673 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:03.855741 15718 raft_consensus.cc:359] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.858807 15718 raft_consensus.cc:385] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.859097 15718 raft_consensus.cc:740] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.860220 15602 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:03.861512 15493 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:03.862437 15724 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:03.859710 15718 consensus_queue.cc:260] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.862982 15724 raft_consensus.cc:697] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:03.864126 15724 consensus_queue.cc:237] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.865177 15718 ts_tablet_manager.cc:1434] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:18:03.866252 15718 tablet_bootstrap.cc:492] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:03.872601 15879 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:03.879799 15895 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.880417 15895 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.890655 15920 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.891219 15920 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.892513 15673 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:03.893481 15920 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:03.896960 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:03.897074 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:03.897713 15525 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:03.899960 15641 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:18:03.900463 15567 leader_election.cc:343] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:03.900828 15892 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:03.901228 15892 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:03.901548 15892 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:03.902901 15803 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:03.903807 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:03.905418 15566 leader_election.cc:343] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:03.905844 15566 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:03.915273 15896 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:03.918372 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:03.919147 15602 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:03.920979 15717 tablet_bootstrap.cc:492] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:03.921816 15426 catalog_manager.cc:5697] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:03.922667 15717 ts_tablet_manager.cc:1403] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.144s	user 0.022s	sys 0.026s
I20260709 13:18:03.909567 15892 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.921571 15895 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:03.927223 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:03.927902 15525 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:03.928504 15717 raft_consensus.cc:359] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.930048 15717 raft_consensus.cc:385] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.930801 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:03.931212 15717 raft_consensus.cc:740] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.929700 15892 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:03.932518 15717 consensus_queue.cc:260] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.931524 15600 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:03.935034 15717 ts_tablet_manager.cc:1434] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.002s
I20260709 13:18:03.936029 15717 tablet_bootstrap.cc:492] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:03.941623 15600 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:03.943040 15642 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:03.943958 15892 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:03.944489 15892 raft_consensus.cc:697] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:03.945361 15892 consensus_queue.cc:237] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.957412 15717 tablet_bootstrap.cc:654] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.957760 15720 tablet_bootstrap.cc:492] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:03.960683 15720 ts_tablet_manager.cc:1403] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.122s	user 0.020s	sys 0.039s
I20260709 13:18:03.959635 15525 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:03.966527 15720 raft_consensus.cc:359] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.967269 15720 raft_consensus.cc:385] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.967532 15720 raft_consensus.cc:740] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.968281 15720 consensus_queue.cc:260] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.970402 15720 ts_tablet_manager.cc:1434] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 13:18:03.971329 15720 tablet_bootstrap.cc:492] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:03.970827 15718 tablet_bootstrap.cc:654] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.971916 15717 tablet_bootstrap.cc:492] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:03.972601 15717 ts_tablet_manager.cc:1403] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.037s	user 0.010s	sys 0.007s
I20260709 13:18:03.975245 15717 raft_consensus.cc:359] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.975997 15717 raft_consensus.cc:385] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.976321 15717 raft_consensus.cc:740] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.977394 15720 tablet_bootstrap.cc:654] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.979095 15717 consensus_queue.cc:260] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:03.981317 15717 ts_tablet_manager.cc:1434] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 13:18:03.982954 15717 tablet_bootstrap.cc:492] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:03.988583 15717 tablet_bootstrap.cc:654] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.015741 15718 tablet_bootstrap.cc:492] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:04.016287 15718 ts_tablet_manager.cc:1403] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.150s	user 0.012s	sys 0.000s
I20260709 13:18:04.019825 15718 raft_consensus.cc:359] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.020629 15718 raft_consensus.cc:385] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.020880 15718 raft_consensus.cc:740] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.021451 15718 consensus_queue.cc:260] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.022915 15718 ts_tablet_manager.cc:1434] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
W20260709 13:18:04.023734 15474 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:04.024097 15426 catalog_manager.cc:5697] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:04.031800 15718 tablet_bootstrap.cc:492] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:04.033085 15717 tablet_bootstrap.cc:492] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:04.033622 15717 ts_tablet_manager.cc:1403] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.051s	user 0.022s	sys 0.020s
I20260709 13:18:04.035599 15720 tablet_bootstrap.cc:492] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:04.036062 15720 ts_tablet_manager.cc:1403] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.065s	user 0.034s	sys 0.018s
I20260709 13:18:04.037730 15718 tablet_bootstrap.cc:654] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.037397 15717 raft_consensus.cc:359] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.038218 15717 raft_consensus.cc:385] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.038853 15717 raft_consensus.cc:740] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.039027 15720 raft_consensus.cc:359] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.039785 15720 raft_consensus.cc:385] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.040078 15720 raft_consensus.cc:740] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.040719 15720 consensus_queue.cc:260] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.042866 15720 ts_tablet_manager.cc:1434] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:18:04.043896 15720 tablet_bootstrap.cc:492] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:04.050257 15892 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.049759 15720 tablet_bootstrap.cc:654] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.050813 15892 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.052996 15892 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:04.053687 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:04.056181 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:04.056370 15879 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.056991 15600 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 1: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:18:04.056869 15879 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.050773 15718 tablet_bootstrap.cc:492] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:04.058434 15641 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:04.058753 15718 ts_tablet_manager.cc:1403] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.002s
I20260709 13:18:04.060626 15879 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:04.062081 15890 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.062135 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:04.064571 15718 raft_consensus.cc:359] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.066941 15718 raft_consensus.cc:385] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.067195 15718 raft_consensus.cc:740] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.067803 15718 consensus_queue.cc:260] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.075479 15897 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.076278 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:04.076027 15897 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.076911 15602 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:04.078564 15897 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:04.079594 15718 ts_tablet_manager.cc:1434] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.001s
I20260709 13:18:04.080567 15718 tablet_bootstrap.cc:492] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:04.081408 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:04.082702 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:18:04.083254 15566 leader_election.cc:343] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:04.083369 15525 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:04.081487 15493 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:04.084364 15879 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:04.084764 15879 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:04.085150 15879 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.086436 15718 tablet_bootstrap.cc:654] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.091774 15890 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:04.092363 15879 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.039613 15717 consensus_queue.cc:260] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.095276 15676 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:04.097853 15797 consensus_queue.cc:1048] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:04.100684 15918 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.101279 15717 ts_tablet_manager.cc:1434] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.067s	user 0.003s	sys 0.004s
I20260709 13:18:04.101357 15567 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:04.101210 15918 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.103807 15717 tablet_bootstrap.cc:492] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:04.105293 15918 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:04.105521 15751 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:04.105990 15751 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:04.106333 15751 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.102607 15718 tablet_bootstrap.cc:492] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:04.107163 15718 ts_tablet_manager.cc:1403] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.004s
I20260709 13:18:04.109862 15718 raft_consensus.cc:359] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.110572 15718 raft_consensus.cc:385] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.110868 15718 raft_consensus.cc:740] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.110997 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:04.111653 15718 consensus_queue.cc:260] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:04.112663 15491 leader_election.cc:343] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:04.113636 15717 tablet_bootstrap.cc:654] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.116266 15720 tablet_bootstrap.cc:492] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:04.116539 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:04.116755 15720 ts_tablet_manager.cc:1403] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.073s	user 0.027s	sys 0.040s
I20260709 13:18:04.113886 15718 ts_tablet_manager.cc:1434] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:18:04.117156 15600 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:04.118081 15718 tablet_bootstrap.cc:492] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:04.119611 15720 raft_consensus.cc:359] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.120302 15720 raft_consensus.cc:385] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.120550 15720 raft_consensus.cc:740] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.121270 15720 consensus_queue.cc:260] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.123242 15720 ts_tablet_manager.cc:1434] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:18:04.123204 15751 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.124284 15720 tablet_bootstrap.cc:492] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:04.124390 15718 tablet_bootstrap.cc:654] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.129858 15493 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:04.131772 15720 tablet_bootstrap.cc:654] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.132364 15918 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:04.132762 15918 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:04.133112 15918 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.136780 15717 tablet_bootstrap.cc:492] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:04.136970 15718 tablet_bootstrap.cc:492] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:04.137269 15717 ts_tablet_manager.cc:1403] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.008s
I20260709 13:18:04.137531 15718 ts_tablet_manager.cc:1403] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.005s
I20260709 13:18:04.137843 15720 tablet_bootstrap.cc:492] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:04.138299 15720 ts_tablet_manager.cc:1403] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 13:18:04.139951 15717 raft_consensus.cc:359] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.140091 15718 raft_consensus.cc:359] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.140712 15717 raft_consensus.cc:385] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.140998 15717 raft_consensus.cc:740] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.140969 15720 raft_consensus.cc:359] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.141535 15918 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.142385 15720 raft_consensus.cc:385] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.142369 15717 consensus_queue.cc:260] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.142829 15720 raft_consensus.cc:740] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.140982 15718 raft_consensus.cc:385] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.143994 15718 raft_consensus.cc:740] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.144554 15717 ts_tablet_manager.cc:1434] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:18:04.144701 15718 consensus_queue.cc:260] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.145519 15717 tablet_bootstrap.cc:492] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:04.146998 15718 ts_tablet_manager.cc:1434] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:18:04.146536 15720 consensus_queue.cc:260] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.148272 15718 tablet_bootstrap.cc:492] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:04.148789 15720 ts_tablet_manager.cc:1434] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.004s
I20260709 13:18:04.149710 15720 tablet_bootstrap.cc:492] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:04.151636 15717 tablet_bootstrap.cc:654] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.155400 15718 tablet_bootstrap.cc:654] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.155483 15720 tablet_bootstrap.cc:654] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.155912 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:18:04.157493 15566 leader_election.cc:343] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:04.158685 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:04.159353 15528 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.168820 15718 tablet_bootstrap.cc:492] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:04.169516 15718 ts_tablet_manager.cc:1403] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.002s
I20260709 13:18:04.172535 15718 raft_consensus.cc:359] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.173271 15718 raft_consensus.cc:385] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.173341 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:04.173630 15528 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:04.173672 15718 raft_consensus.cc:740] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.173969 15602 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.174757 15718 consensus_queue.cc:260] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.175097 15567 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:04.176831 15718 ts_tablet_manager.cc:1434] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:18:04.177846 15718 tablet_bootstrap.cc:492] T 4b9f476ac2ad44cd8124fde4f1aadc78 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:04.180574 15602 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:04.180979 15897 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:04.181777 15897 raft_consensus.cc:697] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:04.181813 15493 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:04.182734 15803 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:04.183260 15803 raft_consensus.cc:697] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:04.184182 15803 consensus_queue.cc:237] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.186025 15718 tablet_bootstrap.cc:654] T 4b9f476ac2ad44cd8124fde4f1aadc78 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.187038 15897 consensus_queue.cc:237] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.192965 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:04.193626 15602 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.200850 15718 tablet_bootstrap.cc:492] T 4b9f476ac2ad44cd8124fde4f1aadc78 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:04.201440 15718 ts_tablet_manager.cc:1403] T 4b9f476ac2ad44cd8124fde4f1aadc78 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.006s
I20260709 13:18:04.203837 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:18:04.205325 15491 leader_election.cc:343] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:04.209275 15918 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:04.209437 15602 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:04.210335 15751 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:04.210829 15493 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:04.204128 15718 raft_consensus.cc:359] T 4b9f476ac2ad44cd8124fde4f1aadc78 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.212133 15718 raft_consensus.cc:385] T 4b9f476ac2ad44cd8124fde4f1aadc78 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.212419 15718 raft_consensus.cc:740] T 4b9f476ac2ad44cd8124fde4f1aadc78 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.213228 15718 consensus_queue.cc:260] T 4b9f476ac2ad44cd8124fde4f1aadc78 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.214218 15771 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:04.214740 15771 raft_consensus.cc:697] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:04.215232 15718 ts_tablet_manager.cc:1434] T 4b9f476ac2ad44cd8124fde4f1aadc78 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.013s	user 0.002s	sys 0.003s
I20260709 13:18:04.216161 15718 tablet_bootstrap.cc:492] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:04.215626 15771 consensus_queue.cc:237] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.218735 15724 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.219303 15724 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.221226 15718 tablet_bootstrap.cc:654] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.223666 15797 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.224236 15797 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.226758 15797 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:04.221598 15724 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:04.228766 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:04.229523 15600 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:04.229652 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:04.231941 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:04.232309 15493 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:04.232591 15677 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:04.233384 15724 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:04.233850 15724 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:04.234279 15724 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:04.234304 15491 leader_election.cc:343] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:04.241150 15717 tablet_bootstrap.cc:492] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:04.241647 15717 ts_tablet_manager.cc:1403] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.096s	user 0.027s	sys 0.018s
I20260709 13:18:04.244218 15717 raft_consensus.cc:359] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.244460 15897 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.244928 15717 raft_consensus.cc:385] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.245246 15717 raft_consensus.cc:740] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.245337 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:04.245297 15897 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.249806 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:04.250595 15525 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:04.254089 15567 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:04.255824 15920 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:04.256249 15920 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:04.256592 15920 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.257731 15724 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.262821 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:04.263444 15600 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.264537 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:04.266559 15566 leader_election.cc:343] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:04.272513 15892 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:04.276862 15600 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:04.278432 15493 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:04.259478 15897 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:04.279706 15797 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:04.280306 15797 raft_consensus.cc:697] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:04.281208 15724 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:04.245985 15717 consensus_queue.cc:260] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.258473 15602 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:04.284745 15493 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:04.284951 15717 ts_tablet_manager.cc:1434] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.043s	user 0.005s	sys 0.000s
I20260709 13:18:04.281243 15797 consensus_queue.cc:237] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.286020 15724 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:04.287319 15724 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:04.287750 15724 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.286963 15717 tablet_bootstrap.cc:492] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:04.292121 15892 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.292825 15892 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.297377 15892 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:04.298281 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:04.299255 15525 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:04.301107 15641 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:04.302294 15890 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:04.302806 15890 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:04.303249 15890 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.303050 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:04.304028 15602 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:04.314083 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:04.314852 15677 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.319052 15890 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.333272 15724 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.362439 15724 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:04.363085 15724 consensus_queue.cc:1048] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:04.311658 15920 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:04.371031 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 61185.7us
W20260709 13:18:04.366060 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 56484.8us
I20260709 13:18:04.375098 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:04.375746 15676 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.375957 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:04.378374 15718 tablet_bootstrap.cc:492] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:04.378870 15718 ts_tablet_manager.cc:1403] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.163s	user 0.073s	sys 0.071s
I20260709 13:18:04.383232 15720 tablet_bootstrap.cc:492] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:04.383723 15720 ts_tablet_manager.cc:1403] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.234s	user 0.067s	sys 0.122s
I20260709 13:18:04.386615 15892 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:04.388777 15890 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:04.390771 15720 raft_consensus.cc:359] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.387444 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:04.391434 15720 raft_consensus.cc:385] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.391721 15720 raft_consensus.cc:740] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.391733 15602 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.392383 15720 consensus_queue.cc:260] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.393013 15717 tablet_bootstrap.cc:654] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.399547 15764 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.400136 15764 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.402354 15764 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:04.404603 15720 ts_tablet_manager.cc:1434] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
I20260709 13:18:04.405606 15720 tablet_bootstrap.cc:492] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:04.407317 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:04.405292 15722 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:18:04.407950 15528 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.414583 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:04.415187 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:04.415822 15525 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
W20260709 13:18:04.416281 15566 leader_election.cc:343] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:04.411926 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:04.420634 15601 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:04.412736 15491 leader_election.cc:343] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:04.419245 15920 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:04.428009 15879 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:04.417065 15966 consensus_queue.cc:1048] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:18:04.421593 15676 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:04.423276 15567 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:04.457669 15602 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:04.458847 15718 raft_consensus.cc:359] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.442751 15677 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:04.459654 15718 raft_consensus.cc:385] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.459903 15718 raft_consensus.cc:740] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.463121 15717 tablet_bootstrap.cc:492] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:04.463742 15717 ts_tablet_manager.cc:1403] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.177s	user 0.006s	sys 0.006s
I20260709 13:18:04.463140 15718 consensus_queue.cc:260] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.465315 15897 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:04.465719 15897 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:04.466070 15897 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.471055 15493 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:04.459489 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:04.474500 15718 ts_tablet_manager.cc:1434] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.095s	user 0.005s	sys 0.003s
I20260709 13:18:04.478685 15718 tablet_bootstrap.cc:492] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:04.474289 15879 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:18:04.470414 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 50214.1us
W20260709 13:18:04.495786 15566 leader_election.cc:343] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:04.498792 15717 raft_consensus.cc:359] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.500723 15717 raft_consensus.cc:385] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.502306 15717 raft_consensus.cc:740] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.503157 15717 consensus_queue.cc:260] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:04.499472 15962 log.cc:927] Time spent T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.082s	user 0.002s	sys 0.000s
I20260709 13:18:04.494263 15879 raft_consensus.cc:697] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:04.509708 15879 consensus_queue.cc:237] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.512041 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:04.512658 15522 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.502362 15528 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:04.521067 15717 ts_tablet_manager.cc:1434] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.057s	user 0.005s	sys 0.004s
I20260709 13:18:04.525405 15717 tablet_bootstrap.cc:492] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:04.530071 15895 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:04.530674 15641 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:04.538084 15722 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:04.539351 15722 raft_consensus.cc:697] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:04.540308 15722 consensus_queue.cc:237] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.549831 15897 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.552228 15897 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:04.553408 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:18:04.554908 15566 leader_election.cc:343] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:04.556227 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:04.556854 15528 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.561165 15601 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:04.577419 15528 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:04.579826 15803 consensus_queue.cc:1048] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:04.593230 15567 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:04.594267 15897 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:04.594841 15897 raft_consensus.cc:697] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:04.599956 15897 consensus_queue.cc:237] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.607810 15720 tablet_bootstrap.cc:654] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.606662 15890 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.630219 15424 catalog_manager.cc:5697] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:04.616917 15890 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.655596 15981 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:04.676443 15975 log.cc:927] Time spent T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:18:04.682534 15718 tablet_bootstrap.cc:654] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.682361 15981 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.712790 15717 tablet_bootstrap.cc:654] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.716154 15981 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:04.705425 15672 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.723193 15720 tablet_bootstrap.cc:492] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:04.723779 15720 ts_tablet_manager.cc:1403] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.318s	user 0.017s	sys 0.011s
I20260709 13:18:04.730700 15671 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:04.742558 15976 log.cc:927] Time spent T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.088s	user 0.001s	sys 0.001s
I20260709 13:18:04.735409 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:04.754997 15601 raft_consensus.cc:2393] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 1: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:18:04.831173 15718 tablet_bootstrap.cc:492] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:04.831660 15718 ts_tablet_manager.cc:1403] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.353s	user 0.060s	sys 0.038s
I20260709 13:18:04.833091 15920 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.833689 15920 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.834281 15718 raft_consensus.cc:359] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.834995 15718 raft_consensus.cc:385] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.835287 15718 raft_consensus.cc:740] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.835109 15720 raft_consensus.cc:359] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.835760 15720 raft_consensus.cc:385] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.836009 15718 consensus_queue.cc:260] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.836462 15890 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:04.838567 15718 ts_tablet_manager.cc:1434] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.006s
I20260709 13:18:04.839462 15718 tablet_bootstrap.cc:492] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:04.839567 15920 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:04.840185 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:04.841342 15967 consensus_queue.cc:1048] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:18:04.841599 15566 leader_election.cc:343] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:04.843564 15967 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:04.845110 15718 tablet_bootstrap.cc:654] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.836024 15720 raft_consensus.cc:740] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.850709 15720 consensus_queue.cc:260] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.854758 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:04.855618 15526 raft_consensus.cc:2393] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 1: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:04.855830 15717 tablet_bootstrap.cc:492] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:04.856202 15718 tablet_bootstrap.cc:492] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:04.856303 15717 ts_tablet_manager.cc:1403] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.334s	user 0.010s	sys 0.010s
I20260709 13:18:04.856739 15718 ts_tablet_manager.cc:1403] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.017s	user 0.007s	sys 0.010s
I20260709 13:18:04.858970 15717 raft_consensus.cc:359] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.859270 15718 raft_consensus.cc:359] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.859686 15717 raft_consensus.cc:385] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.859897 15720 ts_tablet_manager.cc:1434] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.136s	user 0.009s	sys 0.002s
I20260709 13:18:04.859988 15718 raft_consensus.cc:385] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.860110 15717 raft_consensus.cc:740] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.860364 15718 raft_consensus.cc:740] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.860899 15717 consensus_queue.cc:260] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.861125 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:04.861129 15718 consensus_queue.cc:260] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:04.863476 15718 ts_tablet_manager.cc:1434] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:18:04.841681 15522 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:04.864387 15718 tablet_bootstrap.cc:492] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:04.864670 15717 ts_tablet_manager.cc:1434] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:18:04.866823 15717 tablet_bootstrap.cc:492] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:04.870424 15718 tablet_bootstrap.cc:654] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.872455 15717 tablet_bootstrap.cc:654] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.879258 15720 tablet_bootstrap.cc:492] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:04.879853 15920 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:04.885309 15641 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:18:05.023403 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 139558us
I20260709 13:18:05.025266 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:18:05.027819 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 142609us
I20260709 13:18:05.030342 15718 tablet_bootstrap.cc:492] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:05.030834 15718 ts_tablet_manager.cc:1403] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.167s	user 0.011s	sys 0.004s
I20260709 13:18:05.036969 15890 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:05.036254 15879 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.040848 15879 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:05.042062 15567 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:05.042785 15879 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:05.043913 15920 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:05.037825 15722 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.044425 15920 raft_consensus.cc:697] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:05.045418 15803 consensus_queue.cc:1048] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:05.046154 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:05.046825 15526 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:05.047474 15803 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:05.048537 15968 consensus_queue.cc:1048] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:05.049558 15720 tablet_bootstrap.cc:654] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:05.052912 15642 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:05.045300 15920 consensus_queue.cc:237] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:05.067448 15986 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:05.054618 15722 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:05.080556 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:05.081341 15602 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:05.082154 15718 raft_consensus.cc:359] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:05.082896 15718 raft_consensus.cc:385] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:05.083158 15718 raft_consensus.cc:740] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:05.084242 15722 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:05.088346 15717 tablet_bootstrap.cc:492] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:05.089481 15717 ts_tablet_manager.cc:1403] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.223s	user 0.009s	sys 0.007s
I20260709 13:18:05.091326 15718 consensus_queue.cc:260] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:05.093144 15493 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:05.097199 15966 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:05.097653 15966 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.098007 15966 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:05.106355 15718 ts_tablet_manager.cc:1434] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.075s	user 0.005s	sys 0.003s
W20260709 13:18:05.237076 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 177412us
I20260709 13:18:05.096587 15803 consensus_queue.cc:1048] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.147s
W20260709 13:18:05.238782 15491 consensus_peers.cc:597] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 -> Peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Couldn't send request to peer 3ef186ec12554a1d87814af0a0ace9a4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:05.279193 15718 tablet_bootstrap.cc:492] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:05.094412 15997 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.168s
I20260709 13:18:05.126067 15722 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.136s
I20260709 13:18:05.304891 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:05.306473 15720 tablet_bootstrap.cc:492] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:05.306967 15720 ts_tablet_manager.cc:1403] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.428s	user 0.037s	sys 0.035s
W20260709 13:18:05.280324 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 167902us
I20260709 13:18:05.311533 15717 raft_consensus.cc:359] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:05.317788 15717 raft_consensus.cc:385] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:05.318132 15717 raft_consensus.cc:740] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:05.318778 15717 consensus_queue.cc:260] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:05.331382 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 62573.1us
I20260709 13:18:05.332446 15567 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:05.334396 15920 raft_consensus.cc:2804] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:05.334791 15920 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.335109 15920 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:05.337576 15717 ts_tablet_manager.cc:1434] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.248s	user 0.007s	sys 0.000s
W20260709 13:18:05.348109 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 84528us
I20260709 13:18:05.358469 16007 consensus_queue.cc:1048] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
W20260709 13:18:05.408912 16003 log.cc:927] Time spent T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:18:05.359740 15879 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.455500 15966 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:05.375269 15968 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.457561 15879 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:05.535652 15995 log.cc:927] Time spent T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.073s	user 0.003s	sys 0.015s
W20260709 13:18:05.486650 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 52813.9us
I20260709 13:18:05.531031 15968 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:05.547222 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 60476.7us
I20260709 13:18:05.552024 15968 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:05.553485 15879 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:05.553862 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:05.554807 15966 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:05.558156 15920 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:05.557911 15717 tablet_bootstrap.cc:492] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:05.567778 15720 raft_consensus.cc:359] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:05.568602 15720 raft_consensus.cc:385] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:05.568954 15720 raft_consensus.cc:740] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:05.580093 15720 consensus_queue.cc:260] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:05.578411 15674 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:05.649187 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:05.652225 15718 tablet_bootstrap.cc:654] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:05.657989 15717 tablet_bootstrap.cc:654] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:05.661056 15720 ts_tablet_manager.cc:1434] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.354s	user 0.028s	sys 0.041s
I20260709 13:18:05.667316 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:05.669569 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:05.670997 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:05.665333 15525 raft_consensus.cc:2393] T 4b9f476ac2ad44cd8124fde4f1aadc78 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 1: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
W20260709 13:18:05.678289 15491 leader_election.cc:343] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:18:05.679565 15491 leader_election.cc:343] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:18:05.680167 15491 leader_election.cc:343] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:18:05.680748 15491 leader_election.cc:343] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:05.691165 15671 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:05.716856 16012 log.cc:927] Time spent T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:18:05.727066 15426 catalog_manager.cc:5697] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:05.736773 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:05.756579 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:05.757242 15603 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:05.757546 15981 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.760757 15896 consensus_queue.cc:1048] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:05.758073 15981 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:05.762104 15717 tablet_bootstrap.cc:492] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:05.762559 15717 ts_tablet_manager.cc:1403] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.205s	user 0.029s	sys 0.018s
I20260709 13:18:05.764971 15896 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:05.765173 15717 raft_consensus.cc:359] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:05.765941 15717 raft_consensus.cc:385] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:05.766232 15717 raft_consensus.cc:740] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:05.766987 15717 consensus_queue.cc:260] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:05.767360 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:05.767910 15603 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:05.768527 15717 ts_tablet_manager.cc:1434] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:18:05.772508 15720 tablet_bootstrap.cc:492] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:05.776124 15603 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:05.788847 15720 tablet_bootstrap.cc:654] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:05.790338 15718 tablet_bootstrap.cc:492] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:05.790894 15718 ts_tablet_manager.cc:1403] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.684s	user 0.036s	sys 0.112s
I20260709 13:18:05.818795 15981 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:05.821156 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:05.823058 15717 tablet_bootstrap.cc:492] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:05.828433 15981 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:05.832054 15718 raft_consensus.cc:359] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:05.832796 15718 raft_consensus.cc:385] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:05.833074 15718 raft_consensus.cc:740] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:05.838590 15493 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:05.843158 15718 consensus_queue.cc:260] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:05.844133 15493 leader_election.cc:343] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:05.844671 15493 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:05.848142 15920 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:05.849812 15493 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:05.853364 15720 tablet_bootstrap.cc:492] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:05.854081 15720 ts_tablet_manager.cc:1403] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.082s	user 0.009s	sys 0.007s
I20260709 13:18:05.858721 15718 ts_tablet_manager.cc:1434] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.001s
I20260709 13:18:05.859555 16014 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:05.852824 16016 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:05.853657 16006 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:05.865338 15718 tablet_bootstrap.cc:492] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:05.866618 15641 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:05.868067 15895 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:05.872252 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:05.864172 16016 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.860072 16014 raft_consensus.cc:697] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:05.873747 16016 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:05.878182 15717 tablet_bootstrap.cc:654] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:05.878227 16014 consensus_queue.cc:237] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:05.885519 15718 tablet_bootstrap.cc:654] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:05.884399 15720 raft_consensus.cc:359] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:05.886312 15720 raft_consensus.cc:385] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:05.886593 15720 raft_consensus.cc:740] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:05.887223 15720 consensus_queue.cc:260] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:05.889537 15720 ts_tablet_manager.cc:1434] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.035s	user 0.004s	sys 0.003s
I20260709 13:18:05.890102 16016 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:05.891258 15720 tablet_bootstrap.cc:492] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:05.892249 16016 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:05.897893 15720 tablet_bootstrap.cc:654] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
W20260709 13:18:05.930550 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 75039.7us
W20260709 13:18:05.935060 15566 leader_election.cc:343] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:05.944136 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:05.959391 15717 tablet_bootstrap.cc:492] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:05.960408 15717 ts_tablet_manager.cc:1403] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.138s	user 0.009s	sys 0.006s
I20260709 13:18:05.967855 15718 tablet_bootstrap.cc:492] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:05.968770 15718 ts_tablet_manager.cc:1403] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.104s	user 0.009s	sys 0.002s
I20260709 13:18:05.973835 15771 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.974789 15771 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:05.977393 15896 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:05.979485 15491 leader_election.cc:343] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:05.980203 15426 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49144 (request call id 69) took 1279 ms. Trace:
I20260709 13:18:05.978186 15896 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:05.982826 15896 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:05.984006 16016 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:05.988400 15424 catalog_manager.cc:5697] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:05.980715 15426 rpcz_store.cc:276] 0709 13:18:04.705954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.707606 (+  1652us) service_pool.cc:224] Handling call
0709 13:18:05.975928 (+1268322us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:05.804024 (+     0us) spinlock_profiling.cc:243] Waited 1.64 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce19010ac 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191b455 00007f5ce844016c
0709 13:18:05.869063 (+ 65039us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:05.872418 (+  3355us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:05.872449 (+    31us) write_op.cc:435] Acquired schema lock
0709 13:18:05.872471 (+    22us) tablet.cc:662] Decoding operations
0709 13:18:05.872870 (+   399us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:05.872929 (+    59us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:05.872960 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:18:05.873181 (+   221us) tablet.cc:701] Row locks acquired
0709 13:18:05.873199 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:18:05.874174 (+   975us) write_op.cc:270] Start()
0709 13:18:05.879107 (+  4933us) write_op.cc:276] Timestamp: P: 1783603085874117 usec, L: 0
0709 13:18:05.879138 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:18:05.881963 (+  2825us) log.cc:844] Serialized 537 byte log entry
0709 13:18:05.895455 (+ 13492us) op_driver.cc:464] REPLICATION: finished
0709 13:18:05.897496 (+  2041us) write_op.cc:301] APPLY: starting
0709 13:18:05.897568 (+    72us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:05.897658 (+    90us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:05.897679 (+    21us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:05.897906 (+   227us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:05.898074 (+   168us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:18:05.898168 (+    94us) write_op.cc:312] APPLY: finished
0709 13:18:05.898400 (+   232us) log.cc:844] Serialized 22 byte log entry
0709 13:18:05.898622 (+   222us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:05.898716 (+    94us) write_op.cc:454] Released schema lock
0709 13:18:05.898753 (+    37us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:05.899042 (+   289us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:05.899067 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":1816,"apply.run_cpu_time_us":1404,"apply.run_wall_time_us":1536,"num_ops":1,"prepare.queue_time_us":133167,"prepare.run_cpu_time_us":3111,"prepare.run_wall_time_us":13197,"replication_time_us":15187,"spinlock_wait_cycles":3765120,"thread_start_us":118696,"threads_started":2}]]}
I20260709 13:18:05.992643 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:05.997051 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:06.008730 15526 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:06.009337 15491 leader_election.cc:343] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:06.002554 15424 catalog_manager.cc:5697] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:06.012445 15424 catalog_manager.cc:5697] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:18:06.014982 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:06.015851 15675 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:06.017740 15718 raft_consensus.cc:359] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.018692 15718 raft_consensus.cc:385] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:06.019027 15718 raft_consensus.cc:740] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:06.019832 15718 consensus_queue.cc:260] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.017596 15526 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
W20260709 13:18:06.004722 15474 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:05.990113 15771 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:06.027428 15566 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:06.017942 15424 catalog_manager.cc:5697] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:18:06.038892 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:06.042930 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:06.047462 15920 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:06.047941 15920 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.048291 15920 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.968976 15717 raft_consensus.cc:359] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.054553 15717 raft_consensus.cc:385] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:06.054997 15717 raft_consensus.cc:740] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:06.057230 15717 consensus_queue.cc:260] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.021734 15718 ts_tablet_manager.cc:1434] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.053s	user 0.005s	sys 0.001s
I20260709 13:18:06.068871 15718 tablet_bootstrap.cc:492] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:06.071473 15717 ts_tablet_manager.cc:1434] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.111s	user 0.008s	sys 0.004s
I20260709 13:18:06.078524 15717 tablet_bootstrap.cc:492] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:06.092335 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:06.099593 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:06.101012 15598 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:06.105355 15567 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:06.110316 15641 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:06.113560 15717 tablet_bootstrap.cc:654] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:06.113706 16007 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:06.114169 15997 raft_consensus.cc:2804] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:06.115361 15983 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:06.116919 15425 catalog_manager.cc:5697] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:06.120129 15425 catalog_manager.cc:5697] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:06.122203 15997 raft_consensus.cc:697] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:06.123275 15997 consensus_queue.cc:237] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.133735 15677 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:06.137545 15493 leader_election.cc:343] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:06.130553 15920 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.147671 15603 raft_consensus.cc:1240] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:18:06.148569 15718 tablet_bootstrap.cc:654] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:06.137902 15493 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:06.151461 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:06.156188 16006 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:06.163982 15720 tablet_bootstrap.cc:492] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:06.164600 15720 ts_tablet_manager.cc:1403] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.274s	user 0.024s	sys 0.023s
I20260709 13:18:06.171166 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:06.171533 15598 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:06.165340 15920 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:06.175308 15890 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.177181 15890 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.178599 15720 raft_consensus.cc:359] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.179329 15720 raft_consensus.cc:385] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:06.179601 15720 raft_consensus.cc:740] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:06.179890 15890 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:06.180359 15720 consensus_queue.cc:260] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.183063 15567 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:06.183985 16007 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:06.185998 15720 ts_tablet_manager.cc:1434] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.004s
I20260709 13:18:06.187422 15720 tablet_bootstrap.cc:492] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:06.199259 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:06.200170 15600 raft_consensus.cc:2393] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 1: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:18:06.203850 15493 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:06.203303 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:06.205782 16005 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:06.208351 16005 raft_consensus.cc:697] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:06.211930 15425 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49132 (request call id 66) took 1339 ms. Trace:
I20260709 13:18:06.209357 16005 consensus_queue.cc:237] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.224227 15879 consensus_queue.cc:1059] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:18:06.227051 15718 tablet_bootstrap.cc:492] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:06.227454 15718 ts_tablet_manager.cc:1403] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.159s	user 0.018s	sys 0.005s
I20260709 13:18:06.229537 16006 raft_consensus.cc:3055] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:06.212296 15425 rpcz_store.cc:276] 0709 13:18:04.872064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.872223 (+   159us) service_pool.cc:224] Handling call
0709 13:18:06.211870 (+1339647us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:06.140614 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:06.141200 (+   586us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:06.141235 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:18:06.141252 (+    17us) tablet.cc:662] Decoding operations
0709 13:18:06.141671 (+   419us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:06.141722 (+    51us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:06.141763 (+    41us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:18:06.146775 (+  5012us) tablet.cc:701] Row locks acquired
0709 13:18:06.146790 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:18:06.146926 (+   136us) write_op.cc:270] Start()
0709 13:18:06.147040 (+   114us) write_op.cc:276] Timestamp: P: 1783603086146885 usec, L: 0
0709 13:18:06.147055 (+    15us) op_driver.cc:348] REPLICATION: starting
0709 13:18:06.147721 (+   666us) log.cc:844] Serialized 922 byte log entry
0709 13:18:06.152270 (+  4549us) op_driver.cc:464] REPLICATION: finished
0709 13:18:06.152691 (+   421us) write_op.cc:301] APPLY: starting
0709 13:18:06.152769 (+    78us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:06.152881 (+   112us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:06.152972 (+    91us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:06.153267 (+   295us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:06.153405 (+   138us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:18:06.153449 (+    44us) write_op.cc:312] APPLY: finished
0709 13:18:06.153628 (+   179us) log.cc:844] Serialized 28 byte log entry
0709 13:18:06.153808 (+   180us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:06.153895 (+    87us) write_op.cc:454] Released schema lock
0709 13:18:06.153966 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:06.154292 (+   326us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:06.154318 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":13,"child_traces":[["op",{"apply.queue_time_us":223,"apply.run_cpu_time_us":1559,"apply.run_wall_time_us":1558,"num_ops":2,"prepare.queue_time_us":4367,"prepare.run_cpu_time_us":2799,"prepare.run_wall_time_us":7361,"replication_time_us":5095}]]}
I20260709 13:18:06.234684 16006 raft_consensus.cc:740] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:06.248427 15717 tablet_bootstrap.cc:492] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:06.248059 16006 consensus_queue.cc:260] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.251852 16006 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:06.252272 15717 ts_tablet_manager.cc:1403] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.174s	user 0.015s	sys 0.006s
I20260709 13:18:06.262468 15718 raft_consensus.cc:359] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.269425 15718 raft_consensus.cc:385] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:06.270080 15718 raft_consensus.cc:740] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:06.267884 15717 raft_consensus.cc:359] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.272436 15717 raft_consensus.cc:385] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:06.272933 15717 raft_consensus.cc:740] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:06.276007 15997 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.276629 15997 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.278587 15997 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:06.268926 15720 tablet_bootstrap.cc:654] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:06.279296 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:06.279677 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:06.281265 15566 leader_election.cc:343] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:06.271235 15718 consensus_queue.cc:260] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.283356 15528 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:06.284785 15718 ts_tablet_manager.cc:1434] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.057s	user 0.005s	sys 0.001s
I20260709 13:18:06.273703 15717 consensus_queue.cc:260] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.286959 15567 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:06.288303 15717 ts_tablet_manager.cc:1434] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 13:18:06.289386 15717 tablet_bootstrap.cc:492] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:06.289809 15997 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:06.290668 15997 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.291090 15997 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:06.291544 15718 tablet_bootstrap.cc:492] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:06.294637 15641 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:06.296463 15983 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:06.311882 15879 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.312424 15879 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.323096 16005 consensus_queue.cc:1048] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:06.330940 15720 tablet_bootstrap.cc:492] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:06.339308 15720 ts_tablet_manager.cc:1403] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.152s	user 0.014s	sys 0.006s
I20260709 13:18:06.332489 15997 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.332403 15879 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:06.329643 15983 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:06.338291 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:06.341326 15491 leader_election.cc:343] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:06.342340 15997 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:06.338593 15718 tablet_bootstrap.cc:654] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:06.342949 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:18:06.344354 15566 leader_election.cc:343] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:06.334704 15717 tablet_bootstrap.cc:654] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:06.345731 15720 raft_consensus.cc:359] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.346468 15720 raft_consensus.cc:385] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:06.347944 15720 raft_consensus.cc:740] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:06.335405 15425 catalog_manager.cc:5697] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:06.351378 15720 consensus_queue.cc:260] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.353453 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:06.353690 15720 ts_tablet_manager.cc:1434] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:18:06.354861 15720 tablet_bootstrap.cc:492] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
W20260709 13:18:06.357198 15493 leader_election.cc:343] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:06.358338 15493 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:06.360179 15718 tablet_bootstrap.cc:492] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:06.362223 15718 ts_tablet_manager.cc:1403] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.071s	user 0.014s	sys 0.004s
I20260709 13:18:06.362176 15879 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:06.364602 15717 tablet_bootstrap.cc:492] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:06.365129 15717 ts_tablet_manager.cc:1403] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.076s	user 0.014s	sys 0.003s
I20260709 13:18:06.365468 15718 raft_consensus.cc:359] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.366369 15718 raft_consensus.cc:385] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:06.366698 15718 raft_consensus.cc:740] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:06.367479 15718 consensus_queue.cc:260] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.367930 15717 raft_consensus.cc:359] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.368633 15717 raft_consensus.cc:385] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:06.368899 15717 raft_consensus.cc:740] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:06.372257 15717 consensus_queue.cc:260] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.374361 15717 ts_tablet_manager.cc:1434] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:18:06.371379 15720 tablet_bootstrap.cc:654] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:06.377995 15717 tablet_bootstrap.cc:492] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:06.377275 15425 catalog_manager.cc:5697] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:18:06.381501 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:06.382423 15524 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:06.383780 15717 tablet_bootstrap.cc:654] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:06.389353 15524 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:06.394363 15567 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:06.395334 15997 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:06.395848 15997 raft_consensus.cc:697] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:06.396651 15720 tablet_bootstrap.cc:492] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:06.397723 15720 ts_tablet_manager.cc:1403] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.043s	user 0.010s	sys 0.005s
I20260709 13:18:06.396653 15997 consensus_queue.cc:237] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.401101 15675 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:06.401625 15920 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.401810 15720 raft_consensus.cc:359] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.402511 15720 raft_consensus.cc:385] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:06.402240 15920 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.402788 15720 raft_consensus.cc:740] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:06.404587 15720 consensus_queue.cc:260] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.407511 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:06.408322 15720 ts_tablet_manager.cc:1434] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
W20260709 13:18:06.413849 15566 leader_election.cc:343] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:06.414681 15720 tablet_bootstrap.cc:492] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:06.415077 15920 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:06.377135 15718 ts_tablet_manager.cc:1434] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:18:06.425143 15718 tablet_bootstrap.cc:492] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:06.425334 15720 tablet_bootstrap.cc:654] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:06.427331 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:06.430992 15526 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:06.428330 16005 consensus_queue.cc:1048] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:06.446398 15567 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:06.450837 15920 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:06.449177 15426 catalog_manager.cc:5697] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:06.453128 15920 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.457346 15718 tablet_bootstrap.cc:654] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:06.462977 15717 tablet_bootstrap.cc:492] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:06.463464 15717 ts_tablet_manager.cc:1403] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.086s	user 0.014s	sys 0.017s
I20260709 13:18:06.454682 15920 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:06.479080 15720 tablet_bootstrap.cc:492] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:06.479557 15720 ts_tablet_manager.cc:1403] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.065s	user 0.016s	sys 0.004s
I20260709 13:18:06.486014 15717 raft_consensus.cc:359] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.486768 15717 raft_consensus.cc:385] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:06.487118 15717 raft_consensus.cc:740] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:06.495919 15920 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.498169 15920 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:06.499760 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:06.500288 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:06.500418 15524 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:06.502501 15566 leader_election.cc:343] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:06.507915 15524 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:06.510134 15567 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:06.510603 15717 consensus_queue.cc:260] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.511431 15997 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:06.511889 15997 raft_consensus.cc:697] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:06.512179 15720 raft_consensus.cc:359] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.512854 15720 raft_consensus.cc:385] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:06.512704 15997 consensus_queue.cc:237] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.513193 15720 raft_consensus.cc:740] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:06.512880 15717 ts_tablet_manager.cc:1434] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.000s
I20260709 13:18:06.508473 15425 catalog_manager.cc:5697] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:06.513978 15720 consensus_queue.cc:260] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.514671 15717 tablet_bootstrap.cc:492] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:06.516116 15720 ts_tablet_manager.cc:1434] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.001s
I20260709 13:18:06.517117 15720 tablet_bootstrap.cc:492] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:06.518859 15718 tablet_bootstrap.cc:492] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:06.519384 15718 ts_tablet_manager.cc:1403] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.095s	user 0.008s	sys 0.010s
I20260709 13:18:06.522804 15718 raft_consensus.cc:359] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.532661 15718 raft_consensus.cc:385] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:06.527230 15717 tablet_bootstrap.cc:654] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:06.535032 15720 tablet_bootstrap.cc:654] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:06.534314 15718 raft_consensus.cc:740] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:06.536499 15718 consensus_queue.cc:260] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.551163 15718 ts_tablet_manager.cc:1434] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.001s
I20260709 13:18:06.554107 15718 tablet_bootstrap.cc:492] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:06.588336 15717 tablet_bootstrap.cc:492] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:06.588917 15717 ts_tablet_manager.cc:1403] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.074s	user 0.028s	sys 0.025s
I20260709 13:18:06.589457 15720 tablet_bootstrap.cc:492] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:06.589959 15720 ts_tablet_manager.cc:1403] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.073s	user 0.008s	sys 0.012s
I20260709 13:18:06.592935 15720 raft_consensus.cc:359] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.593437 15720 raft_consensus.cc:385] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:06.593643 15720 raft_consensus.cc:740] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:06.594393 15717 raft_consensus.cc:359] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.594841 15718 tablet_bootstrap.cc:654] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:06.594550 15720 consensus_queue.cc:260] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.595108 15717 raft_consensus.cc:385] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:06.595379 15717 raft_consensus.cc:740] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:06.596045 15720 ts_tablet_manager.cc:1434] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:18:06.597623 15717 consensus_queue.cc:260] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:06.599684 15566 consensus_peers.cc:597] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Couldn't send request to peer 3ef186ec12554a1d87814af0a0ace9a4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:06.599788 15717 ts_tablet_manager.cc:1434] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:18:06.600968 15720 tablet_bootstrap.cc:492] T e67cef93bbdb4ff88a2ee2ac558ba090 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:06.602128 15717 tablet_bootstrap.cc:492] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:06.603294 15425 catalog_manager.cc:5697] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:06.611310 15717 tablet_bootstrap.cc:654] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:06.618294 15997 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.618846 15997 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.620891 15997 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:06.625535 15720 tablet_bootstrap.cc:654] T e67cef93bbdb4ff88a2ee2ac558ba090 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:06.626446 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:06.630088 15528 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:06.633838 15567 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:06.634608 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:06.635282 15669 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:06.636224 15764 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:06.636590 15764 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.636915 15764 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:06.631436 15997 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.646549 15997 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.663484 15717 tablet_bootstrap.cc:492] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:06.668118 15997 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:06.671372 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:06.671382 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:06.673269 15528 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:06.674170 15717 ts_tablet_manager.cc:1403] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.072s	user 0.026s	sys 0.019s
I20260709 13:18:06.675767 15879 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.676259 15879 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.677177 15717 raft_consensus.cc:359] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.677933 15717 raft_consensus.cc:385] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:06.678215 15717 raft_consensus.cc:740] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
W20260709 13:18:06.677109 15566 leader_election.cc:343] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:06.678834 15879 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:06.679064 15717 consensus_queue.cc:260] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.682495 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:06.683337 15717 ts_tablet_manager.cc:1434] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:18:06.683625 15720 tablet_bootstrap.cc:492] T e67cef93bbdb4ff88a2ee2ac558ba090 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:06.684098 15720 ts_tablet_manager.cc:1403] T e67cef93bbdb4ff88a2ee2ac558ba090 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.083s	user 0.006s	sys 0.005s
I20260709 13:18:06.684584 15717 tablet_bootstrap.cc:492] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
W20260709 13:18:06.687098 15491 leader_election.cc:343] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:06.691972 15717 tablet_bootstrap.cc:654] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:06.694164 15567 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:06.696084 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:06.696435 15720 raft_consensus.cc:359] T e67cef93bbdb4ff88a2ee2ac558ba090 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.696880 15920 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:06.697096 15720 raft_consensus.cc:385] T e67cef93bbdb4ff88a2ee2ac558ba090 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:06.697400 15720 raft_consensus.cc:740] T e67cef93bbdb4ff88a2ee2ac558ba090 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:06.697439 15920 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.697973 15920 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:06.701728 15720 consensus_queue.cc:260] T e67cef93bbdb4ff88a2ee2ac558ba090 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.702672 15603 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:06.703524 15717 tablet_bootstrap.cc:492] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:06.704054 15717 ts_tablet_manager.cc:1403] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.008s
I20260709 13:18:06.707024 15717 raft_consensus.cc:359] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.707752 15717 raft_consensus.cc:385] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:06.708019 15717 raft_consensus.cc:740] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:06.708828 15717 consensus_queue.cc:260] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.709111 15493 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:06.711158 15717 ts_tablet_manager.cc:1434] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:18:06.712414 15717 tablet_bootstrap.cc:492] T 7b8ed23a60014da39806e750cee98fa6 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:06.711163 15803 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:06.713477 15803 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.713901 15803 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:06.715972 15720 ts_tablet_manager.cc:1434] T e67cef93bbdb4ff88a2ee2ac558ba090 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 13:18:06.717134 15720 tablet_bootstrap.cc:492] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:06.719542 15717 tablet_bootstrap.cc:654] T 7b8ed23a60014da39806e750cee98fa6 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:06.720680 15764 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.727727 15720 tablet_bootstrap.cc:654] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:06.731302 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:06.732059 15524 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:06.750908 15803 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.754002 15879 consensus_queue.cc:1048] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:06.754979 15718 tablet_bootstrap.cc:492] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:06.755437 15718 ts_tablet_manager.cc:1403] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.202s	user 0.060s	sys 0.068s
I20260709 13:18:06.759878 15524 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:06.760489 15997 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.760978 15997 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.761981 15803 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:06.763604 15920 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.764415 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:06.765010 15600 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:06.766531 15717 tablet_bootstrap.cc:492] T 7b8ed23a60014da39806e750cee98fa6 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:06.766987 15717 ts_tablet_manager.cc:1403] T 7b8ed23a60014da39806e750cee98fa6 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.055s	user 0.010s	sys 0.004s
I20260709 13:18:06.769070 15567 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:06.770116 16007 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:06.770570 15764 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:06.773053 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:06.773746 15528 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:06.774802 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:06.775122 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:06.775404 15675 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:06.775657 15670 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:06.779636 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:06.758718 15718 raft_consensus.cc:359] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.783706 15718 raft_consensus.cc:385] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:06.783947 15718 raft_consensus.cc:740] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
W20260709 13:18:06.783170 15566 leader_election.cc:343] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:06.787097 15567 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:06.788056 16008 consensus_queue.cc:1048] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:06.789434 16008 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:06.789757 16008 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.790091 16008 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:06.793789 15675 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:06.796355 15997 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:06.796566 15670 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:06.800521 15600 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:06.802256 15718 consensus_queue.cc:260] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.806478 15920 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:06.808743 15720 tablet_bootstrap.cc:492] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:06.808856 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:06.809302 15720 ts_tablet_manager.cc:1403] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.092s	user 0.034s	sys 0.037s
I20260709 13:18:06.813004 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:06.813622 15528 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:06.820179 15997 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.820791 15997 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.823944 15491 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:06.811214 16007 raft_consensus.cc:697] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:06.826927 15803 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:06.832465 15718 ts_tablet_manager.cc:1434] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.077s	user 0.008s	sys 0.000s
I20260709 13:18:06.818346 16008 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.831197 15803 raft_consensus.cc:697] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:06.831844 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:06.838790 15803 consensus_queue.cc:237] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.839910 15526 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:06.842170 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:06.842851 15567 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:18:06.843411 15566 leader_election.cc:343] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:06.843678 15764 raft_consensus.cc:2804] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:06.844044 15764 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.844357 15764 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:06.845237 15528 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:06.835561 15997 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:06.846560 15567 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:06.846567 16007 consensus_queue.cc:237] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.850983 15764 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.851971 16073 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:06.852494 16073 raft_consensus.cc:697] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:06.852622 15717 raft_consensus.cc:359] T 7b8ed23a60014da39806e750cee98fa6 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.854635 15717 raft_consensus.cc:385] T 7b8ed23a60014da39806e750cee98fa6 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:06.854797 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:06.854708 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:06.855557 15526 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:06.855700 15528 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:06.857896 15718 tablet_bootstrap.cc:492] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:06.864845 15879 consensus_queue.cc:1048] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:06.865543 15717 raft_consensus.cc:740] T 7b8ed23a60014da39806e750cee98fa6 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:06.866392 15717 consensus_queue.cc:260] T 7b8ed23a60014da39806e750cee98fa6 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.867213 15675 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:06.868454 15717 ts_tablet_manager.cc:1434] T 7b8ed23a60014da39806e750cee98fa6 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.101s	user 0.006s	sys 0.000s
I20260709 13:18:06.869488 15717 tablet_bootstrap.cc:492] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:06.873263 15526 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:06.853293 16073 consensus_queue.cc:237] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.865412 15720 raft_consensus.cc:359] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.874918 15720 raft_consensus.cc:385] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:06.875582 15720 raft_consensus.cc:740] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:06.876544 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:18:06.878538 15566 leader_election.cc:343] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:06.879223 15720 consensus_queue.cc:260] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.880810 15720 ts_tablet_manager.cc:1434] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.071s	user 0.002s	sys 0.005s
I20260709 13:18:06.853977 16008 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:06.877101 15764 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:06.883507 15718 tablet_bootstrap.cc:654] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:06.884377 15528 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:06.886480 15567 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:06.887132 15720 tablet_bootstrap.cc:492] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:06.888578 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:06.889698 15764 raft_consensus.cc:2804] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:18:06.890046 15566 leader_election.cc:343] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:06.890515 15764 raft_consensus.cc:697] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:06.891439 15764 consensus_queue.cc:237] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.891957 15675 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:06.893431 15717 tablet_bootstrap.cc:654] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:06.896185 15720 tablet_bootstrap.cc:654] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:06.900264 15718 tablet_bootstrap.cc:492] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:06.900774 15718 ts_tablet_manager.cc:1403] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.043s	user 0.010s	sys 0.000s
I20260709 13:18:06.908756 15567 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:06.915692 15892 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.916272 15892 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.919598 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:06.920418 15526 raft_consensus.cc:2393] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 1: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:06.919394 15426 catalog_manager.cc:5697] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:06.987504 15890 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.987697 15718 raft_consensus.cc:359] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.988059 15890 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.990139 15890 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:06.991252 15720 tablet_bootstrap.cc:492] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:06.991744 15720 ts_tablet_manager.cc:1403] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.105s	user 0.007s	sys 0.007s
I20260709 13:18:06.991276 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:06.992682 15526 raft_consensus.cc:2393] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 1: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:06.988421 15718 raft_consensus.cc:385] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:06.995090 15722 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.995326 15718 raft_consensus.cc:740] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:06.995651 15722 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.996364 15718 consensus_queue.cc:260] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:06.997149 15892 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:06.997898 15722 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:07.002264 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:06.999950 15718 ts_tablet_manager.cc:1434] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.099s	user 0.008s	sys 0.000s
I20260709 13:18:07.009310 15425 catalog_manager.cc:5697] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:07.012888 15718 tablet_bootstrap.cc:492] T f9328896aa5a4fdc8fa7c908efdae695 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:07.014096 15717 tablet_bootstrap.cc:492] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:07.014568 15717 ts_tablet_manager.cc:1403] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.145s	user 0.016s	sys 0.018s
I20260709 13:18:07.017256 15717 raft_consensus.cc:359] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.018035 15717 raft_consensus.cc:385] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:07.018309 15717 raft_consensus.cc:740] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:07.018663 15718 tablet_bootstrap.cc:654] T f9328896aa5a4fdc8fa7c908efdae695 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:07.018963 15717 consensus_queue.cc:260] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.023180 15717 ts_tablet_manager.cc:1434] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 13:18:07.025038 15717 tablet_bootstrap.cc:492] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:07.031737 15717 tablet_bootstrap.cc:654] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:07.032850 15718 tablet_bootstrap.cc:492] T f9328896aa5a4fdc8fa7c908efdae695 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:07.033299 15718 ts_tablet_manager.cc:1403] T f9328896aa5a4fdc8fa7c908efdae695 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:18:07.035549 15718 raft_consensus.cc:359] T f9328896aa5a4fdc8fa7c908efdae695 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.036041 15718 raft_consensus.cc:385] T f9328896aa5a4fdc8fa7c908efdae695 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:07.036252 15718 raft_consensus.cc:740] T f9328896aa5a4fdc8fa7c908efdae695 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:07.036782 15717 tablet_bootstrap.cc:492] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:07.036748 15718 consensus_queue.cc:260] T f9328896aa5a4fdc8fa7c908efdae695 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.037256 15717 ts_tablet_manager.cc:1403] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.012s	user 0.011s	sys 0.000s
I20260709 13:18:07.038594 15718 ts_tablet_manager.cc:1434] T f9328896aa5a4fdc8fa7c908efdae695 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:18:07.039537 15718 tablet_bootstrap.cc:492] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:07.042141 15717 raft_consensus.cc:359] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.042811 15717 raft_consensus.cc:385] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:18:07.042980 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 134301us
I20260709 13:18:07.043074 15717 raft_consensus.cc:740] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:07.057531 15717 consensus_queue.cc:260] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.063009 15720 raft_consensus.cc:359] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.063601 15720 raft_consensus.cc:385] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:07.063835 15720 raft_consensus.cc:740] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:07.054184 16008 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.066629 15720 consensus_queue.cc:260] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.053207 16007 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:07.069552 15717 ts_tablet_manager.cc:1434] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.032s	user 0.018s	sys 0.000s
I20260709 13:18:07.076536 15717 tablet_bootstrap.cc:492] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:07.077736 15675 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:07.074692 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 72163.7us
I20260709 13:18:07.091359 15720 ts_tablet_manager.cc:1434] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.099s	user 0.002s	sys 0.005s
I20260709 13:18:07.103574 15669 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:07.072388 16008 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.111441 15718 tablet_bootstrap.cc:654] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:07.120169 16008 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:07.122843 15717 tablet_bootstrap.cc:654] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
W20260709 13:18:07.131157 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 47115.7us
W20260709 13:18:07.141161 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 51633us
I20260709 13:18:07.144160 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:07.144245 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:07.144837 15528 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:07.144960 15677 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:07.146358 15566 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:07.164814 15720 tablet_bootstrap.cc:492] T 7562b5f342134e39a5e6934d5ca16e46 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:07.139885 16007 raft_consensus.cc:697] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:07.162462 16073 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:07.167214 15670 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:07.161664 15920 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 13:18:07.180464 16073 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.186682 16088 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:18:07.187686 16007 consensus_queue.cc:237] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.186306 16073 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:07.212728 15718 tablet_bootstrap.cc:492] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:07.214591 15718 ts_tablet_manager.cc:1403] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.175s	user 0.014s	sys 0.024s
W20260709 13:18:07.214815 15474 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:07.220151 15720 tablet_bootstrap.cc:654] T 7562b5f342134e39a5e6934d5ca16e46 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:07.220504 15718 raft_consensus.cc:359] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.223137 15718 raft_consensus.cc:385] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:07.223449 15718 raft_consensus.cc:740] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:07.224267 15718 consensus_queue.cc:260] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.241560 15718 ts_tablet_manager.cc:1434] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.002s
I20260709 13:18:07.251511 15426 catalog_manager.cc:5697] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:07.234190 15920 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:18:07.259001 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:07.261485 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:07.263866 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:07.284665 15426 catalog_manager.cc:5697] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:07.286437 15718 tablet_bootstrap.cc:492] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:07.290678 16006 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.301470 15717 tablet_bootstrap.cc:492] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:07.302044 15717 ts_tablet_manager.cc:1403] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.226s	user 0.014s	sys 0.006s
I20260709 13:18:07.298261 16006 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.310557 16006 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:07.330089 15718 tablet_bootstrap.cc:654] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:07.343425 15720 tablet_bootstrap.cc:492] T 7562b5f342134e39a5e6934d5ca16e46 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:07.343941 15720 ts_tablet_manager.cc:1403] T 7562b5f342134e39a5e6934d5ca16e46 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.179s	user 0.012s	sys 0.004s
I20260709 13:18:07.340320 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:07.347311 15720 raft_consensus.cc:359] T 7562b5f342134e39a5e6934d5ca16e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.347906 15720 raft_consensus.cc:385] T 7562b5f342134e39a5e6934d5ca16e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:07.348114 15720 raft_consensus.cc:740] T 7562b5f342134e39a5e6934d5ca16e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:07.348578 15720 consensus_queue.cc:260] T 7562b5f342134e39a5e6934d5ca16e46 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.350497 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:18:07.350991 15491 leader_election.cc:343] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:07.351150 15596 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:07.354328 16073 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.357142 16073 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:07.359496 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:07.360097 15675 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:07.357261 15720 ts_tablet_manager.cc:1434] T 7562b5f342134e39a5e6934d5ca16e46 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:18:07.351146 15717 raft_consensus.cc:359] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.365172 15717 raft_consensus.cc:385] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:07.365471 15717 raft_consensus.cc:740] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:07.364328 16099 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.366730 16099 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.368419 15720 tablet_bootstrap.cc:492] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:07.375352 16096 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:07.375638 15642 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:07.376914 16071 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:07.377794 16082 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:07.381738 15642 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:07.382915 15642 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:07.383733 15890 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:07.387543 15493 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:07.395437 15717 consensus_queue.cc:260] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.396639 15675 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:07.398551 16005 raft_consensus.cc:2804] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:07.398972 16005 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.399335 16005 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:07.393946 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:18:07.401065 15566 consensus_peers.cc:597] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Couldn't send request to peer 3ef186ec12554a1d87814af0a0ace9a4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:07.401813 15717 ts_tablet_manager.cc:1434] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.099s	user 0.007s	sys 0.000s
I20260709 13:18:07.399688 16082 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:07.403187 15566 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:07.404177 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:07.405042 15528 raft_consensus.cc:2393] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 1: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:07.406157 15520 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:07.390408 15997 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 13:18:07.406394 16093 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.409390 15920 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:07.442489 15920 raft_consensus.cc:697] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:07.443343 15920 consensus_queue.cc:237] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:07.445153 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 58902.8us
I20260709 13:18:07.428825 16073 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 13:18:07.449103 16087 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:07.430122 16093 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.432243 15997 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 13:18:07.454265 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:07.458268 15920 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:07.408207 15717 tablet_bootstrap.cc:492] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:07.463938 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:07.464684 15601 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:07.464588 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:07.425033 15424 catalog_manager.cc:5697] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:07.466786 16102 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:07.462548 16093 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:07.428736 16099 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:07.433267 15720 tablet_bootstrap.cc:654] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:07.489650 15718 tablet_bootstrap.cc:492] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:07.490129 15718 ts_tablet_manager.cc:1403] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.239s	user 0.015s	sys 0.009s
I20260709 13:18:07.492914 15718 raft_consensus.cc:359] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.493811 15718 raft_consensus.cc:385] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:07.494112 15718 raft_consensus.cc:740] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:07.494804 15718 consensus_queue.cc:260] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.463986 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:07.496795 16105 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:07.497220 15718 ts_tablet_manager.cc:1434] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:18:07.502451 15718 tablet_bootstrap.cc:492] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:07.503782 15520 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:07.505115 16005 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.503852 16105 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:07.512914 15424 catalog_manager.cc:5697] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:07.514019 16005 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:18:07.517818 15491 leader_election.cc:343] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:07.521661 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:07.522320 15676 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:07.531054 16087 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.531596 16087 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.533798 16087 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:07.538825 15720 tablet_bootstrap.cc:492] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:07.540864 15720 ts_tablet_manager.cc:1403] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.173s	user 0.004s	sys 0.008s
I20260709 13:18:07.543450 15720 raft_consensus.cc:359] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.546960 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:07.571889 15491 leader_election.cc:343] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:07.570967 15720 raft_consensus.cc:385] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:07.559077 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:07.572567 15600 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:07.562799 15493 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:07.578771 15600 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:07.579136 16087 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:07.569008 15676 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:07.579552 16087 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.570657 15718 tablet_bootstrap.cc:654] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:07.553524 15717 tablet_bootstrap.cc:654] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:07.579932 16087 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:07.580160 15491 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:07.580931 16102 raft_consensus.cc:2804] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:07.581389 16102 raft_consensus.cc:697] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:07.582198 16102 consensus_queue.cc:237] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.589469 16087 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.591679 16087 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:07.592681 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:18:07.593878 15491 leader_election.cc:343] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:07.597215 15718 tablet_bootstrap.cc:492] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:07.597697 15718 ts_tablet_manager.cc:1403] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.100s	user 0.009s	sys 0.004s
I20260709 13:18:07.598023 15720 raft_consensus.cc:740] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:07.600469 15720 consensus_queue.cc:260] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.604095 15717 tablet_bootstrap.cc:492] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:07.604580 15717 ts_tablet_manager.cc:1403] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.198s	user 0.015s	sys 0.001s
I20260709 13:18:07.607141 15717 raft_consensus.cc:359] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.607790 15717 raft_consensus.cc:385] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:07.608047 15717 raft_consensus.cc:740] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:07.608681 15717 consensus_queue.cc:260] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.610100 15997 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:07.610661 15717 ts_tablet_manager.cc:1434] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:18:07.611568 15717 tablet_bootstrap.cc:492] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:07.615363 15718 raft_consensus.cc:359] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.615873 15718 raft_consensus.cc:385] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:07.616137 15718 raft_consensus.cc:740] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:07.616792 15718 consensus_queue.cc:260] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.619004 15718 ts_tablet_manager.cc:1434] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 13:18:07.619791 15718 tablet_bootstrap.cc:492] T ab68091b31d248a6ac99ca234a252f77 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:07.622244 15720 ts_tablet_manager.cc:1434] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.081s	user 0.006s	sys 0.001s
I20260709 13:18:07.625276 15718 tablet_bootstrap.cc:654] T ab68091b31d248a6ac99ca234a252f77 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:07.628715 15717 tablet_bootstrap.cc:654] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:07.634074 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:07.639295 15717 tablet_bootstrap.cc:492] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:07.640684 15717 ts_tablet_manager.cc:1403] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.029s	user 0.007s	sys 0.004s
I20260709 13:18:07.641826 15720 tablet_bootstrap.cc:492] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:07.643968 15717 raft_consensus.cc:359] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.644655 15717 raft_consensus.cc:385] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:07.644945 15717 raft_consensus.cc:740] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:07.647502 15717 consensus_queue.cc:260] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.649550 15717 ts_tablet_manager.cc:1434] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.000s
I20260709 13:18:07.650494 15717 tablet_bootstrap.cc:492] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:07.656150 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:07.656843 15601 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:07.663873 15601 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:07.673966 15718 tablet_bootstrap.cc:492] T ab68091b31d248a6ac99ca234a252f77 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:07.674546 15718 ts_tablet_manager.cc:1403] T ab68091b31d248a6ac99ca234a252f77 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.055s	user 0.015s	sys 0.005s
I20260709 13:18:07.675632 16073 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:07.667510 15717 tablet_bootstrap.cc:654] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:07.668051 15920 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.677103 15920 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.679250 15920 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:07.681574 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:07.682917 15566 leader_election.cc:343] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:07.684429 15718 raft_consensus.cc:359] T ab68091b31d248a6ac99ca234a252f77 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.685179 15718 raft_consensus.cc:385] T ab68091b31d248a6ac99ca234a252f77 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:07.685459 15718 raft_consensus.cc:740] T ab68091b31d248a6ac99ca234a252f77 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:07.686096 15718 consensus_queue.cc:260] T ab68091b31d248a6ac99ca234a252f77 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.688380 15718 ts_tablet_manager.cc:1434] T ab68091b31d248a6ac99ca234a252f77 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:18:07.690374 15718 tablet_bootstrap.cc:492] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:07.690310 15720 tablet_bootstrap.cc:654] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:07.691676 16083 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:07.692211 16083 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.696362 16083 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:07.696627 15717 tablet_bootstrap.cc:492] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:07.697098 15717 ts_tablet_manager.cc:1403] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.047s	user 0.007s	sys 0.002s
I20260709 13:18:07.697717 15425 catalog_manager.cc:5697] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:07.699652 15717 raft_consensus.cc:359] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.700314 15717 raft_consensus.cc:385] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:07.700562 15717 raft_consensus.cc:740] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:07.701821 15717 consensus_queue.cc:260] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:07.704418 15493 leader_election.cc:343] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:07.704851 15493 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:07.705508 16093 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:07.708892 15493 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:07.709564 16093 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:07.710052 16093 raft_consensus.cc:697] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:07.709992 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:07.710908 15601 raft_consensus.cc:2393] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 2: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:07.713523 16093 consensus_queue.cc:237] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.716944 15718 tablet_bootstrap.cc:654] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:07.717350 15717 ts_tablet_manager.cc:1434] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:18:07.721125 15717 tablet_bootstrap.cc:492] T 7dbabc834ad54c16ae25f79064932f42 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:07.722550 15720 tablet_bootstrap.cc:492] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:07.723006 15720 ts_tablet_manager.cc:1403] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.081s	user 0.010s	sys 0.000s
I20260709 13:18:07.727545 15717 tablet_bootstrap.cc:654] T 7dbabc834ad54c16ae25f79064932f42 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:07.730083 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:07.725497 15720 raft_consensus.cc:359] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.735842 15720 raft_consensus.cc:385] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:07.736182 15720 raft_consensus.cc:740] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:07.736680 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:07.736809 15720 consensus_queue.cc:260] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.737399 15519 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:07.737978 15718 tablet_bootstrap.cc:492] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:07.738420 15718 ts_tablet_manager.cc:1403] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.048s	user 0.013s	sys 0.000s
I20260709 13:18:07.738788 15720 ts_tablet_manager.cc:1434] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:18:07.740428 15720 tablet_bootstrap.cc:492] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:07.740947 15718 raft_consensus.cc:359] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.741606 16083 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.741626 15718 raft_consensus.cc:385] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:07.742143 15718 raft_consensus.cc:740] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:07.742235 16083 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.742803 15718 consensus_queue.cc:260] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.745064 15718 ts_tablet_manager.cc:1434] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:18:07.746171 15718 tablet_bootstrap.cc:492] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:07.746870 15720 tablet_bootstrap.cc:654] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:07.747188 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:07.747908 15600 raft_consensus.cc:2393] T 7562b5f342134e39a5e6934d5ca16e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 1: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:18:07.752683 16083 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:07.762212 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:18:07.763605 15567 leader_election.cc:343] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:07.764075 15567 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:07.765132 16088 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:07.767733 15641 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:07.771107 15722 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.771627 15722 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.772154 15641 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:07.773826 16105 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:07.774535 16105 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:07.773838 15890 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:07.774921 15722 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:07.774947 16105 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:07.775749 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:07.774784 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:07.778414 15641 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:07.779223 15722 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:07.794065 15718 tablet_bootstrap.cc:654] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:07.800817 16105 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.795679 15890 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:07.805650 16105 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:07.797842 15717 tablet_bootstrap.cc:492] T 7dbabc834ad54c16ae25f79064932f42 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:07.806407 15717 ts_tablet_manager.cc:1403] T 7dbabc834ad54c16ae25f79064932f42 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.085s	user 0.007s	sys 0.012s
I20260709 13:18:07.806710 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:07.807471 15600 raft_consensus.cc:2393] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 2: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:07.810170 15720 tablet_bootstrap.cc:492] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:07.810653 15720 ts_tablet_manager.cc:1403] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.070s	user 0.025s	sys 0.029s
I20260709 13:18:07.813763 15720 raft_consensus.cc:359] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.814477 15720 raft_consensus.cc:385] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:07.814731 15720 raft_consensus.cc:740] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:07.814970 15717 raft_consensus.cc:359] T 7dbabc834ad54c16ae25f79064932f42 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.815621 15717 raft_consensus.cc:385] T 7dbabc834ad54c16ae25f79064932f42 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:07.815451 15720 consensus_queue.cc:260] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.815966 15718 tablet_bootstrap.cc:492] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:07.816465 15718 ts_tablet_manager.cc:1403] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.071s	user 0.007s	sys 0.003s
I20260709 13:18:07.815898 15717 raft_consensus.cc:740] T 7dbabc834ad54c16ae25f79064932f42 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:07.817682 15717 consensus_queue.cc:260] T 7dbabc834ad54c16ae25f79064932f42 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.820245 15676 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:07.821729 15718 raft_consensus.cc:359] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.832837 15718 raft_consensus.cc:385] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:07.833151 15718 raft_consensus.cc:740] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:07.829658 15717 ts_tablet_manager.cc:1434] T 7dbabc834ad54c16ae25f79064932f42 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.023s	user 0.002s	sys 0.004s
I20260709 13:18:07.830250 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:07.818121 15423 catalog_manager.cc:5697] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:07.824321 15720 ts_tablet_manager.cc:1434] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:18:07.839627 16087 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.839875 15717 tablet_bootstrap.cc:492] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:07.840631 16087 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.842710 16087 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:07.847298 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:07.847787 15720 tablet_bootstrap.cc:492] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:07.848369 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:07.848563 15528 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
W20260709 13:18:07.849747 15491 leader_election.cc:343] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:07.847965 15600 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:07.853477 15641 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:07.854627 15890 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:07.853734 15493 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:07.855221 15890 raft_consensus.cc:697] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:07.856181 15890 consensus_queue.cc:237] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.858183 16093 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:07.858633 16093 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.858956 16093 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:07.868806 15718 consensus_queue.cc:260] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.876235 15718 ts_tablet_manager.cc:1434] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.059s	user 0.005s	sys 0.005s
I20260709 13:18:07.877025 16105 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.877653 15718 tablet_bootstrap.cc:492] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:07.877573 16105 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.880684 15720 tablet_bootstrap.cc:654] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:07.882879 16105 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:07.884218 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:07.885504 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:07.886124 15528 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:07.887053 15641 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:18:07.887566 15642 leader_election.cc:343] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:07.891356 16105 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:07.895533 16105 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.896328 16093 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.896857 16105 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:07.903642 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:07.904208 15600 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:07.910965 15600 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:07.915943 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:07.916589 15493 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:18:07.917402 15491 leader_election.cc:343] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:07.917431 16087 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:07.918148 16087 raft_consensus.cc:697] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:07.918399 16093 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:07.919479 16087 consensus_queue.cc:237] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.921900 15423 catalog_manager.cc:5697] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 1 to 2, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:07.924153 15720 tablet_bootstrap.cc:492] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:07.924616 15720 ts_tablet_manager.cc:1403] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.077s	user 0.011s	sys 0.006s
I20260709 13:18:07.925065 15718 tablet_bootstrap.cc:654] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:07.910851 16105 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.928452 16105 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:07.931376 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:07.931594 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:07.932140 15528 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:07.932488 15717 tablet_bootstrap.cc:654] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
W20260709 13:18:07.932673 15642 leader_election.cc:343] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:07.936283 15720 raft_consensus.cc:359] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.936991 15720 raft_consensus.cc:385] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:07.937263 15720 raft_consensus.cc:740] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:07.937672 15528 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:07.937870 15720 consensus_queue.cc:260] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.939018 15641 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:07.940028 15720 ts_tablet_manager.cc:1434] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.015s	user 0.000s	sys 0.004s
I20260709 13:18:07.941200 16105 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:07.941680 16105 raft_consensus.cc:697] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:07.943930 16087 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.944401 16087 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.946097 15423 catalog_manager.cc:5697] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:07.947256 16087 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:07.943943 16105 consensus_queue.cc:237] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.948258 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:07.950362 15491 leader_election.cc:343] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:07.951215 15718 tablet_bootstrap.cc:492] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:07.951668 15718 ts_tablet_manager.cc:1403] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.074s	user 0.008s	sys 0.004s
I20260709 13:18:07.953583 15718 raft_consensus.cc:359] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.954084 15718 raft_consensus.cc:385] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:07.954264 15718 raft_consensus.cc:740] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:07.954737 15718 consensus_queue.cc:260] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.956439 15720 tablet_bootstrap.cc:492] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:07.956687 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:18:07.957880 15493 leader_election.cc:343] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:07.958299 15493 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:07.960527 16102 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:07.961316 15717 tablet_bootstrap.cc:492] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:07.961764 15717 ts_tablet_manager.cc:1403] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.122s	user 0.011s	sys 0.001s
I20260709 13:18:07.962211 16096 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:07.963281 15718 ts_tablet_manager.cc:1434] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:18:07.964349 15718 tablet_bootstrap.cc:492] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69: Bootstrap starting.
I20260709 13:18:07.964741 15717 raft_consensus.cc:359] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.965530 15717 raft_consensus.cc:385] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:07.975515 15718 tablet_bootstrap.cc:654] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:07.976639 16096 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:07.980386 15720 tablet_bootstrap.cc:654] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:07.982512 15717 raft_consensus.cc:740] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:07.983484 15717 consensus_queue.cc:260] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:07.989691 16087 consensus_queue.cc:1048] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:07.997087 15717 ts_tablet_manager.cc:1434] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
I20260709 13:18:07.998387 15717 tablet_bootstrap.cc:492] T 6139553a47b64d798a29413921f18396 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:07.999820 16093 consensus_queue.cc:1048] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:07.997136 15718 tablet_bootstrap.cc:492] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69: No bootstrap required, opened a new log
I20260709 13:18:08.006927 15718 ts_tablet_manager.cc:1403] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent bootstrapping tablet: real 0.043s	user 0.011s	sys 0.003s
I20260709 13:18:08.021987 15423 catalog_manager.cc:5697] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:08.027144 16071 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.028369 16071 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.027733 15718 raft_consensus.cc:359] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.032539 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:08.033238 15676 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:08.030550 15718 raft_consensus.cc:385] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:08.034070 15718 raft_consensus.cc:740] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Initialized, Role: FOLLOWER
I20260709 13:18:08.034353 15566 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:08.035051 16096 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:08.035418 16096 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.035503 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:08.035772 16096 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:08.036072 15528 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:08.045627 16096 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.035069 16071 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:08.056295 15720 tablet_bootstrap.cc:492] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:08.056960 15720 ts_tablet_manager.cc:1403] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.101s	user 0.017s	sys 0.024s
I20260709 13:18:08.061448 15720 raft_consensus.cc:359] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.062150 15720 raft_consensus.cc:385] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:08.062422 15720 raft_consensus.cc:740] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:08.063019 15720 consensus_queue.cc:260] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.054286 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:08.065148 15676 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:08.065824 15720 ts_tablet_manager.cc:1434] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:18:08.066854 15720 tablet_bootstrap.cc:492] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:08.034842 15718 consensus_queue.cc:260] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.070019 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:08.070686 15528 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:08.072027 15720 tablet_bootstrap.cc:654] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:08.078568 15676 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:08.079766 15566 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:08.082144 16071 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:08.049258 16096 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:08.082998 16071 raft_consensus.cc:697] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:08.084008 16071 consensus_queue.cc:237] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.085950 15717 tablet_bootstrap.cc:654] T 6139553a47b64d798a29413921f18396 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:08.088511 15718 ts_tablet_manager.cc:1434] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent starting tablet: real 0.078s	user 0.003s	sys 0.004s
I20260709 13:18:08.116547 16088 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.124207 16088 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.190160 15890 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.190850 15890 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.197826 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:08.229292 15720 tablet_bootstrap.cc:492] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:08.235582 15720 ts_tablet_manager.cc:1403] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.169s	user 0.010s	sys 0.008s
I20260709 13:18:08.235715 15717 tablet_bootstrap.cc:492] T 6139553a47b64d798a29413921f18396 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:08.236183 15717 ts_tablet_manager.cc:1403] T 6139553a47b64d798a29413921f18396 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.238s	user 0.047s	sys 0.052s
I20260709 13:18:08.234156 15890 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:08.237496 15520 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 0.
I20260709 13:18:08.194011 15424 catalog_manager.cc:5697] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:08.238946 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:08.241792 15600 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:08.242018 16088 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:08.242784 15642 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:08.243825 15890 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:08.244122 15890 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.244482 15890 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:08.245162 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 119232us
I20260709 13:18:08.246829 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:08.249823 15720 raft_consensus.cc:359] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.250653 15720 raft_consensus.cc:385] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:08.250926 15720 raft_consensus.cc:740] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:08.251045 15890 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.270360 15720 consensus_queue.cc:260] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.180968 15528 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:08.255625 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:18:08.266333 15566 leader_election.cc:343] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:08.272516 15527 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 0.
I20260709 13:18:08.239475 15717 raft_consensus.cc:359] T 6139553a47b64d798a29413921f18396 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.273089 15720 ts_tablet_manager.cc:1434] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.000s
I20260709 13:18:08.273383 15717 raft_consensus.cc:385] T 6139553a47b64d798a29413921f18396 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:08.273864 15717 raft_consensus.cc:740] T 6139553a47b64d798a29413921f18396 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:08.274418 15720 tablet_bootstrap.cc:492] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: Bootstrap starting.
I20260709 13:18:08.274677 15717 consensus_queue.cc:260] T 6139553a47b64d798a29413921f18396 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.277715 15567 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:08.278725 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:08.279386 15601 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:08.282763 15717 ts_tablet_manager.cc:1434] T 6139553a47b64d798a29413921f18396 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
I20260709 13:18:08.271993 15890 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:08.286995 15717 tablet_bootstrap.cc:492] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:08.290225 15920 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:08.292289 15920 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.292680 15920 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:08.285804 15601 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:08.297056 15642 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:08.297940 16105 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:08.299155 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:08.299806 15527 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:08.301105 15720 tablet_bootstrap.cc:654] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:08.302217 16105 raft_consensus.cc:697] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:08.306773 15527 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:08.303328 16105 consensus_queue.cc:237] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.311902 15717 tablet_bootstrap.cc:654] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:08.312644 15720 tablet_bootstrap.cc:492] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: No bootstrap required, opened a new log
I20260709 13:18:08.313220 15720 ts_tablet_manager.cc:1403] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent bootstrapping tablet: real 0.039s	user 0.004s	sys 0.009s
I20260709 13:18:08.313550 16102 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:18:08.313318 15920 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.322168 16087 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.322656 16087 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.324529 16087 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:08.325213 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:08.326987 15920 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:08.327097 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:08.328023 15566 leader_election.cc:343] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:18:08.328374 15491 leader_election.cc:343] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:18:08.332860 15474 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:08.337065 15717 tablet_bootstrap.cc:492] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:08.341889 15717 ts_tablet_manager.cc:1403] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.055s	user 0.009s	sys 0.011s
I20260709 13:18:08.333999 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:08.343076 15527 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:08.338603 15720 raft_consensus.cc:359] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.343640 15720 raft_consensus.cc:385] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:08.343922 15720 raft_consensus.cc:740] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Initialized, Role: FOLLOWER
I20260709 13:18:08.344547 15720 consensus_queue.cc:260] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.344602 15717 raft_consensus.cc:359] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.345520 15717 raft_consensus.cc:385] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:08.345831 15717 raft_consensus.cc:740] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:08.346210 15720 ts_tablet_manager.cc:1434] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 13:18:08.347126 15717 consensus_queue.cc:260] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.348832 15597 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:08.350416 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:08.351042 15603 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:08.352104 15493 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:08.352869 16102 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:08.353221 16102 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.353544 16102 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:08.357791 15920 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.358306 15920 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.359412 16082 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:08.360700 15527 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:08.361663 15717 ts_tablet_manager.cc:1434] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.003s
I20260709 13:18:08.361547 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:08.363000 15920 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:08.364148 15567 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:08.366598 15920 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:08.367764 15920 raft_consensus.cc:697] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:08.367570 16087 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:18:08.371862 15717 tablet_bootstrap.cc:492] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:08.372917 16148 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:08.374089 15597 raft_consensus.cc:1275] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:08.374629 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:08.375509 15527 raft_consensus.cc:2410] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:18:08.375644 16105 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:18:08.368741 15920 consensus_queue.cc:237] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.381480 15424 catalog_manager.cc:5697] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:08.385603 15567 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:08.386350 16087 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.389460 16087 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.391498 16087 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:08.397382 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:08.398276 15677 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:08.400318 15491 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:08.402691 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:08.403460 15601 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 0.
I20260709 13:18:08.410840 16087 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:08.426004 15890 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:08.426698 16087 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.439790 16087 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:08.396814 16102 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.479043 16102 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:08.507081 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:08.507678 15676 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:08.509600 15717 tablet_bootstrap.cc:654] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:08.511206 15523 raft_consensus.cc:1275] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:08.514158 16082 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:08.522907 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:08.523473 15600 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:08.529590 15600 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:08.532473 16071 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:08.535476 15676 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
W20260709 13:18:08.539815 15491 consensus_peers.cc:597] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 -> Peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Couldn't send request to peer 3ef186ec12554a1d87814af0a0ace9a4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:08.607221 15703 tablet.cc:2406] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260709 13:18:08.607985 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 67173.2us
W20260709 13:18:08.608765 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 68402.3us
I20260709 13:18:08.610536 15717 tablet_bootstrap.cc:492] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:08.611012 15717 ts_tablet_manager.cc:1403] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.239s	user 0.017s	sys 0.012s
W20260709 13:18:08.681531 16159 log.cc:927] Time spent T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 13:18:08.766467 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 191394us
I20260709 13:18:08.769127 15493 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:08.772784 16148 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:08.773303 16148 raft_consensus.cc:697] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:08.774287 16148 consensus_queue.cc:237] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:08.783337 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 173752us
I20260709 13:18:08.773288 16087 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.792438 16087 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:08.787806 15424 catalog_manager.cc:5697] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:08.793605 15717 raft_consensus.cc:359] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.803983 15717 raft_consensus.cc:385] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:08.804705 15717 raft_consensus.cc:740] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:08.807060 15717 consensus_queue.cc:260] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:08.813776 15717 ts_tablet_manager.cc:1434] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.202s	user 0.006s	sys 0.006s
I20260709 13:18:08.842972 15717 tablet_bootstrap.cc:492] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4: Bootstrap starting.
I20260709 13:18:08.843226 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:08.843791 15603 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:08.880429 16088 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:18:08.893927 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 102447us
I20260709 13:18:08.922964 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:08.923591 15675 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:08.928074 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 79759.1us
W20260709 13:18:08.930461 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 91884.4us
I20260709 13:18:08.969094 15717 tablet_bootstrap.cc:654] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:08.971580 15675 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:08.984616 15603 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:08.987080 16170 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:18:08.990865 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 58481.1us
W20260709 13:18:09.021835 15566 consensus_peers.cc:597] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Couldn't send request to peer 3ef186ec12554a1d87814af0a0ace9a4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:09.035413 15675 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:09.049634 15717 tablet_bootstrap.cc:492] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4: No bootstrap required, opened a new log
I20260709 13:18:09.050431 15717 ts_tablet_manager.cc:1403] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent bootstrapping tablet: real 0.210s	user 0.008s	sys 0.009s
I20260709 13:18:09.092374 15717 raft_consensus.cc:359] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:09.126330 15717 raft_consensus.cc:385] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:09.127115 15491 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:09.128301 16177 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:09.130541 16177 raft_consensus.cc:697] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:09.130072 15424 catalog_manager.cc:5697] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:09.131333 16177 consensus_queue.cc:237] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:09.151475 16171 consensus_queue.cc:1048] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:09.127857 15717 raft_consensus.cc:740] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Initialized, Role: FOLLOWER
I20260709 13:18:09.160869 15717 consensus_queue.cc:260] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:09.171346 15717 ts_tablet_manager.cc:1434] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent starting tablet: real 0.121s	user 0.004s	sys 0.003s
I20260709 13:18:09.183246 15425 catalog_manager.cc:5697] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:09.198032 16170 consensus_queue.cc:1048] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:09.201462 16082 consensus_queue.cc:1048] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:09.264253 16178 consensus_queue.cc:1048] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:09.288341 16148 consensus_queue.cc:1048] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:09.294241 16171 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.294765 16171 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:09.296810 16171 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:09.305779 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:09.313297 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:09.314821 15641 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:09.315515 16083 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:09.403887 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:09.415355 16194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:09.416263 16193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:09.418210 16196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:09.442528 14429 server_base.cc:1061] running on GCE node
I20260709 13:18:09.443358 15677 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:09.443706 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:09.443954 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:09.444134 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603089444116 us; error 0 us; skew 500 ppm
I20260709 13:18:09.444770 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:09.448674 14429 webserver.cc:533] Webserver started at http://127.14.23.68:41113/ using document root <none> and password file <none>
I20260709 13:18:09.449261 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:09.449574 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:09.450109 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:09.451850 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-3-root/instance:
uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
format_stamp: "Formatted at 2026-07-09 13:18:09 on dist-test-slave-twc1"
I20260709 13:18:09.461992 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.009s	user 0.004s	sys 0.002s
I20260709 13:18:09.467463 16202 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:09.468624 14429 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:18:09.468999 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-3-root
uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
format_stamp: "Formatted at 2026-07-09 13:18:09 on dist-test-slave-twc1"
I20260709 13:18:09.469363 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:09.498590 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:09.502948 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:09.505175 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:09.508155 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:18:09.508456 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:09.508762 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:18:09.508934 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:09.567152 16148 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:09.582557 16178 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:09.631124 16266 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.68:39627 every 8 connection(s)
I20260709 13:18:09.631688 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.68:39627
I20260709 13:18:09.638082 14429 auto_leader_rebalancer-test.cc:384] add a tserver: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:09.664606 16267 heartbeater.cc:344] Connected to a master server at 127.14.23.126:39251
I20260709 13:18:09.665102 16267 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:09.666018 16267 heartbeater.cc:507] Master 127.14.23.126:39251 requested a full tablet report, sending...
I20260709 13:18:09.667968 15423 ts_manager.cc:194] Registered new tserver with Master: dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:09.669278 15423 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58802
I20260709 13:18:10.488878 15597 consensus_queue.cc:237] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:10.501101 15669 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:10.502902 15519 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:10.502588 16180 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:10.507368 16088 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:18:10.520292 15568 consensus_peers.cc:597] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4b9f476ac2ad44cd8124fde4f1aadc78. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:10.522347 16088 raft_consensus.cc:2955] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:10.523900 15519 raft_consensus.cc:2955] T 4b9f476ac2ad44cd8124fde4f1aadc78 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:10.530028 15669 raft_consensus.cc:2955] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:10.542773 15423 catalog_manager.cc:5697] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:10.553793 15520 consensus_queue.cc:237] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:10.566751 15669 raft_consensus.cc:1275] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:10.570526 16148 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:18:10.693626 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 118402us
I20260709 13:18:10.721665 15602 raft_consensus.cc:1275] T 7562b5f342134e39a5e6934d5ca16e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:10.723697 16267 heartbeater.cc:499] Master 127.14.23.126:39251 was elected leader, sending a full tablet report...
I20260709 13:18:10.725140 16178 raft_consensus.cc:2955] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:10.729346 15677 raft_consensus.cc:2955] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:10.764855 15426 catalog_manager.cc:5697] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:10.765754 15597 consensus_queue.cc:237] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
W20260709 13:18:10.774250 15568 consensus_peers.cc:597] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3088d5241ec64ddc919925d5dccfeda0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:10.775027 15677 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:10.778112 16279 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:10.781780 16284 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:10.798144 15522 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:10.798884 16289 raft_consensus.cc:2955] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:10.801195 16279 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:18:10.801157 15493 consensus_peers.cc:597] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7562b5f342134e39a5e6934d5ca16e46. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:10.801307 15669 raft_consensus.cc:2955] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:10.815860 15602 consensus_queue.cc:237] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:10.821218 15426 catalog_manager.cc:5697] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:10.821012 15522 raft_consensus.cc:2955] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
W20260709 13:18:10.829491 15568 consensus_peers.cc:597] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ef08c91ca56042df89e729f13df83ed9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:10.831446 15677 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:10.834281 16291 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:10.840572 15520 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:10.841842 15597 raft_consensus.cc:2955] T 7562b5f342134e39a5e6934d5ca16e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:10.846060 16291 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:10.859285 16291 raft_consensus.cc:2955] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:10.861425 15669 raft_consensus.cc:2955] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:10.882234 15522 raft_consensus.cc:2955] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:10.889675 15426 catalog_manager.cc:5697] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:10.981312 16301 ts_tablet_manager.cc:933] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:10.983315 16301 tablet_copy_client.cc:323] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.66:41593
I20260709 13:18:10.993813 15613 tablet_copy_service.cc:140] P 5498ef3e89a941e39c25f68cf64e6b0f: Received BeginTabletCopySession request for tablet 4b9f476ac2ad44cd8124fde4f1aadc78 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:34102)
I20260709 13:18:10.994422 15613 tablet_copy_service.cc:161] P 5498ef3e89a941e39c25f68cf64e6b0f: Beginning new tablet copy session on tablet 4b9f476ac2ad44cd8124fde4f1aadc78 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:34102: session id = dd1a14c3173d49dc9b55bb116e3783c9-4b9f476ac2ad44cd8124fde4f1aadc78
I20260709 13:18:11.003355 15613 tablet_copy_source_session.cc:215] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:11.007050 16301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b9f476ac2ad44cd8124fde4f1aadc78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:11.023130 16301 tablet_copy_client.cc:806] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:11.023851 16301 tablet_copy_client.cc:670] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:11.028514 16301 tablet_copy_client.cc:538] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:11.038400 16301 tablet_bootstrap.cc:492] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:11.061168 16301 tablet_bootstrap.cc:492] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:11.061685 16305 ts_tablet_manager.cc:933] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:18:11.062198 16301 tablet_bootstrap.cc:492] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:11.063001 16301 ts_tablet_manager.cc:1403] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.025s	user 0.019s	sys 0.004s
I20260709 13:18:11.063452 16305 tablet_copy_client.cc:323] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.65:42481
I20260709 13:18:11.066671 16301 raft_consensus.cc:359] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:11.067610 16301 raft_consensus.cc:740] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:11.068333 16301 consensus_queue.cc:260] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:11.074206 16301 ts_tablet_manager.cc:1434] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.005s
I20260709 13:18:11.078159 15613 tablet_copy_service.cc:342] P 5498ef3e89a941e39c25f68cf64e6b0f: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-4b9f476ac2ad44cd8124fde4f1aadc78 received from {username='slave'} at 127.0.0.1:34102
I20260709 13:18:11.078683 15613 tablet_copy_service.cc:434] P 5498ef3e89a941e39c25f68cf64e6b0f: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-4b9f476ac2ad44cd8124fde4f1aadc78 on tablet 4b9f476ac2ad44cd8124fde4f1aadc78 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:11.085574 15538 tablet_copy_service.cc:140] P d3c4f5bd76e0445795bad5f6dd93ff69: Received BeginTabletCopySession request for tablet 7562b5f342134e39a5e6934d5ca16e46 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:56840)
I20260709 13:18:11.086077 15538 tablet_copy_service.cc:161] P d3c4f5bd76e0445795bad5f6dd93ff69: Beginning new tablet copy session on tablet 7562b5f342134e39a5e6934d5ca16e46 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:56840: session id = dd1a14c3173d49dc9b55bb116e3783c9-7562b5f342134e39a5e6934d5ca16e46
I20260709 13:18:11.091280 15538 tablet_copy_source_session.cc:215] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:11.093951 16305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7562b5f342134e39a5e6934d5ca16e46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:11.106436 16305 tablet_copy_client.cc:806] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:11.106966 16305 tablet_copy_client.cc:670] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:11.111243 16305 tablet_copy_client.cc:538] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:11.118326 16305 tablet_bootstrap.cc:492] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:11.135044 16305 tablet_bootstrap.cc:492] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:11.135874 16305 tablet_bootstrap.cc:492] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:11.136545 16305 ts_tablet_manager.cc:1403] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.001s
I20260709 13:18:11.139619 16305 raft_consensus.cc:359] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:11.140458 16305 raft_consensus.cc:740] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:11.141078 16305 consensus_queue.cc:260] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:11.143237 16305 ts_tablet_manager.cc:1434] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 13:18:11.145375 15538 tablet_copy_service.cc:342] P d3c4f5bd76e0445795bad5f6dd93ff69: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-7562b5f342134e39a5e6934d5ca16e46 received from {username='slave'} at 127.0.0.1:56840
I20260709 13:18:11.145833 15538 tablet_copy_service.cc:434] P d3c4f5bd76e0445795bad5f6dd93ff69: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-7562b5f342134e39a5e6934d5ca16e46 on tablet 7562b5f342134e39a5e6934d5ca16e46 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:11.344396 16305 ts_tablet_manager.cc:933] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:11.346813 16305 tablet_copy_client.cc:323] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.66:41593
I20260709 13:18:11.348368 15613 tablet_copy_service.cc:140] P 5498ef3e89a941e39c25f68cf64e6b0f: Received BeginTabletCopySession request for tablet ef08c91ca56042df89e729f13df83ed9 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:34102)
I20260709 13:18:11.348932 15613 tablet_copy_service.cc:161] P 5498ef3e89a941e39c25f68cf64e6b0f: Beginning new tablet copy session on tablet ef08c91ca56042df89e729f13df83ed9 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:34102: session id = dd1a14c3173d49dc9b55bb116e3783c9-ef08c91ca56042df89e729f13df83ed9
I20260709 13:18:11.356141 15613 tablet_copy_source_session.cc:215] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:11.359035 16305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef08c91ca56042df89e729f13df83ed9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:11.370447 16305 tablet_copy_client.cc:806] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:11.370994 16305 tablet_copy_client.cc:670] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:11.374951 16305 tablet_copy_client.cc:538] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:11.379225 16301 ts_tablet_manager.cc:933] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:11.385704 16305 tablet_bootstrap.cc:492] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:11.387329 16301 tablet_copy_client.cc:323] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.66:41593
I20260709 13:18:11.389077 15613 tablet_copy_service.cc:140] P 5498ef3e89a941e39c25f68cf64e6b0f: Received BeginTabletCopySession request for tablet 3088d5241ec64ddc919925d5dccfeda0 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:34102)
I20260709 13:18:11.389631 15613 tablet_copy_service.cc:161] P 5498ef3e89a941e39c25f68cf64e6b0f: Beginning new tablet copy session on tablet 3088d5241ec64ddc919925d5dccfeda0 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:34102: session id = dd1a14c3173d49dc9b55bb116e3783c9-3088d5241ec64ddc919925d5dccfeda0
I20260709 13:18:11.397289 15613 tablet_copy_source_session.cc:215] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:11.400754 16301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3088d5241ec64ddc919925d5dccfeda0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:11.406977 16305 tablet_bootstrap.cc:492] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:11.407924 16305 tablet_bootstrap.cc:492] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:11.408586 16305 ts_tablet_manager.cc:1403] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.003s
I20260709 13:18:11.411504 16305 raft_consensus.cc:359] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:11.412091 16305 raft_consensus.cc:740] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:11.412578 16305 consensus_queue.cc:260] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:11.414573 16305 ts_tablet_manager.cc:1434] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 13:18:11.416642 15613 tablet_copy_service.cc:342] P 5498ef3e89a941e39c25f68cf64e6b0f: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-ef08c91ca56042df89e729f13df83ed9 received from {username='slave'} at 127.0.0.1:34102
I20260709 13:18:11.417073 15613 tablet_copy_service.cc:434] P 5498ef3e89a941e39c25f68cf64e6b0f: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-ef08c91ca56042df89e729f13df83ed9 on tablet ef08c91ca56042df89e729f13df83ed9 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:11.419979 16301 tablet_copy_client.cc:806] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:11.420933 16301 tablet_copy_client.cc:670] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:11.431250 16301 tablet_copy_client.cc:538] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:11.438534 16301 tablet_bootstrap.cc:492] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:11.450554 16240 raft_consensus.cc:1217] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:11.468288 16301 tablet_bootstrap.cc:492] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:11.469064 16301 tablet_bootstrap.cc:492] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:11.469607 16301 ts_tablet_manager.cc:1403] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.005s
I20260709 13:18:11.471927 16301 raft_consensus.cc:359] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:11.472553 16301 raft_consensus.cc:740] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:11.473066 16301 consensus_queue.cc:260] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:11.474776 16301 ts_tablet_manager.cc:1434] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:18:11.476403 15613 tablet_copy_service.cc:342] P 5498ef3e89a941e39c25f68cf64e6b0f: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-3088d5241ec64ddc919925d5dccfeda0 received from {username='slave'} at 127.0.0.1:34102
I20260709 13:18:11.476732 15613 tablet_copy_service.cc:434] P 5498ef3e89a941e39c25f68cf64e6b0f: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-3088d5241ec64ddc919925d5dccfeda0 on tablet 3088d5241ec64ddc919925d5dccfeda0 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:11.524410 16240 raft_consensus.cc:1217] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:11.747095 16240 raft_consensus.cc:1217] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:11.814728 16240 raft_consensus.cc:1217] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:12.050491 16276 raft_consensus.cc:1064] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:12.052615 16276 consensus_queue.cc:237] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:12.063345 16240 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:12.064728 16289 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:12.064983 15523 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:12.065019 15669 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:12.066908 16289 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:12.070297 16276 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:12.080703 16290 raft_consensus.cc:2955] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.086241 16240 raft_consensus.cc:2955] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.089354 15520 raft_consensus.cc:2955] T 4b9f476ac2ad44cd8124fde4f1aadc78 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.100558 15424 catalog_manager.cc:5697] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.099892 15669 raft_consensus.cc:2955] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.178638 16178 raft_consensus.cc:1064] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:12.181281 16178 consensus_queue.cc:237] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:12.185976 15602 consensus_queue.cc:237] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:12.194990 16240 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:12.196050 15669 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:12.196761 16320 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:12.197723 16290 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:12.210618 16276 raft_consensus.cc:2955] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.212373 15677 raft_consensus.cc:2955] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.214680 16240 raft_consensus.cc:2955] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.218724 16239 raft_consensus.cc:1275] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:12.221484 16325 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:12.230261 15669 raft_consensus.cc:1275] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:12.232704 15602 raft_consensus.cc:1275] T 7562b5f342134e39a5e6934d5ca16e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:12.231995 16325 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:12.235831 16178 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:12.241516 15423 catalog_manager.cc:5697] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.251349 16320 raft_consensus.cc:1064] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:12.253901 16320 consensus_queue.cc:237] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:12.261904 15514 tablet_service.cc:1558] Processing DeleteTablet for tablet 4b9f476ac2ad44cd8124fde4f1aadc78 with delete_type TABLET_DATA_TOMBSTONED (TS d3c4f5bd76e0445795bad5f6dd93ff69 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57292
I20260709 13:18:12.281775 16289 raft_consensus.cc:1064] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:12.287617 15412 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4b9f476ac2ad44cd8124fde4f1aadc78 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:12.335805 16148 raft_consensus.cc:2955] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.339426 16239 raft_consensus.cc:2955] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.371657 15669 raft_consensus.cc:2955] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.376963 16289 consensus_queue.cc:237] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:12.366202 15426 catalog_manager.cc:5697] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.388913 16334 tablet_replica.cc:333] T 4b9f476ac2ad44cd8124fde4f1aadc78 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
W20260709 13:18:12.402935 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 127537us
I20260709 13:18:12.417209 15603 raft_consensus.cc:2955] T 7562b5f342134e39a5e6934d5ca16e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.439173 16334 raft_consensus.cc:2243] T 4b9f476ac2ad44cd8124fde4f1aadc78 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:12.439954 16334 raft_consensus.cc:2272] T 4b9f476ac2ad44cd8124fde4f1aadc78 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:12.445185 16334 ts_tablet_manager.cc:1916] T 4b9f476ac2ad44cd8124fde4f1aadc78 P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:12.448230 15675 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:12.454146 15519 consensus_queue.cc:237] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:12.466141 15522 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:12.461485 16239 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:12.464221 16240 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:12.465199 15675 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:12.467893 16336 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:12.464236 16337 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:18:12.471823 16333 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:12.474200 16320 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:12.475337 16336 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:18:12.486186 16238 raft_consensus.cc:1275] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:12.488863 15670 raft_consensus.cc:1275] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:12.489833 16178 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:12.493588 16148 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:12.503264 15519 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:12.504856 16289 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:12.507586 16334 ts_tablet_manager.cc:1929] T 4b9f476ac2ad44cd8124fde4f1aadc78 P d3c4f5bd76e0445795bad5f6dd93ff69: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:12.508101 16334 log.cc:1199] T 4b9f476ac2ad44cd8124fde4f1aadc78 P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-0-root/wals/4b9f476ac2ad44cd8124fde4f1aadc78
I20260709 13:18:12.507216 16328 raft_consensus.cc:2955] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.509883 15411 catalog_manager.cc:5028] TS d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): tablet 4b9f476ac2ad44cd8124fde4f1aadc78 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:12.509177 16336 raft_consensus.cc:2955] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.515648 15677 raft_consensus.cc:2955] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.530256 16333 raft_consensus.cc:2955] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.534204 15675 raft_consensus.cc:2955] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.535887 15423 catalog_manager.cc:5697] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.537626 15677 raft_consensus.cc:2955] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.560942 16239 raft_consensus.cc:2955] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.566515 16240 raft_consensus.cc:2955] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.572733 15411 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7562b5f342134e39a5e6934d5ca16e46 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:12.579749 15522 raft_consensus.cc:2955] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.590106 15425 catalog_manager.cc:5697] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:12.591928 15520 raft_consensus.cc:2955] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.595497 16240 raft_consensus.cc:2955] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.610332 15603 consensus_queue.cc:237] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:12.616731 15424 catalog_manager.cc:5697] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.618248 15589 tablet_service.cc:1558] Processing DeleteTablet for tablet 7562b5f342134e39a5e6934d5ca16e46 with delete_type TABLET_DATA_TOMBSTONED (TS 5498ef3e89a941e39c25f68cf64e6b0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53300
I20260709 13:18:12.626379 16346 tablet_replica.cc:333] T 7562b5f342134e39a5e6934d5ca16e46 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:18:12.627481 16346 raft_consensus.cc:2243] T 7562b5f342134e39a5e6934d5ca16e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:12.631469 16346 raft_consensus.cc:2272] T 7562b5f342134e39a5e6934d5ca16e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:12.634945 16346 ts_tablet_manager.cc:1916] T 7562b5f342134e39a5e6934d5ca16e46 P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:12.644722 15669 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:12.646327 16333 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:12.648368 16240 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:12.650323 16333 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:12.653172 15603 consensus_queue.cc:237] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:12.655102 16320 raft_consensus.cc:2955] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.657712 16346 ts_tablet_manager.cc:1929] T 7562b5f342134e39a5e6934d5ca16e46 P 5498ef3e89a941e39c25f68cf64e6b0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:12.658212 16346 log.cc:1199] T 7562b5f342134e39a5e6934d5ca16e46 P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-1-root/wals/7562b5f342134e39a5e6934d5ca16e46
I20260709 13:18:12.659031 16240 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:12.659890 15412 catalog_manager.cc:5028] TS 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): tablet 7562b5f342134e39a5e6934d5ca16e46 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:12.661546 16336 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:12.667557 15412 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ef08c91ca56042df89e729f13df83ed9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:12.668144 15520 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:12.671200 16333 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:12.668198 15669 raft_consensus.cc:2955] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.676043 16336 raft_consensus.cc:2955] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.684929 16239 raft_consensus.cc:2955] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.681267 15520 raft_consensus.cc:2955] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.690112 15426 catalog_manager.cc:5697] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:12.699642 16240 raft_consensus.cc:2955] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.703936 15514 tablet_service.cc:1558] Processing DeleteTablet for tablet ef08c91ca56042df89e729f13df83ed9 with delete_type TABLET_DATA_TOMBSTONED (TS d3c4f5bd76e0445795bad5f6dd93ff69 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57292
I20260709 13:18:12.704931 16334 tablet_replica.cc:333] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:18:12.729488 15426 catalog_manager.cc:5697] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:12.738845 15412 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3088d5241ec64ddc919925d5dccfeda0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:12.747486 15660 tablet_service.cc:1558] Processing DeleteTablet for tablet 3088d5241ec64ddc919925d5dccfeda0 with delete_type TABLET_DATA_TOMBSTONED (TS 3ef186ec12554a1d87814af0a0ace9a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49088
I20260709 13:18:12.755405 16347 tablet_replica.cc:333] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:18:12.758320 16347 raft_consensus.cc:2243] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:12.758772 16334 raft_consensus.cc:2243] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:12.759565 16334 raft_consensus.cc:2272] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:12.759091 16347 raft_consensus.cc:2272] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:12.762499 16334 ts_tablet_manager.cc:1916] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:12.762804 16347 ts_tablet_manager.cc:1916] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:12.778357 16347 ts_tablet_manager.cc:1929] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:12.778357 16334 ts_tablet_manager.cc:1929] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:12.778899 16347 log.cc:1199] T 3088d5241ec64ddc919925d5dccfeda0 P 3ef186ec12554a1d87814af0a0ace9a4: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-2-root/wals/3088d5241ec64ddc919925d5dccfeda0
I20260709 13:18:12.779016 16334 log.cc:1199] T ef08c91ca56042df89e729f13df83ed9 P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-0-root/wals/ef08c91ca56042df89e729f13df83ed9
I20260709 13:18:12.780566 15411 catalog_manager.cc:5028] TS d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): tablet ef08c91ca56042df89e729f13df83ed9 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:12.780822 15410 catalog_manager.cc:5028] TS 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): tablet 3088d5241ec64ddc919925d5dccfeda0 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:13.819106 15603 consensus_queue.cc:237] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
W20260709 13:18:13.825981 15568 consensus_peers.cc:597] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: db313a4cd18a47a2b72587d04ed6e747. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:13.826162 15519 raft_consensus.cc:1275] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:13.826289 15670 raft_consensus.cc:1275] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:13.827469 16336 consensus_queue.cc:1048] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:13.828372 16276 consensus_queue.cc:1048] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:13.836644 16336 raft_consensus.cc:2955] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:13.842685 15519 raft_consensus.cc:2955] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:13.847506 15670 raft_consensus.cc:2955] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:13.850759 15520 consensus_queue.cc:237] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:13.858975 15426 catalog_manager.cc:5697] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:13.860286 15493 consensus_peers.cc:597] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e67cef93bbdb4ff88a2ee2ac558ba090. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:13.861995 15603 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:13.863638 16148 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:13.867537 15670 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:13.869272 16329 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:13.875808 16329 raft_consensus.cc:2955] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:13.877552 15595 raft_consensus.cc:2955] T e67cef93bbdb4ff88a2ee2ac558ba090 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:13.882150 15669 raft_consensus.cc:2955] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:13.890206 15597 consensus_queue.cc:237] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
W20260709 13:18:13.900645 15568 consensus_peers.cc:597] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7b8ed23a60014da39806e750cee98fa6. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:13.901407 15669 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:13.901309 15520 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:13.907207 16276 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:13.908406 16350 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:13.910722 15424 catalog_manager.cc:5697] T e67cef93bbdb4ff88a2ee2ac558ba090 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:13.937175 16276 raft_consensus.cc:2955] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:13.940435 15669 raft_consensus.cc:2955] T 7b8ed23a60014da39806e750cee98fa6 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:13.941498 15519 raft_consensus.cc:2955] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:13.957288 15424 catalog_manager.cc:5697] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:13.972223 15669 consensus_queue.cc:237] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:13.979504 15519 raft_consensus.cc:1275] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:13.981024 16322 consensus_queue.cc:1048] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:13.989880 15595 raft_consensus.cc:1275] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:13.991532 16348 raft_consensus.cc:2955] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:13.992667 16293 consensus_queue.cc:1048] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:13.994418 15519 raft_consensus.cc:2955] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:14.010423 15595 raft_consensus.cc:2955] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:14.016530 15426 catalog_manager.cc:5697] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
W20260709 13:18:14.021508 15642 consensus_peers.cc:597] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0f8365fc4e2f4decb3d554a43e7d4d52. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:14.232203 16373 ts_tablet_manager.cc:933] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:14.234117 16373 tablet_copy_client.cc:323] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.66:41593
I20260709 13:18:14.236461 15613 tablet_copy_service.cc:140] P 5498ef3e89a941e39c25f68cf64e6b0f: Received BeginTabletCopySession request for tablet db313a4cd18a47a2b72587d04ed6e747 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:34102)
I20260709 13:18:14.237023 15613 tablet_copy_service.cc:161] P 5498ef3e89a941e39c25f68cf64e6b0f: Beginning new tablet copy session on tablet db313a4cd18a47a2b72587d04ed6e747 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:34102: session id = dd1a14c3173d49dc9b55bb116e3783c9-db313a4cd18a47a2b72587d04ed6e747
I20260709 13:18:14.244556 15613 tablet_copy_source_session.cc:215] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:14.247684 16373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db313a4cd18a47a2b72587d04ed6e747. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:14.259573 16373 tablet_copy_client.cc:806] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:14.260104 16373 tablet_copy_client.cc:670] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:14.264607 16373 tablet_copy_client.cc:538] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:14.271499 16373 tablet_bootstrap.cc:492] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:14.288767 16373 tablet_bootstrap.cc:492] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:14.289708 16373 tablet_bootstrap.cc:492] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:14.290390 16373 ts_tablet_manager.cc:1403] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.005s
I20260709 13:18:14.292619 16373 raft_consensus.cc:359] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:14.293176 16373 raft_consensus.cc:740] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:14.293666 16373 consensus_queue.cc:260] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:14.297192 16373 ts_tablet_manager.cc:1434] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:18:14.301162 15613 tablet_copy_service.cc:342] P 5498ef3e89a941e39c25f68cf64e6b0f: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-db313a4cd18a47a2b72587d04ed6e747 received from {username='slave'} at 127.0.0.1:34102
I20260709 13:18:14.301606 15613 tablet_copy_service.cc:434] P 5498ef3e89a941e39c25f68cf64e6b0f: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-db313a4cd18a47a2b72587d04ed6e747 on tablet db313a4cd18a47a2b72587d04ed6e747 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:14.305629 16373 ts_tablet_manager.cc:933] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:18:14.323743 16373 tablet_copy_client.cc:323] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.65:42481
I20260709 13:18:14.325608 15538 tablet_copy_service.cc:140] P d3c4f5bd76e0445795bad5f6dd93ff69: Received BeginTabletCopySession request for tablet e67cef93bbdb4ff88a2ee2ac558ba090 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:56840)
I20260709 13:18:14.326176 15538 tablet_copy_service.cc:161] P d3c4f5bd76e0445795bad5f6dd93ff69: Beginning new tablet copy session on tablet e67cef93bbdb4ff88a2ee2ac558ba090 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:56840: session id = dd1a14c3173d49dc9b55bb116e3783c9-e67cef93bbdb4ff88a2ee2ac558ba090
I20260709 13:18:14.333516 15538 tablet_copy_source_session.cc:215] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:14.342146 16373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e67cef93bbdb4ff88a2ee2ac558ba090. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:14.363447 16373 tablet_copy_client.cc:806] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:14.364401 16373 tablet_copy_client.cc:670] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:14.370342 16373 tablet_copy_client.cc:538] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:14.380412 16373 tablet_bootstrap.cc:492] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:14.400876 16373 tablet_bootstrap.cc:492] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:14.401854 16373 tablet_bootstrap.cc:492] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:14.402742 16373 ts_tablet_manager.cc:1403] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.008s
I20260709 13:18:14.406154 16373 raft_consensus.cc:359] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:14.407015 16373 raft_consensus.cc:740] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:14.407657 16373 consensus_queue.cc:260] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:14.410672 16373 ts_tablet_manager.cc:1434] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:18:14.410863 16377 ts_tablet_manager.cc:933] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:14.412495 15538 tablet_copy_service.cc:342] P d3c4f5bd76e0445795bad5f6dd93ff69: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-e67cef93bbdb4ff88a2ee2ac558ba090 received from {username='slave'} at 127.0.0.1:56840
I20260709 13:18:14.412933 15538 tablet_copy_service.cc:434] P d3c4f5bd76e0445795bad5f6dd93ff69: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-e67cef93bbdb4ff88a2ee2ac558ba090 on tablet e67cef93bbdb4ff88a2ee2ac558ba090 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:14.437067 16377 tablet_copy_client.cc:323] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.67:32779
I20260709 13:18:14.452026 15687 tablet_copy_service.cc:140] P 3ef186ec12554a1d87814af0a0ace9a4: Received BeginTabletCopySession request for tablet 0f8365fc4e2f4decb3d554a43e7d4d52 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:57710)
I20260709 13:18:14.452642 15687 tablet_copy_service.cc:161] P 3ef186ec12554a1d87814af0a0ace9a4: Beginning new tablet copy session on tablet 0f8365fc4e2f4decb3d554a43e7d4d52 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:57710: session id = dd1a14c3173d49dc9b55bb116e3783c9-0f8365fc4e2f4decb3d554a43e7d4d52
I20260709 13:18:14.460232 15687 tablet_copy_source_session.cc:215] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:14.464613 16377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f8365fc4e2f4decb3d554a43e7d4d52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:14.481868 16377 tablet_copy_client.cc:806] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:14.482553 16377 tablet_copy_client.cc:670] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:14.487547 16377 tablet_copy_client.cc:538] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:14.496034 16377 tablet_bootstrap.cc:492] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:14.496775 16373 ts_tablet_manager.cc:933] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:14.509604 16373 tablet_copy_client.cc:323] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.66:41593
I20260709 13:18:14.511379 15613 tablet_copy_service.cc:140] P 5498ef3e89a941e39c25f68cf64e6b0f: Received BeginTabletCopySession request for tablet 7b8ed23a60014da39806e750cee98fa6 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:34102)
I20260709 13:18:14.511860 15613 tablet_copy_service.cc:161] P 5498ef3e89a941e39c25f68cf64e6b0f: Beginning new tablet copy session on tablet 7b8ed23a60014da39806e750cee98fa6 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:34102: session id = dd1a14c3173d49dc9b55bb116e3783c9-7b8ed23a60014da39806e750cee98fa6
I20260709 13:18:14.519142 16377 tablet_bootstrap.cc:492] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:14.520073 16377 tablet_bootstrap.cc:492] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:14.520743 16377 ts_tablet_manager.cc:1403] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.005s
I20260709 13:18:14.521860 15613 tablet_copy_source_session.cc:215] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:14.523635 16377 raft_consensus.cc:359] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:14.525812 16377 raft_consensus.cc:740] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:14.526476 16373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b8ed23a60014da39806e750cee98fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:14.526378 16377 consensus_queue.cc:260] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:14.528622 16377 ts_tablet_manager.cc:1434] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:18:14.530664 15687 tablet_copy_service.cc:342] P 3ef186ec12554a1d87814af0a0ace9a4: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-0f8365fc4e2f4decb3d554a43e7d4d52 received from {username='slave'} at 127.0.0.1:57710
I20260709 13:18:14.531143 15687 tablet_copy_service.cc:434] P 3ef186ec12554a1d87814af0a0ace9a4: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-0f8365fc4e2f4decb3d554a43e7d4d52 on tablet 0f8365fc4e2f4decb3d554a43e7d4d52 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:14.547665 16373 tablet_copy_client.cc:806] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:14.548170 16373 tablet_copy_client.cc:670] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:14.551740 16373 tablet_copy_client.cc:538] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:14.558218 16373 tablet_bootstrap.cc:492] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:14.578670 16373 tablet_bootstrap.cc:492] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:14.579589 16373 tablet_bootstrap.cc:492] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:14.580266 16373 ts_tablet_manager.cc:1403] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.000s
I20260709 13:18:14.582891 16373 raft_consensus.cc:359] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:14.583663 16373 raft_consensus.cc:740] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:14.584247 16373 consensus_queue.cc:260] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:14.586392 16373 ts_tablet_manager.cc:1434] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:18:14.587901 15613 tablet_copy_service.cc:342] P 5498ef3e89a941e39c25f68cf64e6b0f: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-7b8ed23a60014da39806e750cee98fa6 received from {username='slave'} at 127.0.0.1:34102
I20260709 13:18:14.588308 15613 tablet_copy_service.cc:434] P 5498ef3e89a941e39c25f68cf64e6b0f: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-7b8ed23a60014da39806e750cee98fa6 on tablet 7b8ed23a60014da39806e750cee98fa6 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:14.735277 16240 raft_consensus.cc:1217] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:14.846494 16240 raft_consensus.cc:1217] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:14.918447 16240 raft_consensus.cc:1217] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:14.978749 16240 raft_consensus.cc:1217] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:15.134302 16353 raft_consensus.cc:1064] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:15.136876 16353 consensus_queue.cc:237] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:15.148905 16240 raft_consensus.cc:1275] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:15.148905 15520 raft_consensus.cc:1275] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:15.149734 15670 raft_consensus.cc:1275] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:15.152146 16388 consensus_queue.cc:1048] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:15.153098 16276 consensus_queue.cc:1048] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:15.154773 16276 consensus_queue.cc:1048] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:15.188071 16276 raft_consensus.cc:2955] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.191957 15519 raft_consensus.cc:2955] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.195999 16239 raft_consensus.cc:2955] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.211158 15425 catalog_manager.cc:5697] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.209707 15670 raft_consensus.cc:2955] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.241600 15595 consensus_queue.cc:237] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:15.248327 16239 raft_consensus.cc:1275] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:15.248327 15519 raft_consensus.cc:1275] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:15.250201 16276 consensus_queue.cc:1048] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:15.252028 16336 consensus_queue.cc:1048] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:15.258850 16336 raft_consensus.cc:2955] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.262585 16239 raft_consensus.cc:2955] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.264415 15519 raft_consensus.cc:2955] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.272658 15412 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet db313a4cd18a47a2b72587d04ed6e747 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:15.286218 15423 catalog_manager.cc:5697] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:15.298583 15660 tablet_service.cc:1558] Processing DeleteTablet for tablet db313a4cd18a47a2b72587d04ed6e747 with delete_type TABLET_DATA_TOMBSTONED (TS 3ef186ec12554a1d87814af0a0ace9a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49088
I20260709 13:18:15.300385 16396 tablet_replica.cc:333] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:18:15.301574 16396 raft_consensus.cc:2243] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:15.302404 16396 raft_consensus.cc:2272] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:15.305472 16396 ts_tablet_manager.cc:1916] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:15.323141 16396 ts_tablet_manager.cc:1929] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:15.323630 16396 log.cc:1199] T db313a4cd18a47a2b72587d04ed6e747 P 3ef186ec12554a1d87814af0a0ace9a4: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-2-root/wals/db313a4cd18a47a2b72587d04ed6e747
I20260709 13:18:15.325225 15410 catalog_manager.cc:5028] TS 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): tablet db313a4cd18a47a2b72587d04ed6e747 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:15.454247 16329 raft_consensus.cc:1064] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:15.457376 16329 consensus_queue.cc:237] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:15.480392 15677 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:15.480420 15597 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:15.482723 16329 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:15.487483 16239 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:15.489625 16329 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:15.491605 16148 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:15.494480 16380 raft_consensus.cc:1064] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:15.498096 16380 consensus_queue.cc:237] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:15.506418 16240 raft_consensus.cc:1275] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:15.508412 16380 consensus_queue.cc:1048] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:15.509298 15597 raft_consensus.cc:1275] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:15.514302 16397 consensus_queue.cc:1048] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:15.517133 16385 raft_consensus.cc:2955] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.542768 15522 raft_consensus.cc:1275] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:15.546995 16397 consensus_queue.cc:1048] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:15.552834 15602 raft_consensus.cc:2955] T e67cef93bbdb4ff88a2ee2ac558ba090 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.568714 16380 raft_consensus.cc:2955] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.568957 15677 raft_consensus.cc:2955] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.564981 15426 catalog_manager.cc:5697] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:15.579605 15519 raft_consensus.cc:2955] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.581888 16336 raft_consensus.cc:1064] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:15.581907 16240 raft_consensus.cc:2955] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.584510 16336 consensus_queue.cc:237] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:15.586354 16239 raft_consensus.cc:2955] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.608099 15603 raft_consensus.cc:2955] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.624431 15426 catalog_manager.cc:5697] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.626219 15519 consensus_queue.cc:237] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:15.632169 16240 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:15.635756 15670 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:15.635985 16385 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:15.633450 15677 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:15.638851 15522 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:15.639325 16239 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:15.641114 16336 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:15.641901 16388 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:15.642369 16385 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:15.646113 16388 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:15.653991 16405 raft_consensus.cc:2955] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.659722 15670 consensus_queue.cc:237] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:15.674787 15411 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e67cef93bbdb4ff88a2ee2ac558ba090 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:15.676493 15675 raft_consensus.cc:2955] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.679811 16238 raft_consensus.cc:1275] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:15.681969 16401 consensus_queue.cc:1048] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:15.687048 15522 raft_consensus.cc:1275] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:15.690483 16401 consensus_queue.cc:1048] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:15.691913 16240 raft_consensus.cc:2955] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.699966 16397 raft_consensus.cc:2955] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.707086 16353 raft_consensus.cc:2955] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.712110 16239 raft_consensus.cc:2955] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.718815 15677 raft_consensus.cc:2955] T 7b8ed23a60014da39806e750cee98fa6 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.719655 16237 raft_consensus.cc:2955] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.725140 15423 catalog_manager.cc:5697] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:15.743211 15589 tablet_service.cc:1558] Processing DeleteTablet for tablet e67cef93bbdb4ff88a2ee2ac558ba090 with delete_type TABLET_DATA_TOMBSTONED (TS 5498ef3e89a941e39c25f68cf64e6b0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53300
I20260709 13:18:15.764976 15410 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0f8365fc4e2f4decb3d554a43e7d4d52 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:15.778617 16415 tablet_replica.cc:333] T e67cef93bbdb4ff88a2ee2ac558ba090 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:18:15.779755 16415 raft_consensus.cc:2243] T e67cef93bbdb4ff88a2ee2ac558ba090 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:15.779491 15423 catalog_manager.cc:5697] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.780612 16415 raft_consensus.cc:2272] T e67cef93bbdb4ff88a2ee2ac558ba090 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:15.791121 16415 ts_tablet_manager.cc:1916] T e67cef93bbdb4ff88a2ee2ac558ba090 P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:15.797288 15522 raft_consensus.cc:2955] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.803906 15520 raft_consensus.cc:2955] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.809937 15424 catalog_manager.cc:5697] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:15.813007 16415 ts_tablet_manager.cc:1929] T e67cef93bbdb4ff88a2ee2ac558ba090 P 5498ef3e89a941e39c25f68cf64e6b0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:15.813455 16415 log.cc:1199] T e67cef93bbdb4ff88a2ee2ac558ba090 P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-1-root/wals/e67cef93bbdb4ff88a2ee2ac558ba090
I20260709 13:18:15.816035 15589 tablet_service.cc:1558] Processing DeleteTablet for tablet 0f8365fc4e2f4decb3d554a43e7d4d52 with delete_type TABLET_DATA_TOMBSTONED (TS 5498ef3e89a941e39c25f68cf64e6b0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53300
I20260709 13:18:15.817032 16415 tablet_replica.cc:333] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:18:15.817051 15412 catalog_manager.cc:5028] TS 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): tablet e67cef93bbdb4ff88a2ee2ac558ba090 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:15.824934 15603 consensus_queue.cc:237] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:15.839354 16415 raft_consensus.cc:2243] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:15.840126 16415 raft_consensus.cc:2272] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:15.842321 16239 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:15.844094 16276 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:15.852804 16388 raft_consensus.cc:2955] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.866899 16415 ts_tablet_manager.cc:1916] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:15.869112 16239 raft_consensus.cc:2955] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.873306 15520 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:15.878943 16276 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:15.884718 15520 raft_consensus.cc:2955] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:15.890347 16415 ts_tablet_manager.cc:1929] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 5498ef3e89a941e39c25f68cf64e6b0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:15.890748 15412 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7b8ed23a60014da39806e750cee98fa6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:15.890938 16415 log.cc:1199] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-1-root/wals/0f8365fc4e2f4decb3d554a43e7d4d52
I20260709 13:18:15.895188 15412 catalog_manager.cc:5028] TS 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): tablet 0f8365fc4e2f4decb3d554a43e7d4d52 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:15.901275 15425 catalog_manager.cc:5697] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:15.918521 15660 tablet_service.cc:1558] Processing DeleteTablet for tablet 7b8ed23a60014da39806e750cee98fa6 with delete_type TABLET_DATA_TOMBSTONED (TS 3ef186ec12554a1d87814af0a0ace9a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49088
I20260709 13:18:15.920831 16416 tablet_replica.cc:333] T 7b8ed23a60014da39806e750cee98fa6 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:18:15.921891 16416 raft_consensus.cc:2243] T 7b8ed23a60014da39806e750cee98fa6 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:15.922616 16416 raft_consensus.cc:2272] T 7b8ed23a60014da39806e750cee98fa6 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:15.925714 16416 ts_tablet_manager.cc:1916] T 7b8ed23a60014da39806e750cee98fa6 P 3ef186ec12554a1d87814af0a0ace9a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:15.937879 16416 ts_tablet_manager.cc:1929] T 7b8ed23a60014da39806e750cee98fa6 P 3ef186ec12554a1d87814af0a0ace9a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:15.938262 16416 log.cc:1199] T 7b8ed23a60014da39806e750cee98fa6 P 3ef186ec12554a1d87814af0a0ace9a4: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-2-root/wals/7b8ed23a60014da39806e750cee98fa6
I20260709 13:18:15.939529 15410 catalog_manager.cc:5028] TS 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): tablet 7b8ed23a60014da39806e750cee98fa6 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:16.986756 15677 consensus_queue.cc:237] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
W20260709 13:18:16.994051 15642 consensus_peers.cc:597] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b87d5c4641304e03a8a3fd4782b043e5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:16.994300 15520 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:16.994637 15595 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:16.995911 16398 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:16.996917 16420 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:17.006906 16398 raft_consensus.cc:2955] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:17.009377 15602 raft_consensus.cc:2955] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:17.011085 15519 raft_consensus.cc:2955] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:17.026031 15520 consensus_queue.cc:237] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:17.026604 15423 catalog_manager.cc:5697] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:17.033870 15677 raft_consensus.cc:1275] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:18:17.034655 15493 consensus_peers.cc:597] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 15e5bd79113c48c5a0b0f265bdd1d7b2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:17.038324 16148 consensus_queue.cc:1048] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:17.039824 15602 raft_consensus.cc:1275] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:17.041584 16148 consensus_queue.cc:1048] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:17.050089 16418 raft_consensus.cc:2955] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:17.051820 15675 raft_consensus.cc:2955] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:17.052526 15602 raft_consensus.cc:2955] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:17.069140 15520 consensus_queue.cc:237] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:17.085335 15424 catalog_manager.cc:5697] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:17.091374 15675 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:18:17.092052 15493 consensus_peers.cc:597] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 01ad9d7b8dba45be8fdf625f27e8d273. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:17.093284 16148 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:17.093441 15602 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:17.094990 16418 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:17.110064 16148 raft_consensus.cc:2955] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:17.111750 15602 raft_consensus.cc:2955] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:17.111956 15675 raft_consensus.cc:2955] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:17.124672 15595 consensus_queue.cc:237] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:17.132656 15675 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:17.133876 15520 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:17.134266 16276 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:17.135440 16388 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:18:17.136438 15568 consensus_peers.cc:597] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f9328896aa5a4fdc8fa7c908efdae695. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:17.128952 15423 catalog_manager.cc:5697] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:17.160632 16336 raft_consensus.cc:2955] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:17.171015 15520 raft_consensus.cc:2955] T f9328896aa5a4fdc8fa7c908efdae695 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:17.176487 15675 raft_consensus.cc:2955] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:17.197453 15425 catalog_manager.cc:5697] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:17.456714 16437 ts_tablet_manager.cc:933] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:17.462713 16437 tablet_copy_client.cc:323] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.67:32779
I20260709 13:18:17.464682 15687 tablet_copy_service.cc:140] P 3ef186ec12554a1d87814af0a0ace9a4: Received BeginTabletCopySession request for tablet b87d5c4641304e03a8a3fd4782b043e5 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:57710)
I20260709 13:18:17.465335 15687 tablet_copy_service.cc:161] P 3ef186ec12554a1d87814af0a0ace9a4: Beginning new tablet copy session on tablet b87d5c4641304e03a8a3fd4782b043e5 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:57710: session id = dd1a14c3173d49dc9b55bb116e3783c9-b87d5c4641304e03a8a3fd4782b043e5
I20260709 13:18:17.478582 15687 tablet_copy_source_session.cc:215] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:17.481621 16437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b87d5c4641304e03a8a3fd4782b043e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:17.499017 16437 tablet_copy_client.cc:806] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:17.499689 16437 tablet_copy_client.cc:670] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:17.503575 16437 tablet_copy_client.cc:538] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:17.510954 16437 tablet_bootstrap.cc:492] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:17.529846 16437 tablet_bootstrap.cc:492] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:17.530889 16437 tablet_bootstrap.cc:492] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:17.531601 16437 ts_tablet_manager.cc:1403] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.008s
I20260709 13:18:17.534693 16437 raft_consensus.cc:359] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:17.535533 16437 raft_consensus.cc:740] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:17.536219 16437 consensus_queue.cc:260] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:17.539343 16437 ts_tablet_manager.cc:1434] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:18:17.541777 15687 tablet_copy_service.cc:342] P 3ef186ec12554a1d87814af0a0ace9a4: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-b87d5c4641304e03a8a3fd4782b043e5 received from {username='slave'} at 127.0.0.1:57710
I20260709 13:18:17.542289 15687 tablet_copy_service.cc:434] P 3ef186ec12554a1d87814af0a0ace9a4: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-b87d5c4641304e03a8a3fd4782b043e5 on tablet b87d5c4641304e03a8a3fd4782b043e5 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:17.577224 16437 ts_tablet_manager.cc:933] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:18:17.579097 16437 tablet_copy_client.cc:323] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.65:42481
I20260709 13:18:17.580734 15538 tablet_copy_service.cc:140] P d3c4f5bd76e0445795bad5f6dd93ff69: Received BeginTabletCopySession request for tablet 01ad9d7b8dba45be8fdf625f27e8d273 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:56840)
I20260709 13:18:17.581190 15538 tablet_copy_service.cc:161] P d3c4f5bd76e0445795bad5f6dd93ff69: Beginning new tablet copy session on tablet 01ad9d7b8dba45be8fdf625f27e8d273 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:56840: session id = dd1a14c3173d49dc9b55bb116e3783c9-01ad9d7b8dba45be8fdf625f27e8d273
I20260709 13:18:17.586915 15538 tablet_copy_source_session.cc:215] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:17.592684 16437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01ad9d7b8dba45be8fdf625f27e8d273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:17.595211 16442 ts_tablet_manager.cc:933] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:18:17.596992 16442 tablet_copy_client.cc:323] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.65:42481
I20260709 13:18:17.598392 15538 tablet_copy_service.cc:140] P d3c4f5bd76e0445795bad5f6dd93ff69: Received BeginTabletCopySession request for tablet 15e5bd79113c48c5a0b0f265bdd1d7b2 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:56840)
I20260709 13:18:17.598814 15538 tablet_copy_service.cc:161] P d3c4f5bd76e0445795bad5f6dd93ff69: Beginning new tablet copy session on tablet 15e5bd79113c48c5a0b0f265bdd1d7b2 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:56840: session id = dd1a14c3173d49dc9b55bb116e3783c9-15e5bd79113c48c5a0b0f265bdd1d7b2
I20260709 13:18:17.604856 15538 tablet_copy_source_session.cc:215] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:17.607311 16442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15e5bd79113c48c5a0b0f265bdd1d7b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:17.612880 16437 tablet_copy_client.cc:806] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:17.613587 16437 tablet_copy_client.cc:670] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:17.617888 16437 tablet_copy_client.cc:538] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:17.625618 16442 tablet_copy_client.cc:806] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:17.626266 16442 tablet_copy_client.cc:670] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:17.627439 16437 tablet_bootstrap.cc:492] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:17.631404 16442 tablet_copy_client.cc:538] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:17.646311 16442 tablet_bootstrap.cc:492] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:17.666831 16437 tablet_bootstrap.cc:492] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:17.667847 16437 tablet_bootstrap.cc:492] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:17.671872 16445 ts_tablet_manager.cc:933] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:17.672864 16437 ts_tablet_manager.cc:1403] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.046s	user 0.022s	sys 0.007s
I20260709 13:18:17.677163 16442 tablet_bootstrap.cc:492] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:17.678244 16442 tablet_bootstrap.cc:492] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:17.677634 16437 raft_consensus.cc:359] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:17.679136 16442 ts_tablet_manager.cc:1403] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.015s
I20260709 13:18:17.679244 16437 raft_consensus.cc:740] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:17.681216 16437 consensus_queue.cc:260] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:17.682413 16442 raft_consensus.cc:359] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:17.683363 16442 raft_consensus.cc:740] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:17.685453 16437 ts_tablet_manager.cc:1434] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:18:17.721000 15538 tablet_copy_service.cc:342] P d3c4f5bd76e0445795bad5f6dd93ff69: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-01ad9d7b8dba45be8fdf625f27e8d273 received from {username='slave'} at 127.0.0.1:56840
I20260709 13:18:17.723390 15538 tablet_copy_service.cc:434] P d3c4f5bd76e0445795bad5f6dd93ff69: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-01ad9d7b8dba45be8fdf625f27e8d273 on tablet 01ad9d7b8dba45be8fdf625f27e8d273 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:17.728797 16445 tablet_copy_client.cc:323] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.66:41593
I20260709 13:18:17.721134 16442 consensus_queue.cc:260] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:17.731755 15613 tablet_copy_service.cc:140] P 5498ef3e89a941e39c25f68cf64e6b0f: Received BeginTabletCopySession request for tablet f9328896aa5a4fdc8fa7c908efdae695 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:34102)
I20260709 13:18:17.732313 15613 tablet_copy_service.cc:161] P 5498ef3e89a941e39c25f68cf64e6b0f: Beginning new tablet copy session on tablet f9328896aa5a4fdc8fa7c908efdae695 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:34102: session id = dd1a14c3173d49dc9b55bb116e3783c9-f9328896aa5a4fdc8fa7c908efdae695
I20260709 13:18:17.740381 16442 ts_tablet_manager.cc:1434] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.000s
I20260709 13:18:17.741513 15613 tablet_copy_source_session.cc:215] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:17.745752 15538 tablet_copy_service.cc:342] P d3c4f5bd76e0445795bad5f6dd93ff69: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-15e5bd79113c48c5a0b0f265bdd1d7b2 received from {username='slave'} at 127.0.0.1:56840
I20260709 13:18:17.746235 15538 tablet_copy_service.cc:434] P d3c4f5bd76e0445795bad5f6dd93ff69: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-15e5bd79113c48c5a0b0f265bdd1d7b2 on tablet 15e5bd79113c48c5a0b0f265bdd1d7b2 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:17.748860 16445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9328896aa5a4fdc8fa7c908efdae695. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:17.766445 16445 tablet_copy_client.cc:806] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:17.767015 16445 tablet_copy_client.cc:670] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:17.771653 16445 tablet_copy_client.cc:538] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:17.780889 16445 tablet_bootstrap.cc:492] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:17.801455 16445 tablet_bootstrap.cc:492] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:17.802455 16445 tablet_bootstrap.cc:492] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:17.803139 16445 ts_tablet_manager.cc:1403] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.022s	user 0.023s	sys 0.000s
I20260709 13:18:17.806226 16445 raft_consensus.cc:359] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:17.807076 16445 raft_consensus.cc:740] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:17.807724 16445 consensus_queue.cc:260] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:17.810235 16445 ts_tablet_manager.cc:1434] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:18:17.812378 15613 tablet_copy_service.cc:342] P 5498ef3e89a941e39c25f68cf64e6b0f: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-f9328896aa5a4fdc8fa7c908efdae695 received from {username='slave'} at 127.0.0.1:34102
I20260709 13:18:17.812830 15613 tablet_copy_service.cc:434] P 5498ef3e89a941e39c25f68cf64e6b0f: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-f9328896aa5a4fdc8fa7c908efdae695 on tablet f9328896aa5a4fdc8fa7c908efdae695 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:17.999855 16240 raft_consensus.cc:1217] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:18.056524 16240 raft_consensus.cc:1217] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:18.108398 16240 raft_consensus.cc:1217] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:18.126194 16240 raft_consensus.cc:1217] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:18.516256 16329 raft_consensus.cc:1064] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:18.518924 16329 consensus_queue.cc:237] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:18.563779 15595 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:18.563872 16240 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:18.567792 16453 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:18.568797 16418 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:18.570562 16454 raft_consensus.cc:1064] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:18.573478 16454 consensus_queue.cc:237] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:18.591925 16239 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:18.594568 16454 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:18.595535 15602 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:18.596099 15677 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:18.598913 15519 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:18.599788 16397 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:18.601450 16418 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:18.610826 16329 raft_consensus.cc:1064] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:18.610687 16454 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:18.613212 16329 consensus_queue.cc:237] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:18.631448 16237 raft_consensus.cc:1275] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:18.631937 15602 raft_consensus.cc:1275] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:18.633488 16448 consensus_queue.cc:1048] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:18.634979 16455 consensus_queue.cc:1048] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:18.640025 15669 raft_consensus.cc:1275] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:18.643570 16397 raft_consensus.cc:2955] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.645509 15520 raft_consensus.cc:2955] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.650749 16456 consensus_queue.cc:1048] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:18.662637 16448 raft_consensus.cc:2955] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.671614 16238 raft_consensus.cc:2955] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.673347 15425 catalog_manager.cc:5697] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.686244 15602 raft_consensus.cc:2955] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.691906 15595 raft_consensus.cc:2955] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.694766 15424 catalog_manager.cc:5697] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.696770 16336 raft_consensus.cc:1064] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:18.698946 16336 consensus_queue.cc:237] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:18.705242 15675 raft_consensus.cc:2955] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.714413 16453 raft_consensus.cc:2955] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.721547 15675 raft_consensus.cc:2955] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.722668 16239 raft_consensus.cc:2955] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.724838 15602 raft_consensus.cc:2955] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.743322 16238 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:18.743634 15677 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:18.745390 16451 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:18.747686 16336 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:18.752566 15520 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:18.752592 16239 raft_consensus.cc:2955] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.754439 16336 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:18.784698 16451 raft_consensus.cc:2955] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.806744 16239 raft_consensus.cc:2955] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.828830 15675 consensus_queue.cc:237] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:18.844867 16238 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:18.846501 16397 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:18.846263 15424 catalog_manager.cc:5697] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.854739 16454 raft_consensus.cc:2955] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.856249 16238 raft_consensus.cc:2955] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.868054 15519 raft_consensus.cc:2955] T f9328896aa5a4fdc8fa7c908efdae695 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.882033 15675 raft_consensus.cc:2955] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.904114 15519 consensus_queue.cc:237] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:18.904917 15423 catalog_manager.cc:5697] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:18.917079 15520 consensus_queue.cc:237] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:18.919284 15410 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b87d5c4641304e03a8a3fd4782b043e5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:18.920614 15602 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:18.928213 16238 raft_consensus.cc:1275] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:18.929381 15602 raft_consensus.cc:1275] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:18.932274 15425 catalog_manager.cc:5697] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.933558 16456 consensus_queue.cc:1048] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:18.936465 16453 consensus_queue.cc:1048] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:18.946641 16420 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:18.947373 16239 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:18.948022 16453 raft_consensus.cc:2955] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.952410 15675 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:18.953816 16329 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:18.956939 15514 tablet_service.cc:1558] Processing DeleteTablet for tablet b87d5c4641304e03a8a3fd4782b043e5 with delete_type TABLET_DATA_TOMBSTONED (TS d3c4f5bd76e0445795bad5f6dd93ff69 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57292
I20260709 13:18:18.964162 15602 consensus_queue.cc:237] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:18.965325 15595 raft_consensus.cc:2955] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.967751 16239 raft_consensus.cc:2955] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.974406 15677 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:18.975381 16329 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:18.977316 16238 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:18.980840 15411 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 15e5bd79113c48c5a0b0f265bdd1d7b2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:18.980279 16478 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:18:18.982671 16484 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:18.985639 15602 raft_consensus.cc:2955] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.994356 16418 raft_consensus.cc:2955] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.996052 16481 raft_consensus.cc:2955] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:18.994488 16487 tablet_replica.cc:333] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:18:19.002036 16487 raft_consensus.cc:2243] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:19.002771 16487 raft_consensus.cc:2272] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:19.002844 16238 raft_consensus.cc:2955] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:19.006943 16487 ts_tablet_manager.cc:1916] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:19.020568 15677 raft_consensus.cc:2955] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:19.027222 15425 catalog_manager.cc:5697] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:19.032076 15411 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 01ad9d7b8dba45be8fdf625f27e8d273 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:19.036219 15412 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f9328896aa5a4fdc8fa7c908efdae695 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:19.050559 15677 raft_consensus.cc:2955] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:19.066864 15426 catalog_manager.cc:5697] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:19.067485 15425 catalog_manager.cc:5697] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:19.078421 16238 raft_consensus.cc:2955] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:19.080183 15514 tablet_service.cc:1558] Processing DeleteTablet for tablet f9328896aa5a4fdc8fa7c908efdae695 with delete_type TABLET_DATA_TOMBSTONED (TS d3c4f5bd76e0445795bad5f6dd93ff69 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57292
I20260709 13:18:19.084357 15589 tablet_service.cc:1558] Processing DeleteTablet for tablet 01ad9d7b8dba45be8fdf625f27e8d273 with delete_type TABLET_DATA_TOMBSTONED (TS 5498ef3e89a941e39c25f68cf64e6b0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53300
I20260709 13:18:19.089859 16487 ts_tablet_manager.cc:1929] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:19.091643 16487 log.cc:1199] T b87d5c4641304e03a8a3fd4782b043e5 P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-0-root/wals/b87d5c4641304e03a8a3fd4782b043e5
I20260709 13:18:19.097283 15411 catalog_manager.cc:5028] TS d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): tablet b87d5c4641304e03a8a3fd4782b043e5 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:19.098559 16487 tablet_replica.cc:333] T f9328896aa5a4fdc8fa7c908efdae695 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:18:19.134644 16488 tablet_replica.cc:333] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:18:19.138409 15660 tablet_service.cc:1558] Processing DeleteTablet for tablet 15e5bd79113c48c5a0b0f265bdd1d7b2 with delete_type TABLET_DATA_TOMBSTONED (TS 3ef186ec12554a1d87814af0a0ace9a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49088
I20260709 13:18:19.140933 16487 raft_consensus.cc:2243] T f9328896aa5a4fdc8fa7c908efdae695 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:19.142935 16488 raft_consensus.cc:2243] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:19.143674 16487 raft_consensus.cc:2272] T f9328896aa5a4fdc8fa7c908efdae695 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:19.143889 16488 raft_consensus.cc:2272] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:19.146312 16490 tablet_replica.cc:333] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:18:19.147527 16490 raft_consensus.cc:2243] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:19.147724 16488 ts_tablet_manager.cc:1916] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:19.148375 16490 raft_consensus.cc:2272] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:19.149132 16487 ts_tablet_manager.cc:1916] T f9328896aa5a4fdc8fa7c908efdae695 P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:19.152920 16490 ts_tablet_manager.cc:1916] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:19.163384 16487 ts_tablet_manager.cc:1929] T f9328896aa5a4fdc8fa7c908efdae695 P d3c4f5bd76e0445795bad5f6dd93ff69: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:19.163789 16487 log.cc:1199] T f9328896aa5a4fdc8fa7c908efdae695 P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-0-root/wals/f9328896aa5a4fdc8fa7c908efdae695
I20260709 13:18:19.164206 16488 ts_tablet_manager.cc:1929] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:19.164673 16488 log.cc:1199] T 01ad9d7b8dba45be8fdf625f27e8d273 P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-1-root/wals/01ad9d7b8dba45be8fdf625f27e8d273
I20260709 13:18:19.165223 15411 catalog_manager.cc:5028] TS d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): tablet f9328896aa5a4fdc8fa7c908efdae695 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:19.166472 15412 catalog_manager.cc:5028] TS 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): tablet 01ad9d7b8dba45be8fdf625f27e8d273 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:19.168573 16490 ts_tablet_manager.cc:1929] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:19.168918 16490 log.cc:1199] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 3ef186ec12554a1d87814af0a0ace9a4: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-2-root/wals/15e5bd79113c48c5a0b0f265bdd1d7b2
I20260709 13:18:19.170270 15410 catalog_manager.cc:5028] TS 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): tablet 15e5bd79113c48c5a0b0f265bdd1d7b2 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:20.150221 15603 consensus_queue.cc:237] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:20.160648 15670 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:18:20.160737 15568 consensus_peers.cc:597] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 061ed80a3fca4096b7c13dc0e2256ef0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:20.161940 15519 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:20.162082 16485 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:20.163452 16491 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:20.171697 16491 raft_consensus.cc:2955] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:20.174237 15520 raft_consensus.cc:2955] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:20.176026 15670 raft_consensus.cc:2955] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:20.190267 15519 consensus_queue.cc:237] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:20.196617 15426 catalog_manager.cc:5697] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:20.204862 15493 consensus_peers.cc:597] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4c48036212bb4277acf36aac79d8b35e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:20.205808 15603 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:20.216445 15670 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:20.209439 16456 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:20.222477 16453 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:20.239452 16499 raft_consensus.cc:2955] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:20.247727 15603 raft_consensus.cc:2955] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:20.250882 15675 raft_consensus.cc:2955] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:20.260619 15602 consensus_queue.cc:237] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:20.268841 15425 catalog_manager.cc:5697] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:20.275297 15568 consensus_peers.cc:597] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7b7542687ad43acb128a39bb1f40829. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:20.277491 15675 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:20.278395 15520 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:20.279466 16491 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:20.280328 16481 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:20.290024 16504 raft_consensus.cc:2955] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:20.293113 15520 raft_consensus.cc:2955] T d7b7542687ad43acb128a39bb1f40829 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:20.308040 15519 consensus_queue.cc:237] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:20.310096 15426 catalog_manager.cc:5697] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:20.325300 15670 raft_consensus.cc:1275] T 271b5d375f0145b78b28a0c13328ce7f P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:18:20.327500 15493 consensus_peers.cc:597] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 271b5d375f0145b78b28a0c13328ce7f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:20.328837 15603 raft_consensus.cc:1275] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:20.320973 15675 raft_consensus.cc:2955] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:20.330624 16418 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:20.331795 16499 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:20.346572 16453 raft_consensus.cc:2955] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:20.348268 15603 raft_consensus.cc:2955] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:20.355489 15670 raft_consensus.cc:2955] T 271b5d375f0145b78b28a0c13328ce7f P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:20.372941 15425 catalog_manager.cc:5697] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:20.608552 16510 ts_tablet_manager.cc:933] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:20.610639 16510 tablet_copy_client.cc:323] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.66:41593
I20260709 13:18:20.612085 15613 tablet_copy_service.cc:140] P 5498ef3e89a941e39c25f68cf64e6b0f: Received BeginTabletCopySession request for tablet 061ed80a3fca4096b7c13dc0e2256ef0 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:34102)
I20260709 13:18:20.612548 15613 tablet_copy_service.cc:161] P 5498ef3e89a941e39c25f68cf64e6b0f: Beginning new tablet copy session on tablet 061ed80a3fca4096b7c13dc0e2256ef0 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:34102: session id = dd1a14c3173d49dc9b55bb116e3783c9-061ed80a3fca4096b7c13dc0e2256ef0
I20260709 13:18:20.618839 15613 tablet_copy_source_session.cc:215] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:20.621694 16510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 061ed80a3fca4096b7c13dc0e2256ef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:20.636698 16510 tablet_copy_client.cc:806] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:20.637269 16510 tablet_copy_client.cc:670] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:20.640503 16510 tablet_copy_client.cc:538] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:20.646683 16510 tablet_bootstrap.cc:492] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:20.663820 16510 tablet_bootstrap.cc:492] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:20.664729 16510 tablet_bootstrap.cc:492] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:20.665306 16510 ts_tablet_manager.cc:1403] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.006s
I20260709 13:18:20.667873 16510 raft_consensus.cc:359] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:20.668744 16510 raft_consensus.cc:740] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:20.669404 16510 consensus_queue.cc:260] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:20.673727 16510 ts_tablet_manager.cc:1434] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 13:18:20.675863 15613 tablet_copy_service.cc:342] P 5498ef3e89a941e39c25f68cf64e6b0f: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-061ed80a3fca4096b7c13dc0e2256ef0 received from {username='slave'} at 127.0.0.1:34102
I20260709 13:18:20.676308 15613 tablet_copy_service.cc:434] P 5498ef3e89a941e39c25f68cf64e6b0f: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-061ed80a3fca4096b7c13dc0e2256ef0 on tablet 061ed80a3fca4096b7c13dc0e2256ef0 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:20.709306 16510 ts_tablet_manager.cc:933] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:18:20.714809 16510 tablet_copy_client.cc:323] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.65:42481
I20260709 13:18:20.716516 15538 tablet_copy_service.cc:140] P d3c4f5bd76e0445795bad5f6dd93ff69: Received BeginTabletCopySession request for tablet 271b5d375f0145b78b28a0c13328ce7f from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:56840)
I20260709 13:18:20.716981 15538 tablet_copy_service.cc:161] P d3c4f5bd76e0445795bad5f6dd93ff69: Beginning new tablet copy session on tablet 271b5d375f0145b78b28a0c13328ce7f from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:56840: session id = dd1a14c3173d49dc9b55bb116e3783c9-271b5d375f0145b78b28a0c13328ce7f
I20260709 13:18:20.723032 15538 tablet_copy_source_session.cc:215] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:20.725750 16510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 271b5d375f0145b78b28a0c13328ce7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:20.726788 16515 ts_tablet_manager.cc:933] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:20.728832 16515 tablet_copy_client.cc:323] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.66:41593
I20260709 13:18:20.730571 15613 tablet_copy_service.cc:140] P 5498ef3e89a941e39c25f68cf64e6b0f: Received BeginTabletCopySession request for tablet d7b7542687ad43acb128a39bb1f40829 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:34102)
I20260709 13:18:20.731118 15613 tablet_copy_service.cc:161] P 5498ef3e89a941e39c25f68cf64e6b0f: Beginning new tablet copy session on tablet d7b7542687ad43acb128a39bb1f40829 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:34102: session id = dd1a14c3173d49dc9b55bb116e3783c9-d7b7542687ad43acb128a39bb1f40829
I20260709 13:18:20.744617 16510 tablet_copy_client.cc:806] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:20.745393 16510 tablet_copy_client.cc:670] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:20.749079 16510 tablet_copy_client.cc:538] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:20.755178 15613 tablet_copy_source_session.cc:215] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:20.757980 16510 tablet_bootstrap.cc:492] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:20.758957 16515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7b7542687ad43acb128a39bb1f40829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:20.779135 16515 tablet_copy_client.cc:806] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:20.779809 16515 tablet_copy_client.cc:670] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:20.785902 16515 tablet_copy_client.cc:538] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:20.786640 16510 tablet_bootstrap.cc:492] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:20.787627 16510 tablet_bootstrap.cc:492] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:20.788383 16510 ts_tablet_manager.cc:1403] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.031s	user 0.017s	sys 0.012s
I20260709 13:18:20.793560 16510 raft_consensus.cc:359] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:20.794549 16510 raft_consensus.cc:740] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:20.795285 16510 consensus_queue.cc:260] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:20.797856 16510 ts_tablet_manager.cc:1434] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:18:20.794237 16517 ts_tablet_manager.cc:933] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:18:20.799701 15538 tablet_copy_service.cc:342] P d3c4f5bd76e0445795bad5f6dd93ff69: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-271b5d375f0145b78b28a0c13328ce7f received from {username='slave'} at 127.0.0.1:56840
I20260709 13:18:20.800123 15538 tablet_copy_service.cc:434] P d3c4f5bd76e0445795bad5f6dd93ff69: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-271b5d375f0145b78b28a0c13328ce7f on tablet 271b5d375f0145b78b28a0c13328ce7f with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:20.801949 16515 tablet_bootstrap.cc:492] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:20.818843 16517 tablet_copy_client.cc:323] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.65:42481
I20260709 13:18:20.821736 15538 tablet_copy_service.cc:140] P d3c4f5bd76e0445795bad5f6dd93ff69: Received BeginTabletCopySession request for tablet 4c48036212bb4277acf36aac79d8b35e from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:56840)
I20260709 13:18:20.822263 15538 tablet_copy_service.cc:161] P d3c4f5bd76e0445795bad5f6dd93ff69: Beginning new tablet copy session on tablet 4c48036212bb4277acf36aac79d8b35e from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:56840: session id = dd1a14c3173d49dc9b55bb116e3783c9-4c48036212bb4277acf36aac79d8b35e
I20260709 13:18:20.828181 16515 tablet_bootstrap.cc:492] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:20.829154 16515 tablet_bootstrap.cc:492] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:20.829860 16515 ts_tablet_manager.cc:1403] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.028s	user 0.016s	sys 0.003s
I20260709 13:18:20.830888 15538 tablet_copy_source_session.cc:215] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:20.832929 16515 raft_consensus.cc:359] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:20.834343 16515 raft_consensus.cc:740] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:20.834693 16517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c48036212bb4277acf36aac79d8b35e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:20.834962 16515 consensus_queue.cc:260] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:20.837499 16515 ts_tablet_manager.cc:1434] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:18:20.839766 15613 tablet_copy_service.cc:342] P 5498ef3e89a941e39c25f68cf64e6b0f: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-d7b7542687ad43acb128a39bb1f40829 received from {username='slave'} at 127.0.0.1:34102
I20260709 13:18:20.840163 15613 tablet_copy_service.cc:434] P 5498ef3e89a941e39c25f68cf64e6b0f: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-d7b7542687ad43acb128a39bb1f40829 on tablet d7b7542687ad43acb128a39bb1f40829 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:20.851114 16517 tablet_copy_client.cc:806] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:20.851687 16517 tablet_copy_client.cc:670] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:20.856606 16517 tablet_copy_client.cc:538] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:20.866715 16517 tablet_bootstrap.cc:492] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:20.887583 16517 tablet_bootstrap.cc:492] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:20.888440 16517 tablet_bootstrap.cc:492] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:20.889058 16517 ts_tablet_manager.cc:1403] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.005s
I20260709 13:18:20.891472 16517 raft_consensus.cc:359] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:20.892055 16517 raft_consensus.cc:740] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:20.892539 16517 consensus_queue.cc:260] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:20.894291 16517 ts_tablet_manager.cc:1434] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:18:20.896027 15538 tablet_copy_service.cc:342] P d3c4f5bd76e0445795bad5f6dd93ff69: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-4c48036212bb4277acf36aac79d8b35e received from {username='slave'} at 127.0.0.1:56840
I20260709 13:18:20.896426 15538 tablet_copy_service.cc:434] P d3c4f5bd76e0445795bad5f6dd93ff69: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-4c48036212bb4277acf36aac79d8b35e on tablet 4c48036212bb4277acf36aac79d8b35e with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:21.126464 16238 raft_consensus.cc:1217] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:21.227689 16238 raft_consensus.cc:1217] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:21.239259 16238 raft_consensus.cc:1217] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:21.257754 16238 raft_consensus.cc:1217] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:21.606906 16481 raft_consensus.cc:1064] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:21.609225 16481 consensus_queue.cc:237] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:21.615381 15670 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:21.615696 16239 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:21.617090 15519 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:21.617031 16481 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:21.618011 16491 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:21.620543 16491 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:21.638392 16524 raft_consensus.cc:2955] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:21.640103 15670 raft_consensus.cc:2955] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:21.641418 15519 raft_consensus.cc:2955] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:21.655154 16239 raft_consensus.cc:2955] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:21.663545 15422 catalog_manager.cc:5697] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:21.676088 16418 raft_consensus.cc:1064] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:21.680531 16418 consensus_queue.cc:237] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:21.689260 16238 raft_consensus.cc:1275] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:21.690577 15595 raft_consensus.cc:1275] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:21.692214 16418 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:21.694898 16523 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:21.699338 15669 raft_consensus.cc:1275] T 271b5d375f0145b78b28a0c13328ce7f P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:21.707091 16523 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:21.715903 16523 raft_consensus.cc:2955] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:21.721326 15595 raft_consensus.cc:2955] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:21.723770 15669 raft_consensus.cc:2955] T 271b5d375f0145b78b28a0c13328ce7f P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:21.731945 16238 raft_consensus.cc:2955] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:21.746558 16523 raft_consensus.cc:1064] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:21.750067 16523 consensus_queue.cc:237] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:21.756585 15595 consensus_queue.cc:237] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:21.761484 16239 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:21.763473 16491 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:21.767546 15422 catalog_manager.cc:5697] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:21.771354 16491 raft_consensus.cc:2955] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:21.773029 16239 raft_consensus.cc:2955] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:21.788484 15519 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:21.790153 16481 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:21.799580 15519 raft_consensus.cc:2955] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:21.809192 16524 raft_consensus.cc:1064] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:21.812630 15412 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 061ed80a3fca4096b7c13dc0e2256ef0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:21.811625 16524 consensus_queue.cc:237] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:21.826813 15424 catalog_manager.cc:5697] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:21.827777 15520 consensus_queue.cc:237] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:21.831590 15669 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:21.832064 15523 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:21.833485 16238 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:21.841437 16481 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:21.843073 16539 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:21.845011 16539 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:21.847937 16238 raft_consensus.cc:1275] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:21.850894 16418 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:21.846289 15595 raft_consensus.cc:1275] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:21.861488 16499 raft_consensus.cc:2955] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:21.867769 15660 tablet_service.cc:1558] Processing DeleteTablet for tablet 061ed80a3fca4096b7c13dc0e2256ef0 with delete_type TABLET_DATA_TOMBSTONED (TS 3ef186ec12554a1d87814af0a0ace9a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49088
I20260709 13:18:21.872901 16418 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:21.878562 16240 raft_consensus.cc:2955] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:21.882393 16543 tablet_replica.cc:333] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:18:21.893805 16238 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:21.895701 16453 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:21.901979 15595 raft_consensus.cc:2955] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:21.902691 15425 catalog_manager.cc:5697] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:21.912335 15675 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:21.912552 15602 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:21.916388 15660 tablet_service.cc:1558] Processing DeleteTablet for tablet 271b5d375f0145b78b28a0c13328ce7f with delete_type TABLET_DATA_TOMBSTONED (TS 3ef186ec12554a1d87814af0a0ace9a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49088
I20260709 13:18:21.918486 16538 raft_consensus.cc:2955] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:21.920274 16453 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:21.921818 15670 raft_consensus.cc:2955] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:21.926301 16418 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:21.927740 15411 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 271b5d375f0145b78b28a0c13328ce7f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:21.937475 16543 raft_consensus.cc:2243] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.938377 16543 raft_consensus.cc:2272] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.944027 16240 raft_consensus.cc:2955] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:21.945943 15520 raft_consensus.cc:2955] T d7b7542687ad43acb128a39bb1f40829 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:21.963014 16543 ts_tablet_manager.cc:1916] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:21.966975 15424 catalog_manager.cc:5697] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:21.999101 15603 consensus_queue.cc:237] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:22.002230 16545 raft_consensus.cc:2955] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:22.005836 15677 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:22.009181 15595 raft_consensus.cc:2955] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:22.010454 16240 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:22.013686 16538 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:22.014878 16543 ts_tablet_manager.cc:1929] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:22.015411 16543 log.cc:1199] T 061ed80a3fca4096b7c13dc0e2256ef0 P 3ef186ec12554a1d87814af0a0ace9a4: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-2-root/wals/061ed80a3fca4096b7c13dc0e2256ef0
I20260709 13:18:22.017577 15410 catalog_manager.cc:5028] TS 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): tablet 061ed80a3fca4096b7c13dc0e2256ef0 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:22.018635 16543 tablet_replica.cc:333] T 271b5d375f0145b78b28a0c13328ce7f P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:18:22.019104 16524 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:22.019776 16543 raft_consensus.cc:2243] T 271b5d375f0145b78b28a0c13328ce7f P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.020642 16543 raft_consensus.cc:2272] T 271b5d375f0145b78b28a0c13328ce7f P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.029354 16491 raft_consensus.cc:2955] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:22.040220 15677 raft_consensus.cc:2955] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:22.041209 16543 ts_tablet_manager.cc:1916] T 271b5d375f0145b78b28a0c13328ce7f P 3ef186ec12554a1d87814af0a0ace9a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:22.045600 16240 raft_consensus.cc:2955] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:22.040237 15425 catalog_manager.cc:5697] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:22.053956 15670 raft_consensus.cc:2955] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:22.058166 16238 raft_consensus.cc:2955] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:22.084952 16543 ts_tablet_manager.cc:1929] T 271b5d375f0145b78b28a0c13328ce7f P 3ef186ec12554a1d87814af0a0ace9a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:22.085433 16543 log.cc:1199] T 271b5d375f0145b78b28a0c13328ce7f P 3ef186ec12554a1d87814af0a0ace9a4: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-2-root/wals/271b5d375f0145b78b28a0c13328ce7f
I20260709 13:18:22.087085 15410 catalog_manager.cc:5028] TS 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): tablet 271b5d375f0145b78b28a0c13328ce7f (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:22.097806 15412 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d7b7542687ad43acb128a39bb1f40829 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:22.112906 15425 catalog_manager.cc:5697] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:22.115686 15520 consensus_queue.cc:237] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:22.125527 15514 tablet_service.cc:1558] Processing DeleteTablet for tablet d7b7542687ad43acb128a39bb1f40829 with delete_type TABLET_DATA_TOMBSTONED (TS d3c4f5bd76e0445795bad5f6dd93ff69 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57292
I20260709 13:18:22.129338 16550 tablet_replica.cc:333] T d7b7542687ad43acb128a39bb1f40829 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:18:22.133951 15669 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:22.134162 16550 raft_consensus.cc:2243] T d7b7542687ad43acb128a39bb1f40829 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.135133 16550 raft_consensus.cc:2272] T d7b7542687ad43acb128a39bb1f40829 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.135816 16545 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:22.138779 16240 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:22.147125 16499 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:22.148944 16550 ts_tablet_manager.cc:1916] T d7b7542687ad43acb128a39bb1f40829 P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:22.145722 16545 raft_consensus.cc:2955] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:22.149782 15669 raft_consensus.cc:2955] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:22.154413 16240 raft_consensus.cc:2955] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:22.173947 16550 ts_tablet_manager.cc:1929] T d7b7542687ad43acb128a39bb1f40829 P d3c4f5bd76e0445795bad5f6dd93ff69: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:22.174433 16550 log.cc:1199] T d7b7542687ad43acb128a39bb1f40829 P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-0-root/wals/d7b7542687ad43acb128a39bb1f40829
I20260709 13:18:22.175246 15411 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4c48036212bb4277acf36aac79d8b35e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:22.176103 15411 catalog_manager.cc:5028] TS d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): tablet d7b7542687ad43acb128a39bb1f40829 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:22.183852 15425 catalog_manager.cc:5697] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:22.197741 15589 tablet_service.cc:1558] Processing DeleteTablet for tablet 4c48036212bb4277acf36aac79d8b35e with delete_type TABLET_DATA_TOMBSTONED (TS 5498ef3e89a941e39c25f68cf64e6b0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53300
I20260709 13:18:22.204037 16551 tablet_replica.cc:333] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:18:22.205184 16551 raft_consensus.cc:2243] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.206038 16551 raft_consensus.cc:2272] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.208920 16551 ts_tablet_manager.cc:1916] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:22.219748 16551 ts_tablet_manager.cc:1929] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:22.220095 16551 log.cc:1199] T 4c48036212bb4277acf36aac79d8b35e P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-1-root/wals/4c48036212bb4277acf36aac79d8b35e
I20260709 13:18:22.221694 15412 catalog_manager.cc:5028] TS 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): tablet 4c48036212bb4277acf36aac79d8b35e (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:23.214569 16555 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:23.215324 16555 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:23.231087 16555 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:23.276149 16567 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:23.276878 16567 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.297528 16561 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:23.306640 16561 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.308930 16561 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:23.279255 16567 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:23.328274 16561 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:23.329110 16561 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.331049 16565 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:23.331772 16565 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.333114 16561 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:23.334654 16555 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:23.335233 16555 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:23.337680 16555 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:23.342718 16565 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:23.343542 16565 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:23.351970 16564 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:23.357313 15422 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49140 (request call id 190) took 1128 ms. Trace:
I20260709 13:18:23.357853 15422 rpcz_store.cc:276] 0709 13:18:22.228564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.229877 (+  1313us) service_pool.cc:224] Handling call
0709 13:18:23.357221 (+1127344us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8240,"reported_tablets":1}
I20260709 13:18:23.359660 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:23.357128 16564 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.363685 16564 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:23.359534 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:23.345666 16565 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.369107 16567 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:23.369880 16567 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.371711 16561 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:23.372115 16565 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:23.372474 16561 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:23.376364 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 69551.7us
I20260709 13:18:23.377769 16560 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:23.378304 16560 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.380566 16552 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:23.381062 16552 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.382782 16558 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:23.383390 16558 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.384008 16555 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:23.384505 16555 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:23.386508 16555 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:23.386518 16558 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:23.363739 16562 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:23.388013 16562 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.390259 16562 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:23.355476 16568 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:23.391114 16560 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:23.391013 16568 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.395856 16568 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:23.354303 16566 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:23.391615 16552 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:23.397022 16552 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:23.397028 16566 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.397840 16552 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.399587 16566 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:23.399904 16552 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:23.401618 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:23.402371 15670 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 1.
I20260709 13:18:23.403611 16203 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:23.404603 16555 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:23.404991 16555 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:23.405325 16555 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:23.412778 16555 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:23.414652 16561 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:23.415930 16567 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:23.421447 16555 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:23.422951 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:23.423647 15677 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:23.429373 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:23.430935 15677 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:23.431123 16561 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:23.432219 16203 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:23.432459 16561 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.433194 16555 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:23.437577 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:23.442418 16555 raft_consensus.cc:697] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:23.443120 16562 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:23.443792 16555 consensus_queue.cc:237] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:23.444746 16561 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:23.443878 16562 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.448493 16562 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:23.449870 16552 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:23.450587 16552 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.452718 16552 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:23.450532 16572 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:23.456660 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:23.457847 16562 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:23.466570 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:23.461390 15422 catalog_manager.cc:5697] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 1 to 2, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 2 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:23.467252 16572 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.469305 16562 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:23.470009 16572 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:23.471225 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:23.471843 15519 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:23.471904 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:23.472149 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:23.473140 16562 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:23.464394 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:23.466401 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:23.473543 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:23.475848 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:23.475754 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:23.477411 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:23.478240 16552 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:23.478701 16552 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.480656 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:23.480278 16552 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:23.483861 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:23.483913 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:23.485555 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:23.486598 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:23.487574 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:23.488197 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:23.488952 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:23.489074 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:23.489675 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:23.490146 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:23.492237 16238 raft_consensus.cc:1240] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:23.493324 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:23.494211 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:23.494496 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:23.499162 16562 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:23.500316 16562 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.499733 16567 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:23.501240 16204 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:23.501207 16567 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.502604 16204 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:23.503444 16567 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:23.503881 16576 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.504209 16555 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.505374 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:23.506124 16562 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:23.506598 15567 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:23.507421 16524 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:23.507758 16552 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:23.508339 16552 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.510455 16552 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:23.523130 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:23.523245 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:23.523792 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:23.527577 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:23.530382 15670 consensus_queue.cc:237] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:23.535756 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:23.536036 16552 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:23.536602 16552 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.537547 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:23.538709 16552 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:23.539492 15669 raft_consensus.cc:1240] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:23.540215 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:23.542352 15567 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:23.543084 16575 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.544014 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:23.545699 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:23.548775 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:23.550791 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:23.550923 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:23.551944 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:23.552325 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:23.552512 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:23.553120 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:23.554988 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:23.556910 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:23.558200 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:23.559463 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:23.560629 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:23.561842 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:23.563035 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:23.570887 16559 consensus_queue.cc:1059] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:23.575573 15595 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:18:23.575848 15642 consensus_peers.cc:597] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2a5f6755312b4dc09781ecf8a9468c17. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:23.575410 15567 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:23.577015 16575 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:23.577189 16567 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:23.577404 16575 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:23.577376 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:23.578001 16575 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:23.578256 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:23.579905 16552 consensus_queue.cc:1059] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:23.580720 16552 consensus_queue.cc:1243] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer dd1a14c3173d49dc9b55bb116e3783c9 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:18:23.582317 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:23.579299 15567 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:23.584110 16556 raft_consensus.cc:3055] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:23.584384 16558 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.585755 15567 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:23.587373 15493 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:23.587571 15641 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:23.588165 16552 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.588600 15493 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:23.588896 16558 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.589254 16572 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.585532 16575 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.589720 16552 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.590583 15493 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:23.591218 15567 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:23.590095 15641 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:23.592861 16573 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.593293 15493 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:23.592818 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:23.593796 15641 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:23.590186 16556 raft_consensus.cc:740] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:23.594200 16573 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.595232 15493 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:23.595000 16556 consensus_queue.cc:260] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:23.598124 16552 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.598392 16556 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:23.598534 15493 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:23.598829 16558 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.599498 16552 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.599686 15493 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:23.600394 16552 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.592437 16575 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:23.596145 16562 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.596596 16567 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.597466 15641 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:23.601802 15526 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:23.601284 15567 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:23.601348 16571 raft_consensus.cc:2955] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:23.603621 16562 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.604545 16575 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.603747 15641 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:23.606590 16567 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.607972 15641 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:23.608670 16562 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.609285 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:23.609706 15641 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:23.610428 16562 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.610541 15567 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:23.611578 16575 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.611789 15641 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:23.612623 16562 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.611953 15603 raft_consensus.cc:2955] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:23.612969 15641 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:23.613656 16562 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.614058 15641 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:23.614749 16562 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.615087 15641 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:23.615758 16562 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:23.617013 16562 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:23.622330 15422 catalog_manager.cc:5697] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:23.627652 15670 consensus_queue.cc:237] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:23.633193 15526 raft_consensus.cc:2955] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:23.636474 15603 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:18:23.639215 15642 consensus_peers.cc:597] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ab68091b31d248a6ac99ca234a252f77. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:23.639562 15522 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:23.639734 16562 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:23.641562 16562 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:23.655239 16570 raft_consensus.cc:2955] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:23.658082 15603 raft_consensus.cc:2955] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:23.679610 15426 catalog_manager.cc:5697] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:23.681931 15595 consensus_queue.cc:237] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:23.689249 15526 raft_consensus.cc:2955] T ab68091b31d248a6ac99ca234a252f77 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:23.715163 15670 raft_consensus.cc:1275] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:18:23.715755 15568 consensus_peers.cc:597] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f25e63ed82fa4e008109b74fe8df71ba. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:23.716886 16575 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:23.717242 15526 raft_consensus.cc:1275] T f25e63ed82fa4e008109b74fe8df71ba P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:23.718703 16558 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:23.729386 16575 raft_consensus.cc:2955] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:23.735477 15526 raft_consensus.cc:2955] T f25e63ed82fa4e008109b74fe8df71ba P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:23.732034 15670 raft_consensus.cc:2955] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:23.743458 15595 raft_consensus.cc:1240] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:23.754294 16571 consensus_queue.cc:1059] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:23.755307 16571 raft_consensus.cc:3055] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:23.755682 16571 raft_consensus.cc:740] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:23.756445 16571 consensus_queue.cc:260] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:23.757692 16571 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:23.757787 15422 catalog_manager.cc:5697] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:23.777010 15526 consensus_queue.cc:237] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
W20260709 13:18:23.808037 15493 consensus_peers.cc:597] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 83022a40fade44a2988f3de3b6b36aec. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:23.815315 15669 raft_consensus.cc:1275] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:23.818430 16557 consensus_queue.cc:1048] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:23.819260 15595 raft_consensus.cc:1275] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:23.822369 16557 consensus_queue.cc:1048] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:23.827320 15526 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:23.828651 16555 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:23.832870 16552 raft_consensus.cc:2955] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:23.834775 15669 raft_consensus.cc:2955] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:23.847962 15595 raft_consensus.cc:2955] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:23.852809 15422 catalog_manager.cc:5697] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:23.865212 15669 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:23.866933 16555 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:23.980572 16599 ts_tablet_manager.cc:933] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:23.984552 16599 tablet_copy_client.cc:323] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.67:32779
I20260709 13:18:23.986120 15687 tablet_copy_service.cc:140] P 3ef186ec12554a1d87814af0a0ace9a4: Received BeginTabletCopySession request for tablet 2a5f6755312b4dc09781ecf8a9468c17 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:57710)
I20260709 13:18:23.986678 15687 tablet_copy_service.cc:161] P 3ef186ec12554a1d87814af0a0ace9a4: Beginning new tablet copy session on tablet 2a5f6755312b4dc09781ecf8a9468c17 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:57710: session id = dd1a14c3173d49dc9b55bb116e3783c9-2a5f6755312b4dc09781ecf8a9468c17
I20260709 13:18:23.994061 15687 tablet_copy_source_session.cc:215] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:23.997237 16599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a5f6755312b4dc09781ecf8a9468c17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:24.011615 16599 tablet_copy_client.cc:806] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:24.012187 16599 tablet_copy_client.cc:670] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:24.015669 16599 tablet_copy_client.cc:538] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:24.021809 16599 tablet_bootstrap.cc:492] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:24.038076 16599 tablet_bootstrap.cc:492] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:24.039029 16599 tablet_bootstrap.cc:492] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:24.039646 16599 ts_tablet_manager.cc:1403] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.018s	user 0.016s	sys 0.004s
I20260709 13:18:24.042136 16599 raft_consensus.cc:359] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:24.042712 16599 raft_consensus.cc:740] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:24.043223 16599 consensus_queue.cc:260] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:24.045419 16599 ts_tablet_manager.cc:1434] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.003s
I20260709 13:18:24.048570 15687 tablet_copy_service.cc:342] P 3ef186ec12554a1d87814af0a0ace9a4: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-2a5f6755312b4dc09781ecf8a9468c17 received from {username='slave'} at 127.0.0.1:57710
I20260709 13:18:24.049001 15687 tablet_copy_service.cc:434] P 3ef186ec12554a1d87814af0a0ace9a4: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-2a5f6755312b4dc09781ecf8a9468c17 on tablet 2a5f6755312b4dc09781ecf8a9468c17 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:24.104663 16599 ts_tablet_manager.cc:933] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:24.106299 16599 tablet_copy_client.cc:323] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.67:32779
I20260709 13:18:24.107754 15687 tablet_copy_service.cc:140] P 3ef186ec12554a1d87814af0a0ace9a4: Received BeginTabletCopySession request for tablet ab68091b31d248a6ac99ca234a252f77 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:57710)
I20260709 13:18:24.108181 15687 tablet_copy_service.cc:161] P 3ef186ec12554a1d87814af0a0ace9a4: Beginning new tablet copy session on tablet ab68091b31d248a6ac99ca234a252f77 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:57710: session id = dd1a14c3173d49dc9b55bb116e3783c9-ab68091b31d248a6ac99ca234a252f77
I20260709 13:18:24.114563 15687 tablet_copy_source_session.cc:215] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:24.117313 16599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab68091b31d248a6ac99ca234a252f77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:24.121405 16603 ts_tablet_manager.cc:933] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:24.127112 16603 tablet_copy_client.cc:323] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.66:41593
I20260709 13:18:24.128773 15613 tablet_copy_service.cc:140] P 5498ef3e89a941e39c25f68cf64e6b0f: Received BeginTabletCopySession request for tablet f25e63ed82fa4e008109b74fe8df71ba from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:34102)
I20260709 13:18:24.129276 15613 tablet_copy_service.cc:161] P 5498ef3e89a941e39c25f68cf64e6b0f: Beginning new tablet copy session on tablet f25e63ed82fa4e008109b74fe8df71ba from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:34102: session id = dd1a14c3173d49dc9b55bb116e3783c9-f25e63ed82fa4e008109b74fe8df71ba
I20260709 13:18:24.134091 16599 tablet_copy_client.cc:806] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:24.135008 16599 tablet_copy_client.cc:670] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:24.136806 15613 tablet_copy_source_session.cc:215] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:24.138378 16599 tablet_copy_client.cc:538] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:24.154241 16599 tablet_bootstrap.cc:492] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:24.156565 16603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f25e63ed82fa4e008109b74fe8df71ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:24.184024 16603 tablet_copy_client.cc:806] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:24.184667 16603 tablet_copy_client.cc:670] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:24.184736 16599 tablet_bootstrap.cc:492] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:24.187892 16599 tablet_bootstrap.cc:492] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:24.188617 16599 ts_tablet_manager.cc:1403] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.007s
I20260709 13:18:24.190222 16603 tablet_copy_client.cc:538] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:24.191721 16599 raft_consensus.cc:359] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:24.192962 16599 raft_consensus.cc:740] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:24.193672 16599 consensus_queue.cc:260] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:24.196132 16599 ts_tablet_manager.cc:1434] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:18:24.198120 15687 tablet_copy_service.cc:342] P 3ef186ec12554a1d87814af0a0ace9a4: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-ab68091b31d248a6ac99ca234a252f77 received from {username='slave'} at 127.0.0.1:57710
I20260709 13:18:24.198535 15687 tablet_copy_service.cc:434] P 3ef186ec12554a1d87814af0a0ace9a4: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-ab68091b31d248a6ac99ca234a252f77 on tablet ab68091b31d248a6ac99ca234a252f77 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:24.203238 16603 tablet_bootstrap.cc:492] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:24.223960 16603 tablet_bootstrap.cc:492] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:24.224988 16603 tablet_bootstrap.cc:492] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:24.225742 16603 ts_tablet_manager.cc:1403] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.004s
I20260709 13:18:24.228628 16603 raft_consensus.cc:359] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:24.229578 16603 raft_consensus.cc:740] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:24.231302 16603 consensus_queue.cc:260] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:24.233806 16603 ts_tablet_manager.cc:1434] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 13:18:24.235620 15613 tablet_copy_service.cc:342] P 5498ef3e89a941e39c25f68cf64e6b0f: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-f25e63ed82fa4e008109b74fe8df71ba received from {username='slave'} at 127.0.0.1:34102
I20260709 13:18:24.236012 15613 tablet_copy_service.cc:434] P 5498ef3e89a941e39c25f68cf64e6b0f: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-f25e63ed82fa4e008109b74fe8df71ba on tablet f25e63ed82fa4e008109b74fe8df71ba with peer dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:18:24.266366 15698 debug-util.cc:398] Leaking SignalData structure 0x7b0800137ec0 after lost signal to thread 14432
W20260709 13:18:24.267261 15698 debug-util.cc:398] Leaking SignalData structure 0x7b08000cee00 after lost signal to thread 15460
W20260709 13:18:24.268357 15698 debug-util.cc:398] Leaking SignalData structure 0x7b0800082e00 after lost signal to thread 15552
W20260709 13:18:24.269420 15698 debug-util.cc:398] Leaking SignalData structure 0x7b08002f68e0 after lost signal to thread 15627
W20260709 13:18:24.270475 15698 debug-util.cc:398] Leaking SignalData structure 0x7b0800190960 after lost signal to thread 15701
W20260709 13:18:24.271544 15698 debug-util.cc:398] Leaking SignalData structure 0x7b080032c200 after lost signal to thread 16266
I20260709 13:18:24.307703 16603 ts_tablet_manager.cc:933] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:18:24.347014 16603 tablet_copy_client.cc:323] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.65:42481
I20260709 13:18:24.349251 15538 tablet_copy_service.cc:140] P d3c4f5bd76e0445795bad5f6dd93ff69: Received BeginTabletCopySession request for tablet 83022a40fade44a2988f3de3b6b36aec from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:56840)
I20260709 13:18:24.349804 15538 tablet_copy_service.cc:161] P d3c4f5bd76e0445795bad5f6dd93ff69: Beginning new tablet copy session on tablet 83022a40fade44a2988f3de3b6b36aec from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:56840: session id = dd1a14c3173d49dc9b55bb116e3783c9-83022a40fade44a2988f3de3b6b36aec
I20260709 13:18:24.363152 15538 tablet_copy_source_session.cc:215] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:24.366621 16603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83022a40fade44a2988f3de3b6b36aec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:24.390632 16603 tablet_copy_client.cc:806] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:24.391309 16603 tablet_copy_client.cc:670] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:24.396231 16603 tablet_copy_client.cc:538] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:24.406191 16603 tablet_bootstrap.cc:492] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:24.473575 16236 raft_consensus.cc:1217] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:24.501117 16603 tablet_bootstrap.cc:492] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:24.502725 16603 tablet_bootstrap.cc:492] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:24.503575 16603 ts_tablet_manager.cc:1403] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.098s	user 0.015s	sys 0.011s
I20260709 13:18:24.508724 16603 raft_consensus.cc:359] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:24.512276 16603 raft_consensus.cc:740] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:24.513576 16603 consensus_queue.cc:260] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:24.517076 16603 ts_tablet_manager.cc:1434] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:18:24.520433 15538 tablet_copy_service.cc:342] P d3c4f5bd76e0445795bad5f6dd93ff69: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-83022a40fade44a2988f3de3b6b36aec received from {username='slave'} at 127.0.0.1:56840
I20260709 13:18:24.520884 15538 tablet_copy_service.cc:434] P d3c4f5bd76e0445795bad5f6dd93ff69: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-83022a40fade44a2988f3de3b6b36aec on tablet 83022a40fade44a2988f3de3b6b36aec with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:24.664454 16236 raft_consensus.cc:1217] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:24.752822 16236 raft_consensus.cc:1217] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:24.848052 16236 raft_consensus.cc:1217] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:24.927963 16571 raft_consensus.cc:1064] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:24.930583 16571 consensus_queue.cc:237] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:24.940352 16236 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:24.942283 15595 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:24.943642 15524 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:24.945322 16572 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:24.954329 16572 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:24.962561 16571 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:24.970494 16562 raft_consensus.cc:2955] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:24.972162 16236 raft_consensus.cc:2955] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:24.997577 15425 catalog_manager.cc:5697] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.026904 15595 raft_consensus.cc:2955] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.041508 15526 raft_consensus.cc:2955] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.090466 15669 consensus_queue.cc:237] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:25.096583 15526 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:25.097939 16562 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:25.098603 16236 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:25.109987 16562 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:25.112052 16617 raft_consensus.cc:2955] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.116155 16236 raft_consensus.cc:2955] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.118115 15526 raft_consensus.cc:2955] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.147605 16557 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:25.147006 15424 catalog_manager.cc:5697] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:25.148242 16557 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:25.151564 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:25.152259 15595 raft_consensus.cc:2393] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 2: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:25.154034 16557 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:25.155736 15410 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2a5f6755312b4dc09781ecf8a9468c17 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:25.161125 15589 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a5f6755312b4dc09781ecf8a9468c17 with delete_type TABLET_DATA_TOMBSTONED (TS 5498ef3e89a941e39c25f68cf64e6b0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53300
I20260709 13:18:25.170020 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:25.170739 15669 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:25.171083 16622 tablet_replica.cc:333] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:18:25.172147 16622 raft_consensus.cc:2243] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.172438 15491 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:25.172796 16622 raft_consensus.cc:2272] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.174697 16557 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:25.176774 16557 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:25.183372 16557 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:25.175932 16622 ts_tablet_manager.cc:1916] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:25.187103 16615 raft_consensus.cc:1064] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:25.192713 16615 consensus_queue.cc:237] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:25.202054 16622 ts_tablet_manager.cc:1929] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:25.202608 16622 log.cc:1199] T 2a5f6755312b4dc09781ecf8a9468c17 P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-1-root/wals/2a5f6755312b4dc09781ecf8a9468c17
I20260709 13:18:25.204586 16557 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:25.204960 15412 catalog_manager.cc:5028] TS 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): tablet 2a5f6755312b4dc09781ecf8a9468c17 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:25.207866 16557 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:25.208343 16575 raft_consensus.cc:1064] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:25.210547 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:25.229311 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:25.230230 15595 raft_consensus.cc:2393] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 2: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:25.233374 16575 consensus_queue.cc:237] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:25.265625 16238 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:25.266389 15595 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:25.268100 16562 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:25.271088 16617 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:25.278858 15520 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:25.282088 16615 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:18:25.310277 16617 raft_consensus.cc:2955] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.315611 15520 raft_consensus.cc:2955] T ab68091b31d248a6ac99ca234a252f77 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.323745 16236 raft_consensus.cc:2955] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.333877 15422 catalog_manager.cc:5697] T ab68091b31d248a6ac99ca234a252f77 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.341001 15674 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:25.346184 16552 raft_consensus.cc:1064] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:25.348920 15491 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:25.350404 16573 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:25.350843 15520 raft_consensus.cc:1275] T f25e63ed82fa4e008109b74fe8df71ba P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:25.350896 16552 consensus_queue.cc:237] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:25.354218 15677 raft_consensus.cc:1275] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:25.358246 16565 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:25.358592 16239 raft_consensus.cc:1275] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:25.363550 16574 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:25.370823 16626 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:25.397151 16573 raft_consensus.cc:697] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:25.399410 16573 consensus_queue.cc:237] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:25.408342 16236 raft_consensus.cc:1275] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:25.409437 16565 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.409981 16565 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:25.410625 16630 consensus_queue.cc:1048] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:25.411963 16565 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:25.416349 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:25.418385 15595 raft_consensus.cc:2955] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.421370 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:25.423594 15674 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:25.430327 15566 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:25.431221 16565 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:25.431735 16565 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:25.432163 16565 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:25.440753 16565 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:25.445479 15597 raft_consensus.cc:1275] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:25.462786 16629 consensus_queue.cc:1048] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:25.467710 15674 raft_consensus.cc:1275] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:25.471134 16565 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:25.473853 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:25.477129 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:25.477768 15674 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:25.487123 15422 catalog_manager.cc:5697] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 1 to 2, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 2 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:25.489120 16573 raft_consensus.cc:2955] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.494666 15603 raft_consensus.cc:2955] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.504001 16239 raft_consensus.cc:2955] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.496773 15677 consensus_queue.cc:237] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:25.497879 16626 raft_consensus.cc:2955] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.500447 16557 consensus_queue.cc:1048] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:25.516417 16236 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:25.518211 16236 raft_consensus.cc:2955] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.526597 15595 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:25.504194 15524 raft_consensus.cc:2955] T f25e63ed82fa4e008109b74fe8df71ba P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.501560 15674 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:18:25.535739 15677 raft_consensus.cc:2955] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.539597 15422 catalog_manager.cc:5697] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.547435 16617 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:25.548231 16562 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:25.546540 15424 catalog_manager.cc:5697] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:25.552228 15566 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:25.553773 16575 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:25.554363 16575 raft_consensus.cc:697] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:25.556421 16575 consensus_queue.cc:237] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:25.576599 16562 raft_consensus.cc:2955] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.584540 15603 consensus_queue.cc:237] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:25.585449 15597 raft_consensus.cc:2955] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.602622 16238 raft_consensus.cc:1275] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:25.602345 15670 raft_consensus.cc:2955] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.604206 16574 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:25.626027 15410 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ab68091b31d248a6ac99ca234a252f77 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:25.627374 16239 raft_consensus.cc:2955] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:18:25.665963 16625 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.057s	user 0.001s	sys 0.009s
W20260709 13:18:25.669221 16623 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.053s	user 0.003s	sys 0.007s
I20260709 13:18:25.670373 15677 raft_consensus.cc:1275] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:25.680368 15426 catalog_manager.cc:5697] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.686515 16626 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:25.700891 15514 tablet_service.cc:1558] Processing DeleteTablet for tablet ab68091b31d248a6ac99ca234a252f77 with delete_type TABLET_DATA_TOMBSTONED (TS d3c4f5bd76e0445795bad5f6dd93ff69 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57292
I20260709 13:18:25.718187 15424 catalog_manager.cc:5697] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 2 to 3, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 3 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:25.732605 16644 tablet_replica.cc:333] T ab68091b31d248a6ac99ca234a252f77 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:18:25.733764 16644 raft_consensus.cc:2243] T ab68091b31d248a6ac99ca234a252f77 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.734530 16644 raft_consensus.cc:2272] T ab68091b31d248a6ac99ca234a252f77 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.740610 16626 raft_consensus.cc:2955] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.746829 16644 ts_tablet_manager.cc:1916] T ab68091b31d248a6ac99ca234a252f77 P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:25.748250 15670 raft_consensus.cc:2955] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:18:25.738517 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 48481.7us
I20260709 13:18:25.762660 16644 ts_tablet_manager.cc:1929] T ab68091b31d248a6ac99ca234a252f77 P d3c4f5bd76e0445795bad5f6dd93ff69: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:25.764031 16644 log.cc:1199] T ab68091b31d248a6ac99ca234a252f77 P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-0-root/wals/ab68091b31d248a6ac99ca234a252f77
I20260709 13:18:25.765743 15411 catalog_manager.cc:5028] TS d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): tablet ab68091b31d248a6ac99ca234a252f77 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:25.793162 16236 raft_consensus.cc:2955] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:25.797313 15426 catalog_manager.cc:5697] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:25.800386 15412 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f25e63ed82fa4e008109b74fe8df71ba with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:25.810818 15514 tablet_service.cc:1558] Processing DeleteTablet for tablet f25e63ed82fa4e008109b74fe8df71ba with delete_type TABLET_DATA_TOMBSTONED (TS d3c4f5bd76e0445795bad5f6dd93ff69 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57292
I20260709 13:18:25.814457 16644 tablet_replica.cc:333] T f25e63ed82fa4e008109b74fe8df71ba P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:18:25.815495 16644 raft_consensus.cc:2243] T f25e63ed82fa4e008109b74fe8df71ba P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.816246 16644 raft_consensus.cc:2272] T f25e63ed82fa4e008109b74fe8df71ba P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.829201 16644 ts_tablet_manager.cc:1916] T f25e63ed82fa4e008109b74fe8df71ba P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:25.863965 16644 ts_tablet_manager.cc:1929] T f25e63ed82fa4e008109b74fe8df71ba P d3c4f5bd76e0445795bad5f6dd93ff69: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:25.864470 16644 log.cc:1199] T f25e63ed82fa4e008109b74fe8df71ba P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-0-root/wals/f25e63ed82fa4e008109b74fe8df71ba
I20260709 13:18:25.866120 15411 catalog_manager.cc:5028] TS d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): tablet f25e63ed82fa4e008109b74fe8df71ba (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:25.974714 15595 raft_consensus.cc:1240] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:25.976274 16552 consensus_queue.cc:1059] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:25.976954 16552 consensus_queue.cc:1243] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer 5498ef3e89a941e39c25f68cf64e6b0f log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:18:25.977339 16631 raft_consensus.cc:3055] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:25.977698 16631 raft_consensus.cc:740] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:25.978516 16631 consensus_queue.cc:260] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:25.979925 16631 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:26.016480 15524 consensus_queue.cc:237] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:26.022017 16236 raft_consensus.cc:1275] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:26.022776 15674 raft_consensus.cc:1275] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:26.023962 16631 consensus_queue.cc:1048] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:26.024991 16552 consensus_queue.cc:1048] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:26.032539 16552 raft_consensus.cc:2955] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:26.037873 15670 raft_consensus.cc:2955] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:26.039309 16238 raft_consensus.cc:2955] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:26.065474 15411 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 83022a40fade44a2988f3de3b6b36aec with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:26.073298 15424 catalog_manager.cc:5697] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:26.084185 15677 raft_consensus.cc:1275] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:26.085780 16626 consensus_queue.cc:1048] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:26.108759 15589 tablet_service.cc:1558] Processing DeleteTablet for tablet 83022a40fade44a2988f3de3b6b36aec with delete_type TABLET_DATA_TOMBSTONED (TS 5498ef3e89a941e39c25f68cf64e6b0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53300
I20260709 13:18:26.110617 16648 tablet_replica.cc:333] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:18:26.111747 16648 raft_consensus.cc:2243] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:26.112823 16648 raft_consensus.cc:2272] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:26.118449 16648 ts_tablet_manager.cc:1916] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:26.129447 15526 raft_consensus.cc:1275] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:18:26.129875 15526 pending_rounds.cc:85] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 1
I20260709 13:18:26.130164 15526 pending_rounds.cc:107] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:18:26.130363 15526 raft_consensus.cc:2889] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:26.131912 16626 consensus_queue.cc:1048] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:26.144217 16648 ts_tablet_manager.cc:1929] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:26.144773 16648 log.cc:1199] T 83022a40fade44a2988f3de3b6b36aec P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-1-root/wals/83022a40fade44a2988f3de3b6b36aec
I20260709 13:18:26.178463 15412 catalog_manager.cc:5028] TS 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): tablet 83022a40fade44a2988f3de3b6b36aec (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
W20260709 13:18:26.305083 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 68316.1us
I20260709 13:18:27.204360 15522 consensus_queue.cc:237] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:27.211335 15677 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:18:27.211949 15493 consensus_peers.cc:597] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ecb5853a05664246a26d3fc7ecf92777. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:27.212585 15603 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:27.213224 16653 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:27.214468 16552 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:27.226686 16652 raft_consensus.cc:2955] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:27.228986 15603 raft_consensus.cc:2955] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:27.236559 15674 raft_consensus.cc:2955] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:27.253942 15524 consensus_queue.cc:237] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
W20260709 13:18:27.278105 15493 consensus_peers.cc:597] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6139553a47b64d798a29413921f18396. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:27.265906 15422 catalog_manager.cc:5697] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:27.285080 15603 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:27.278261 15677 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:27.287336 16652 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:18:27.290940 16653 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:27.303808 16652 raft_consensus.cc:2955] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:27.307615 15674 raft_consensus.cc:2955] T 6139553a47b64d798a29413921f18396 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:27.328572 15595 consensus_queue.cc:237] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:27.333621 15603 raft_consensus.cc:2955] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
W20260709 13:18:27.338894 15568 consensus_peers.cc:597] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ccb13c2addbf41569d0c2c23750bd4c9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:27.341560 15677 raft_consensus.cc:1275] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:27.342357 15520 raft_consensus.cc:1275] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:27.343492 16575 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:27.345098 16626 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:27.372756 15424 catalog_manager.cc:5697] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:27.382517 16574 raft_consensus.cc:2955] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:27.390690 15677 raft_consensus.cc:2955] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:27.421216 15670 consensus_queue.cc:237] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:27.450606 15524 raft_consensus.cc:1275] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:27.452193 16571 consensus_queue.cc:1048] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:27.452729 15422 catalog_manager.cc:5697] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
W20260709 13:18:27.467458 15642 consensus_peers.cc:597] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a3d461474623405ab8ee3ca7fdff1722. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:27.474679 15603 raft_consensus.cc:1275] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:27.485683 16562 consensus_queue.cc:1048] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:27.488317 15520 raft_consensus.cc:2955] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:27.518155 16562 raft_consensus.cc:2955] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:27.519914 15524 raft_consensus.cc:2955] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:27.527964 15597 raft_consensus.cc:2955] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:27.545647 15422 catalog_manager.cc:5697] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:27.726181 16674 ts_tablet_manager.cc:933] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:18:27.784598 16676 ts_tablet_manager.cc:933] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:27.808058 16674 tablet_copy_client.cc:323] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.65:42481
I20260709 13:18:27.810302 15538 tablet_copy_service.cc:140] P d3c4f5bd76e0445795bad5f6dd93ff69: Received BeginTabletCopySession request for tablet ecb5853a05664246a26d3fc7ecf92777 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:56840)
I20260709 13:18:27.810736 16676 tablet_copy_client.cc:323] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.66:41593
I20260709 13:18:27.810951 15538 tablet_copy_service.cc:161] P d3c4f5bd76e0445795bad5f6dd93ff69: Beginning new tablet copy session on tablet ecb5853a05664246a26d3fc7ecf92777 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:56840: session id = dd1a14c3173d49dc9b55bb116e3783c9-ecb5853a05664246a26d3fc7ecf92777
I20260709 13:18:27.818434 15613 tablet_copy_service.cc:140] P 5498ef3e89a941e39c25f68cf64e6b0f: Received BeginTabletCopySession request for tablet ccb13c2addbf41569d0c2c23750bd4c9 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:34102)
I20260709 13:18:27.819005 15613 tablet_copy_service.cc:161] P 5498ef3e89a941e39c25f68cf64e6b0f: Beginning new tablet copy session on tablet ccb13c2addbf41569d0c2c23750bd4c9 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:34102: session id = dd1a14c3173d49dc9b55bb116e3783c9-ccb13c2addbf41569d0c2c23750bd4c9
I20260709 13:18:27.838637 15613 tablet_copy_source_session.cc:215] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:27.839344 15538 tablet_copy_source_session.cc:215] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:27.844071 16674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecb5853a05664246a26d3fc7ecf92777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.846597 16676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccb13c2addbf41569d0c2c23750bd4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.854655 16678 ts_tablet_manager.cc:933] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:18:27.876034 16678 tablet_copy_client.cc:323] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.65:42481
I20260709 13:18:27.884949 16674 tablet_copy_client.cc:806] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:27.889168 16674 tablet_copy_client.cc:670] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:27.895862 15538 tablet_copy_service.cc:140] P d3c4f5bd76e0445795bad5f6dd93ff69: Received BeginTabletCopySession request for tablet 6139553a47b64d798a29413921f18396 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:56840)
I20260709 13:18:27.896471 15538 tablet_copy_service.cc:161] P d3c4f5bd76e0445795bad5f6dd93ff69: Beginning new tablet copy session on tablet 6139553a47b64d798a29413921f18396 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:56840: session id = dd1a14c3173d49dc9b55bb116e3783c9-6139553a47b64d798a29413921f18396
I20260709 13:18:27.898044 16676 tablet_copy_client.cc:806] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:27.898659 16676 tablet_copy_client.cc:670] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:27.903944 16676 tablet_copy_client.cc:538] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:27.905807 15538 tablet_copy_source_session.cc:215] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:27.911756 16674 tablet_copy_client.cc:538] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:27.913625 16676 tablet_bootstrap.cc:492] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:27.913204 16683 ts_tablet_manager.cc:933] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:27.940534 16683 tablet_copy_client.cc:323] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.67:32779
I20260709 13:18:27.943014 15687 tablet_copy_service.cc:140] P 3ef186ec12554a1d87814af0a0ace9a4: Received BeginTabletCopySession request for tablet a3d461474623405ab8ee3ca7fdff1722 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:57710)
I20260709 13:18:27.943660 15687 tablet_copy_service.cc:161] P 3ef186ec12554a1d87814af0a0ace9a4: Beginning new tablet copy session on tablet a3d461474623405ab8ee3ca7fdff1722 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:57710: session id = dd1a14c3173d49dc9b55bb116e3783c9-a3d461474623405ab8ee3ca7fdff1722
I20260709 13:18:27.946168 16674 tablet_bootstrap.cc:492] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:27.968745 16676 tablet_bootstrap.cc:492] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:27.972705 16676 tablet_bootstrap.cc:492] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:27.973546 16676 ts_tablet_manager.cc:1403] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.060s	user 0.024s	sys 0.006s
I20260709 13:18:27.976873 16676 raft_consensus.cc:359] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:27.980000 15687 tablet_copy_source_session.cc:215] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:27.984097 16678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6139553a47b64d798a29413921f18396. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:28.009972 16683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3d461474623405ab8ee3ca7fdff1722. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:28.006202 16676 raft_consensus.cc:740] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:28.022742 16676 consensus_queue.cc:260] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:28.025555 16676 ts_tablet_manager.cc:1434] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.000s
I20260709 13:18:28.027933 15613 tablet_copy_service.cc:342] P 5498ef3e89a941e39c25f68cf64e6b0f: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-ccb13c2addbf41569d0c2c23750bd4c9 received from {username='slave'} at 127.0.0.1:34102
I20260709 13:18:28.028405 15613 tablet_copy_service.cc:434] P 5498ef3e89a941e39c25f68cf64e6b0f: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-ccb13c2addbf41569d0c2c23750bd4c9 on tablet ccb13c2addbf41569d0c2c23750bd4c9 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:28.038019 16683 tablet_copy_client.cc:806] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:28.041497 16683 tablet_copy_client.cc:670] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:28.041765 16678 tablet_copy_client.cc:806] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:28.045217 16678 tablet_copy_client.cc:670] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:28.045470 16683 tablet_copy_client.cc:538] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:28.062561 16678 tablet_copy_client.cc:538] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:28.063921 16683 tablet_bootstrap.cc:492] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:28.074028 16674 tablet_bootstrap.cc:492] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:28.075129 16674 tablet_bootstrap.cc:492] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:28.076082 16674 ts_tablet_manager.cc:1403] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.130s	user 0.027s	sys 0.007s
I20260709 13:18:28.079276 16674 raft_consensus.cc:359] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:28.080171 16674 raft_consensus.cc:740] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:28.080904 16674 consensus_queue.cc:260] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:28.086401 16674 ts_tablet_manager.cc:1434] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:18:28.088949 15538 tablet_copy_service.cc:342] P d3c4f5bd76e0445795bad5f6dd93ff69: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-ecb5853a05664246a26d3fc7ecf92777 received from {username='slave'} at 127.0.0.1:56840
I20260709 13:18:28.089391 15538 tablet_copy_service.cc:434] P d3c4f5bd76e0445795bad5f6dd93ff69: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-ecb5853a05664246a26d3fc7ecf92777 on tablet ecb5853a05664246a26d3fc7ecf92777 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:28.092651 16678 tablet_bootstrap.cc:492] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:28.100068 16238 raft_consensus.cc:1217] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:28.128610 16683 tablet_bootstrap.cc:492] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:28.129685 16683 tablet_bootstrap.cc:492] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:28.130570 16683 ts_tablet_manager.cc:1403] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.067s	user 0.025s	sys 0.003s
I20260709 13:18:28.133880 16683 raft_consensus.cc:359] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:28.144253 16683 raft_consensus.cc:740] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:28.144992 16683 consensus_queue.cc:260] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:28.147791 16683 ts_tablet_manager.cc:1434] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.017s	user 0.000s	sys 0.007s
I20260709 13:18:28.149806 15687 tablet_copy_service.cc:342] P 3ef186ec12554a1d87814af0a0ace9a4: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-a3d461474623405ab8ee3ca7fdff1722 received from {username='slave'} at 127.0.0.1:57710
I20260709 13:18:28.150281 15687 tablet_copy_service.cc:434] P 3ef186ec12554a1d87814af0a0ace9a4: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-a3d461474623405ab8ee3ca7fdff1722 on tablet a3d461474623405ab8ee3ca7fdff1722 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:28.157260 16678 tablet_bootstrap.cc:492] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:28.159996 16678 tablet_bootstrap.cc:492] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:28.160743 16678 ts_tablet_manager.cc:1403] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.068s	user 0.018s	sys 0.027s
I20260709 13:18:28.163774 16678 raft_consensus.cc:359] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:28.164603 16678 raft_consensus.cc:740] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:28.165256 16678 consensus_queue.cc:260] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:28.168130 16678 ts_tablet_manager.cc:1434] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:18:28.169902 15538 tablet_copy_service.cc:342] P d3c4f5bd76e0445795bad5f6dd93ff69: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-6139553a47b64d798a29413921f18396 received from {username='slave'} at 127.0.0.1:56840
I20260709 13:18:28.170346 15538 tablet_copy_service.cc:434] P d3c4f5bd76e0445795bad5f6dd93ff69: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-6139553a47b64d798a29413921f18396 on tablet 6139553a47b64d798a29413921f18396 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:28.204272 16238 raft_consensus.cc:1217] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:28.320979 16238 raft_consensus.cc:1217] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:28.491447 16653 raft_consensus.cc:1064] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:28.495144 16238 raft_consensus.cc:1217] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:28.499195 16653 consensus_queue.cc:237] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:28.544093 15670 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:28.546875 16552 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:28.576803 16236 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:28.582473 15603 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:28.586284 16653 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:28.589270 16690 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:28.625653 16652 raft_consensus.cc:2955] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:28.636099 16236 raft_consensus.cc:2955] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:28.644060 15669 raft_consensus.cc:2955] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:28.652377 15603 raft_consensus.cc:2955] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:28.663781 15426 catalog_manager.cc:5697] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:28.739814 15522 consensus_queue.cc:237] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:28.745383 16236 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:28.748202 16690 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:28.751585 15603 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:28.753648 16652 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:28.755286 16552 raft_consensus.cc:2955] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:28.767839 15411 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ecb5853a05664246a26d3fc7ecf92777 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:28.765098 15603 raft_consensus.cc:2955] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:28.774751 16236 raft_consensus.cc:2955] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:28.781101 15424 catalog_manager.cc:5697] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:28.798794 15660 tablet_service.cc:1558] Processing DeleteTablet for tablet ecb5853a05664246a26d3fc7ecf92777 with delete_type TABLET_DATA_TOMBSTONED (TS 3ef186ec12554a1d87814af0a0ace9a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49088
I20260709 13:18:28.804133 16626 raft_consensus.cc:1064] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:28.806958 16626 consensus_queue.cc:237] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:28.822736 16236 raft_consensus.cc:1275] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:28.824882 16671 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:28.841401 15669 raft_consensus.cc:1275] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:28.844388 16574 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:28.854444 15522 raft_consensus.cc:1275] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:28.856608 16671 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:18:28.880625 16700 log.cc:927] Time spent T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:18:28.882081 16699 tablet_replica.cc:333] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:18:28.889247 16699 raft_consensus.cc:2243] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:28.902694 16699 raft_consensus.cc:2272] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:28.907529 16652 raft_consensus.cc:1064] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:28.907812 16626 raft_consensus.cc:2955] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:28.908910 16699 ts_tablet_manager.cc:1916] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:18:28.910357 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 59111us
I20260709 13:18:28.911199 16652 consensus_queue.cc:237] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:28.910431 15522 raft_consensus.cc:2955] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:28.928851 16699 ts_tablet_manager.cc:1929] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:28.929443 16699 log.cc:1199] T ecb5853a05664246a26d3fc7ecf92777 P 3ef186ec12554a1d87814af0a0ace9a4: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-2-root/wals/ecb5853a05664246a26d3fc7ecf92777
I20260709 13:18:28.934629 15410 catalog_manager.cc:5028] TS 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): tablet ecb5853a05664246a26d3fc7ecf92777 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:28.940961 16236 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:28.942322 15597 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:28.942509 16690 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:28.945147 15669 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:28.945618 15674 raft_consensus.cc:2955] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:28.947031 16653 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:28.956238 15424 catalog_manager.cc:5697] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:28.958068 16652 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:28.958698 16239 raft_consensus.cc:2955] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:28.991570 15597 consensus_queue.cc:237] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:28.990908 16653 raft_consensus.cc:2955] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:28.993633 16236 raft_consensus.cc:2955] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:29.002588 16240 raft_consensus.cc:1275] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:29.006573 16702 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:29.007339 15674 raft_consensus.cc:1275] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:29.011629 16704 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:29.012401 15602 raft_consensus.cc:2955] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:29.016587 16702 raft_consensus.cc:2955] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:29.019763 15677 raft_consensus.cc:2955] T 6139553a47b64d798a29413921f18396 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:29.024286 15424 catalog_manager.cc:5697] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:29.026513 15674 raft_consensus.cc:2955] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:29.033942 16669 raft_consensus.cc:1064] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:29.042217 16669 consensus_queue.cc:237] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:29.025862 16240 raft_consensus.cc:2955] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:29.053751 15412 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ccb13c2addbf41569d0c2c23750bd4c9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:29.060945 15522 consensus_queue.cc:237] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:29.071821 16240 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:29.072719 15426 catalog_manager.cc:5697] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:29.075052 15602 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:29.076395 16240 raft_consensus.cc:1275] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:29.076757 15597 raft_consensus.cc:1275] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:29.077073 16653 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:29.078764 16562 consensus_queue.cc:1048] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:29.079665 16692 consensus_queue.cc:1048] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:29.080101 16653 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:29.103895 16690 raft_consensus.cc:2955] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:29.113272 15520 raft_consensus.cc:1275] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:29.117276 15514 tablet_service.cc:1558] Processing DeleteTablet for tablet ccb13c2addbf41569d0c2c23750bd4c9 with delete_type TABLET_DATA_TOMBSTONED (TS d3c4f5bd76e0445795bad5f6dd93ff69 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57292
I20260709 13:18:29.116539 15595 raft_consensus.cc:2955] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:29.120538 16236 raft_consensus.cc:2955] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:29.127610 15411 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6139553a47b64d798a29413921f18396 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:29.136341 16562 raft_consensus.cc:2955] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:29.139320 16571 consensus_queue.cc:1048] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:29.139307 15425 catalog_manager.cc:5697] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:29.141654 16240 raft_consensus.cc:2955] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:29.149080 16714 tablet_replica.cc:333] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:18:29.150971 16714 raft_consensus.cc:2243] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:29.152186 16714 raft_consensus.cc:2272] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:29.150916 15660 tablet_service.cc:1558] Processing DeleteTablet for tablet 6139553a47b64d798a29413921f18396 with delete_type TABLET_DATA_TOMBSTONED (TS 3ef186ec12554a1d87814af0a0ace9a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49088
I20260709 13:18:29.158663 16699 tablet_replica.cc:333] T 6139553a47b64d798a29413921f18396 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:18:29.174530 15597 raft_consensus.cc:2955] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:29.185698 16714 ts_tablet_manager.cc:1916] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:29.186165 15426 catalog_manager.cc:5697] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:29.200189 16699 raft_consensus.cc:2243] T 6139553a47b64d798a29413921f18396 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:29.200963 16699 raft_consensus.cc:2272] T 6139553a47b64d798a29413921f18396 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:29.195725 15522 raft_consensus.cc:2955] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:29.204043 16699 ts_tablet_manager.cc:1916] T 6139553a47b64d798a29413921f18396 P 3ef186ec12554a1d87814af0a0ace9a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:29.221175 16699 ts_tablet_manager.cc:1929] T 6139553a47b64d798a29413921f18396 P 3ef186ec12554a1d87814af0a0ace9a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:29.221560 16699 log.cc:1199] T 6139553a47b64d798a29413921f18396 P 3ef186ec12554a1d87814af0a0ace9a4: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-2-root/wals/6139553a47b64d798a29413921f18396
I20260709 13:18:29.223181 15410 catalog_manager.cc:5028] TS 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): tablet 6139553a47b64d798a29413921f18396 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:29.236868 16714 ts_tablet_manager.cc:1929] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:29.237366 16714 log.cc:1199] T ccb13c2addbf41569d0c2c23750bd4c9 P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-0-root/wals/ccb13c2addbf41569d0c2c23750bd4c9
I20260709 13:18:29.239118 15411 catalog_manager.cc:5028] TS d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): tablet ccb13c2addbf41569d0c2c23750bd4c9 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:29.676833 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:29.690742 15674 consensus_queue.cc:237] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:29.696756 16240 raft_consensus.cc:1275] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:29.699584 16692 consensus_queue.cc:1048] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:29.707626 15522 raft_consensus.cc:1275] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:29.708437 16669 raft_consensus.cc:2955] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:29.710005 16562 consensus_queue.cc:1048] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:29.722188 15522 raft_consensus.cc:2955] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:29.721612 16240 raft_consensus.cc:2955] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:29.736691 15410 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a3d461474623405ab8ee3ca7fdff1722 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:29.746932 15426 catalog_manager.cc:5697] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:29.765407 15589 tablet_service.cc:1558] Processing DeleteTablet for tablet a3d461474623405ab8ee3ca7fdff1722 with delete_type TABLET_DATA_TOMBSTONED (TS 5498ef3e89a941e39c25f68cf64e6b0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53300
I20260709 13:18:29.769507 16719 tablet_replica.cc:333] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:18:29.770583 16719 raft_consensus.cc:2243] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:29.771327 16719 raft_consensus.cc:2272] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:29.775496 16719 ts_tablet_manager.cc:1916] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:29.795862 16719 ts_tablet_manager.cc:1929] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:29.796255 16719 log.cc:1199] T a3d461474623405ab8ee3ca7fdff1722 P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-1-root/wals/a3d461474623405ab8ee3ca7fdff1722
I20260709 13:18:29.797886 15412 catalog_manager.cc:5028] TS 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): tablet a3d461474623405ab8ee3ca7fdff1722 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:29.799185 15597 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:18:29.799883 15597 raft_consensus.cc:606] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:29.802416 15597 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:18:29.802954 15597 raft_consensus.cc:606] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Received request to transfer leadership to TS dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:29.805380 15597 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:18:29.805886 15597 raft_consensus.cc:606] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:29.817679 15674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:18:29.818212 15674 raft_consensus.cc:606] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Received request to transfer leadership to TS dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:29.820716 15597 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:18:29.821269 15597 raft_consensus.cc:606] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:29.826675 16626 raft_consensus.cc:993] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:29.827206 16702 raft_consensus.cc:1081] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Signalling peer 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:29.828716 15674 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:49100
I20260709 13:18:29.829421 15674 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:29.829833 15674 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:29.838508 16626 raft_consensus.cc:993] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:29.839013 16626 raft_consensus.cc:1081] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Signalling peer dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:29.840494 16240 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:57244
I20260709 13:18:29.840994 16240 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:29.841313 16240 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:29.841571 15524 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
mode: GRACEFUL
new_leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:34862
I20260709 13:18:29.842056 15524 raft_consensus.cc:606] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Received request to transfer leadership to TS dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:29.843297 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:18:29.843696 15674 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:29.844887 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:29.845963 15674 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:29.846617 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:29.847249 15524 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:29.847656 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:29.848109 15597 raft_consensus.cc:3055] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:29.848397 15597 raft_consensus.cc:740] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:29.849174 15597 consensus_queue.cc:260] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:29.850322 15597 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:29.851702 16702 raft_consensus.cc:993] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:29.852718 16704 raft_consensus.cc:1081] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Signalling peer 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:29.852819 15524 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:29.846267 16240 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:29.854833 15674 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:49100
I20260709 13:18:29.854914 15641 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:29.855545 15674 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:29.855923 15674 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:29.856912 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:29.857486 15595 raft_consensus.cc:3055] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:29.857753 15595 raft_consensus.cc:740] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:29.858397 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:29.858513 15595 consensus_queue.cc:260] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:29.859236 15524 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:29.855836 16240 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:29.859305 15597 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:29.862861 15674 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:29.859689 15595 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:29.865576 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:29.866226 15522 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:29.866492 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:29.855947 16669 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:29.867755 16669 raft_consensus.cc:697] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:29.868511 16669 consensus_queue.cc:237] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:29.869169 15524 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:29.870626 16204 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:29.871516 16680 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:29.874070 15522 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:29.864750 15674 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:29.875188 15641 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:29.876109 16562 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:29.876574 16562 raft_consensus.cc:697] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:29.867059 15597 raft_consensus.cc:3055] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:29.877815 15597 raft_consensus.cc:740] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:29.877403 16562 consensus_queue.cc:237] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:29.878662 15597 consensus_queue.cc:260] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:29.879926 15597 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:29.880870 16680 raft_consensus.cc:697] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:29.882414 16680 consensus_queue.cc:237] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:29.883103 15595 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:29.887586 15597 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:29.895357 15424 catalog_manager.cc:5697] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 1 to 2, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:29.896175 15426 catalog_manager.cc:5697] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 1 to 2, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 2 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:29.915112 15426 catalog_manager.cc:5697] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 1 to 2, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:29.928601 16690 raft_consensus.cc:993] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69: : Instructing follower dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:29.929114 16690 raft_consensus.cc:1081] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Signalling peer dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:29.930680 16240 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:57258
I20260709 13:18:29.931154 16240 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:29.931434 16240 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:29.937491 16240 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:29.939531 16240 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:29.940424 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:29.940788 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:29.941025 15524 raft_consensus.cc:3055] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:29.941238 15677 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:29.941447 15524 raft_consensus.cc:740] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:29.942173 15524 consensus_queue.cc:260] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:29.942951 15524 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:29.946082 15677 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:29.947489 16203 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:29.947777 15524 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:29.948241 16680 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:29.948761 16680 raft_consensus.cc:697] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:29.949632 16680 consensus_queue.cc:237] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:29.957692 15426 catalog_manager.cc:5697] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 1 to 2, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 2 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:30.029484 16562 raft_consensus.cc:993] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4: : Instructing follower dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:30.029937 16669 raft_consensus.cc:1081] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Signalling peer dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:30.031277 16240 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:53976
I20260709 13:18:30.031718 16240 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:30.031984 16240 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:30.037228 16240 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:30.038851 16240 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:30.039554 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:30.039846 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:30.040223 15597 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:30.040432 15674 raft_consensus.cc:3055] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:30.040884 15674 raft_consensus.cc:740] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:30.041641 15674 consensus_queue.cc:260] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:30.042801 15674 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:30.045671 15597 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:30.046821 16205 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:30.047729 16724 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:30.047755 15674 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:30.048262 16724 raft_consensus.cc:697] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:30.049266 16724 consensus_queue.cc:237] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:30.057957 15426 catalog_manager.cc:5697] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 1 to 2, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 2 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:30.111351 16626 raft_consensus.cc:993] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:30.111796 16702 raft_consensus.cc:1081] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Signalling peer 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:30.113143 15674 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:49100
I20260709 13:18:30.113556 15674 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:30.113785 15674 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:30.118855 15674 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:30.120617 15674 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:30.121671 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:30.121691 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:30.122252 15524 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:30.122411 15597 raft_consensus.cc:3055] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:30.122779 15597 raft_consensus.cc:740] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:30.123755 15597 consensus_queue.cc:260] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:30.124958 15597 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:30.128656 15524 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:30.130357 15641 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:30.131224 16562 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:30.131608 16562 raft_consensus.cc:697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:30.132414 15597 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:30.132372 16562 consensus_queue.cc:237] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:30.141240 15426 catalog_manager.cc:5697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 1 to 2, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:30.274058 15524 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:30.275174 16724 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:30.285224 15597 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:30.290364 15597 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:30.290923 16562 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:30.291771 16724 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:30.356571 15524 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:30.367692 16562 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:18:30.394794 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 62850.8us
I20260709 13:18:30.421080 15524 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:18:30.425072 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 43572.8us
I20260709 13:18:30.444597 16692 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:30.463629 15676 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:30.466840 16735 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:30.472380 15597 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:30.477893 15677 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:30.475167 16669 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:18:30.479668 16735 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:30.503894 15603 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:30.506649 16680 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:30.533705 15522 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:30.535714 16680 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:30.646425 15595 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:30.647614 16669 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:30.658803 15524 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:30.663231 16669 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:30.832751 15670 consensus_queue.cc:237] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
W20260709 13:18:30.840395 15642 consensus_peers.cc:597] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 68dd26481a1b4442b702cb5976b1fb41. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:30.840715 15520 raft_consensus.cc:1275] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:30.840715 15595 raft_consensus.cc:1275] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:30.842236 16745 consensus_queue.cc:1048] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:30.842995 16669 consensus_queue.cc:1048] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:30.846014 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:30.850965 16562 raft_consensus.cc:2955] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:30.852969 15520 raft_consensus.cc:2955] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:30.865975 15603 consensus_queue.cc:237] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:30.865801 15595 raft_consensus.cc:2955] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:30.876215 15424 catalog_manager.cc:5697] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:30.882325 15524 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:30.882825 15670 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:18:30.883245 15568 consensus_peers.cc:597] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1fa44270fe124591b1e8c82c82b89efe. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:30.885012 16739 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:30.887099 16738 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:30.899962 16739 raft_consensus.cc:2955] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:30.911103 15677 raft_consensus.cc:2955] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:30.923666 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:30.926118 15670 consensus_queue.cc:237] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:30.930018 15524 raft_consensus.cc:2955] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:30.944672 15524 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:18:30.945488 15642 consensus_peers.cc:597] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1c436bd48e6b4ad9a1bb596e2ecb15fc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:30.946363 16669 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:30.945732 15595 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:30.950201 16562 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:30.971236 15422 catalog_manager.cc:5697] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:30.974054 16745 raft_consensus.cc:2955] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:30.983714 15603 raft_consensus.cc:2955] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:30.996886 15595 consensus_queue.cc:237] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:31.008776 15524 raft_consensus.cc:2955] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:31.016412 15424 catalog_manager.cc:5697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:31.021011 15603 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:57344
W20260709 13:18:31.023314 15568 consensus_peers.cc:597] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b9a4c0b115544577a855e9c7383d9a48. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:31.023476 15677 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:31.025764 15603 raft_consensus.cc:606] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:31.028357 16739 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:31.025557 15524 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:31.033601 15524 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:34862
I20260709 13:18:31.034271 15524 raft_consensus.cc:606] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:31.037179 16739 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:31.037786 15603 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:18:31.038441 15603 raft_consensus.cc:606] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Received request to transfer leadership to TS dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:31.042413 15670 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:18:31.042990 15670 raft_consensus.cc:606] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Received request to transfer leadership to TS dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:31.046602 15670 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:18:31.047104 15670 raft_consensus.cc:606] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Received request to transfer leadership to TS dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:31.055744 15670 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:18:31.056293 15670 raft_consensus.cc:606] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Received request to transfer leadership to TS 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:31.058363 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:18:31.058399 16626 raft_consensus.cc:2955] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:31.059973 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:31.063067 15677 raft_consensus.cc:2955] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:31.089866 15524 raft_consensus.cc:2955] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:31.079905 15424 catalog_manager.cc:5697] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:31.162065 16562 raft_consensus.cc:993] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4: : Instructing follower dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:31.162513 16745 raft_consensus.cc:1081] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Signalling peer dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:31.163908 16236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:53976
I20260709 13:18:31.164460 16236 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:31.164795 16236 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:31.171664 16236 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:31.174306 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:31.174887 15522 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:31.179718 16236 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:31.179893 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:31.180444 15677 raft_consensus.cc:3055] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:31.180722 15677 raft_consensus.cc:740] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:31.181392 15677 consensus_queue.cc:260] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:31.182145 15522 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:31.182515 15677 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:31.183496 16204 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:31.184360 16735 raft_consensus.cc:2804] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:31.184814 16735 raft_consensus.cc:697] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:31.185643 16735 consensus_queue.cc:237] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:31.194980 15677 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:31.199767 15424 catalog_manager.cc:5697] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 1 to 2, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 2 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:31.232107 16748 raft_consensus.cc:993] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69: : Instructing follower 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:31.232599 16653 raft_consensus.cc:1081] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Signalling peer 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:31.234081 15677 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:49094
I20260709 13:18:31.234668 15677 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:31.235052 15677 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:31.240144 15677 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:31.241722 15677 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:31.242450 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:31.242923 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:31.243072 15524 raft_consensus.cc:3055] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:31.243463 15595 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:31.243755 15524 raft_consensus.cc:740] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:31.244506 15524 consensus_queue.cc:260] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:31.245734 15524 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:31.249146 15595 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:31.252441 15524 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:31.253015 15642 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:31.254537 16745 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:31.256345 16745 raft_consensus.cc:697] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:31.257299 16745 consensus_queue.cc:237] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:31.280035 15424 catalog_manager.cc:5697] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 1 to 2, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:31.310418 16777 ts_tablet_manager.cc:933] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:31.314594 16777 tablet_copy_client.cc:323] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.67:32779
I20260709 13:18:31.316437 15687 tablet_copy_service.cc:140] P 3ef186ec12554a1d87814af0a0ace9a4: Received BeginTabletCopySession request for tablet 68dd26481a1b4442b702cb5976b1fb41 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:57710)
I20260709 13:18:31.317005 15687 tablet_copy_service.cc:161] P 3ef186ec12554a1d87814af0a0ace9a4: Beginning new tablet copy session on tablet 68dd26481a1b4442b702cb5976b1fb41 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:57710: session id = dd1a14c3173d49dc9b55bb116e3783c9-68dd26481a1b4442b702cb5976b1fb41
I20260709 13:18:31.328286 15687 tablet_copy_source_session.cc:215] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:31.331542 16777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68dd26481a1b4442b702cb5976b1fb41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:31.352811 16777 tablet_copy_client.cc:806] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:31.353484 16777 tablet_copy_client.cc:670] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:31.357601 16777 tablet_copy_client.cc:538] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:31.365907 16777 tablet_bootstrap.cc:492] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:31.383162 16781 ts_tablet_manager.cc:933] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:31.384827 16781 tablet_copy_client.cc:323] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.67:32779
I20260709 13:18:31.386211 15687 tablet_copy_service.cc:140] P 3ef186ec12554a1d87814af0a0ace9a4: Received BeginTabletCopySession request for tablet 1c436bd48e6b4ad9a1bb596e2ecb15fc from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:57710)
I20260709 13:18:31.386647 15687 tablet_copy_service.cc:161] P 3ef186ec12554a1d87814af0a0ace9a4: Beginning new tablet copy session on tablet 1c436bd48e6b4ad9a1bb596e2ecb15fc from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:57710: session id = dd1a14c3173d49dc9b55bb116e3783c9-1c436bd48e6b4ad9a1bb596e2ecb15fc
I20260709 13:18:31.386890 16777 tablet_bootstrap.cc:492] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:31.388079 16777 tablet_bootstrap.cc:492] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:31.388775 16777 ts_tablet_manager.cc:1403] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.023s	user 0.023s	sys 0.001s
I20260709 13:18:31.394093 15687 tablet_copy_source_session.cc:215] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:31.395149 16777 raft_consensus.cc:359] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:31.396075 16777 raft_consensus.cc:740] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:31.397212 16781 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c436bd48e6b4ad9a1bb596e2ecb15fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:31.396752 16777 consensus_queue.cc:260] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:31.402261 16777 ts_tablet_manager.cc:1434] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:18:31.404771 16783 ts_tablet_manager.cc:933] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:31.420710 16781 tablet_copy_client.cc:806] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:31.421332 16781 tablet_copy_client.cc:670] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:31.428723 15687 tablet_copy_service.cc:342] P 3ef186ec12554a1d87814af0a0ace9a4: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-68dd26481a1b4442b702cb5976b1fb41 received from {username='slave'} at 127.0.0.1:57710
I20260709 13:18:31.429169 15687 tablet_copy_service.cc:434] P 3ef186ec12554a1d87814af0a0ace9a4: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-68dd26481a1b4442b702cb5976b1fb41 on tablet 68dd26481a1b4442b702cb5976b1fb41 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:31.430451 16781 tablet_copy_client.cc:538] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:31.432546 16783 tablet_copy_client.cc:323] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.66:41593
I20260709 13:18:31.434341 15613 tablet_copy_service.cc:140] P 5498ef3e89a941e39c25f68cf64e6b0f: Received BeginTabletCopySession request for tablet 1fa44270fe124591b1e8c82c82b89efe from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:34102)
I20260709 13:18:31.434890 15613 tablet_copy_service.cc:161] P 5498ef3e89a941e39c25f68cf64e6b0f: Beginning new tablet copy session on tablet 1fa44270fe124591b1e8c82c82b89efe from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:34102: session id = dd1a14c3173d49dc9b55bb116e3783c9-1fa44270fe124591b1e8c82c82b89efe
I20260709 13:18:31.443593 15613 tablet_copy_source_session.cc:215] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:31.445609 16781 tablet_bootstrap.cc:492] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:31.452311 16745 raft_consensus.cc:993] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: : Instructing follower dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:31.453007 16745 raft_consensus.cc:1081] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Signalling peer dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:31.454442 16783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fa44270fe124591b1e8c82c82b89efe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:31.454839 16236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:53976
I20260709 13:18:31.455389 16236 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:31.455713 16236 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:31.462734 16236 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:31.465739 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:31.466320 15595 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:31.470860 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:31.471551 15677 raft_consensus.cc:3055] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:31.471913 15677 raft_consensus.cc:740] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:31.472843 15677 consensus_queue.cc:260] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:31.476330 16783 tablet_copy_client.cc:806] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:31.476955 16783 tablet_copy_client.cc:670] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:31.480901 15677 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:31.495488 15595 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:31.499953 16205 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:31.500180 16236 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:31.500934 16735 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:31.501425 16735 raft_consensus.cc:697] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:31.502323 16735 consensus_queue.cc:237] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:31.498597 15677 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:31.500005 16783 tablet_copy_client.cc:538] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:18:31.563832 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 52431.3us
I20260709 13:18:31.578088 16702 raft_consensus.cc:993] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:31.611709 16626 raft_consensus.cc:1081] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Signalling peer dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:31.616367 16783 tablet_bootstrap.cc:492] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:31.616569 16777 ts_tablet_manager.cc:933] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:31.619210 16236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:57244
I20260709 13:18:31.619796 16236 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:31.620142 16236 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:31.642606 15524 raft_consensus.cc:1275] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:31.642922 16236 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:31.647555 16236 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:31.642000 16781 tablet_bootstrap.cc:492] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:31.650247 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:31.654132 15670 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:31.663779 16781 tablet_bootstrap.cc:492] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:31.664341 16789 raft_consensus.cc:993] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:31.664808 16781 ts_tablet_manager.cc:1403] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.219s	user 0.037s	sys 0.027s
I20260709 13:18:31.667626 16755 consensus_queue.cc:1048] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:31.673069 16781 raft_consensus.cc:359] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:31.682113 16781 raft_consensus.cc:740] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:31.688459 16789 raft_consensus.cc:1081] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Signalling peer 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:31.695446 16777 tablet_copy_client.cc:323] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.66:41593
I20260709 13:18:31.682852 16781 consensus_queue.cc:260] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:31.698464 16781 ts_tablet_manager.cc:1434] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.004s
I20260709 13:18:31.702739 15687 tablet_copy_service.cc:342] P 3ef186ec12554a1d87814af0a0ace9a4: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-1c436bd48e6b4ad9a1bb596e2ecb15fc received from {username='slave'} at 127.0.0.1:57710
I20260709 13:18:31.703225 15687 tablet_copy_service.cc:434] P 3ef186ec12554a1d87814af0a0ace9a4: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-1c436bd48e6b4ad9a1bb596e2ecb15fc on tablet 1c436bd48e6b4ad9a1bb596e2ecb15fc with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:31.706951 15670 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:31.708120 16203 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:31.708932 16724 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:31.709457 16724 raft_consensus.cc:697] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:31.710824 16724 consensus_queue.cc:237] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:31.720328 15424 catalog_manager.cc:5697] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 1 to 2, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 2 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:31.724841 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:31.725453 15595 raft_consensus.cc:3055] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:31.725716 15595 raft_consensus.cc:740] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:31.726429 15595 consensus_queue.cc:260] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:31.727596 15595 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:31.734304 15595 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:31.735401 15613 tablet_copy_service.cc:140] P 5498ef3e89a941e39c25f68cf64e6b0f: Received BeginTabletCopySession request for tablet b9a4c0b115544577a855e9c7383d9a48 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:34102)
W20260709 13:18:31.734993 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 49598.8us
I20260709 13:18:31.754724 15613 tablet_copy_service.cc:161] P 5498ef3e89a941e39c25f68cf64e6b0f: Beginning new tablet copy session on tablet b9a4c0b115544577a855e9c7383d9a48 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:34102: session id = dd1a14c3173d49dc9b55bb116e3783c9-b9a4c0b115544577a855e9c7383d9a48
W20260709 13:18:31.746542 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 139455us
I20260709 13:18:31.749595 15677 raft_consensus.cc:1275] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:31.759830 16755 consensus_queue.cc:1048] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:31.761561 15670 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:49100
I20260709 13:18:31.764456 16790 raft_consensus.cc:993] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4: : Instructing follower 5498ef3e89a941e39c25f68cf64e6b0f to start an election
I20260709 13:18:31.765290 15670 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:31.766892 15670 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:31.767409 15613 tablet_copy_source_session.cc:215] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:31.768087 16790 raft_consensus.cc:1081] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Signalling peer 5498ef3e89a941e39c25f68cf64e6b0f to start an election
I20260709 13:18:31.769973 15524 raft_consensus.cc:1275] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:31.779709 15595 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:53310
I20260709 13:18:31.780390 15595 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:31.780772 15595 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:31.785531 16777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9a4c0b115544577a855e9c7383d9a48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:31.804311 16777 tablet_copy_client.cc:806] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:31.804908 16777 tablet_copy_client.cc:670] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:31.807212 16240 raft_consensus.cc:1217] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:31.802286 15424 catalog_manager.cc:5697] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 1 to 2, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 2 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:31.814466 15595 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:31.817485 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:31.818042 15522 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:31.819792 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:31.820379 15677 raft_consensus.cc:3055] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:31.820660 15677 raft_consensus.cc:740] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:31.821060 15670 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:31.823598 15670 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:31.824873 15522 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
W20260709 13:18:31.817165 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 51107.8us
I20260709 13:18:31.821384 15677 consensus_queue.cc:260] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:31.826613 15677 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:31.830119 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:31.830328 15567 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:31.830755 15524 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:31.832571 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:31.833173 15603 raft_consensus.cc:3055] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:31.833487 15603 raft_consensus.cc:740] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:31.834273 15603 consensus_queue.cc:260] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:31.835493 15603 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:31.836400 15597 raft_consensus.cc:1275] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:31.832593 16702 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:31.837873 15595 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:31.838829 15524 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:31.840888 16692 consensus_queue.cc:1048] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:31.848867 16745 consensus_queue.cc:1048] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:31.851008 16777 tablet_copy_client.cc:538] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:31.853246 15641 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:31.857235 16745 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:31.857744 16745 raft_consensus.cc:697] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:31.860225 16777 tablet_bootstrap.cc:492] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:31.841331 15677 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:18:31.839933 16702 raft_consensus.cc:697] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:31.858609 16745 consensus_queue.cc:237] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:31.862303 16702 consensus_queue.cc:237] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:31.845631 16783 tablet_bootstrap.cc:492] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:31.866353 16783 tablet_bootstrap.cc:492] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:31.867159 16783 ts_tablet_manager.cc:1403] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.267s	user 0.021s	sys 0.006s
I20260709 13:18:31.902717 16783 raft_consensus.cc:359] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:31.903795 16783 raft_consensus.cc:740] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:31.904544 16783 consensus_queue.cc:260] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:31.874101 15603 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:31.936164 16783 ts_tablet_manager.cc:1434] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.069s	user 0.006s	sys 0.001s
I20260709 13:18:31.973989 15613 tablet_copy_service.cc:342] P 5498ef3e89a941e39c25f68cf64e6b0f: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-1fa44270fe124591b1e8c82c82b89efe received from {username='slave'} at 127.0.0.1:34102
I20260709 13:18:31.974442 15613 tablet_copy_service.cc:434] P 5498ef3e89a941e39c25f68cf64e6b0f: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-1fa44270fe124591b1e8c82c82b89efe on tablet 1fa44270fe124591b1e8c82c82b89efe with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:31.998781 15422 catalog_manager.cc:5697] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 2 to 3, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 3 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:32.024590 15603 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:32.024703 16777 tablet_bootstrap.cc:492] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:32.026764 15425 catalog_manager.cc:5697] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 1 to 2, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:32.027601 16724 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:18:32.029084 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 70463.2us
I20260709 13:18:32.030645 16777 tablet_bootstrap.cc:492] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:32.036639 16777 ts_tablet_manager.cc:1403] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.177s	user 0.040s	sys 0.046s
I20260709 13:18:32.044627 16240 raft_consensus.cc:1217] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:32.046438 16777 raft_consensus.cc:359] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:32.047361 16777 raft_consensus.cc:740] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:32.048190 16777 consensus_queue.cc:260] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:32.050758 16777 ts_tablet_manager.cc:1434] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:18:32.058772 15613 tablet_copy_service.cc:342] P 5498ef3e89a941e39c25f68cf64e6b0f: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-b9a4c0b115544577a855e9c7383d9a48 received from {username='slave'} at 127.0.0.1:34102
I20260709 13:18:32.059235 15613 tablet_copy_service.cc:434] P 5498ef3e89a941e39c25f68cf64e6b0f: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-b9a4c0b115544577a855e9c7383d9a48 on tablet b9a4c0b115544577a855e9c7383d9a48 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:32.061113 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:32.075608 15677 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:32.077412 16755 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:32.134672 15595 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:32.136247 16755 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:32.150480 15677 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:32.157320 16735 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:32.157764 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:32.208033 16240 raft_consensus.cc:1217] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:32.250507 15677 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:18:32.251121 15677 raft_consensus.cc:606] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Received request to transfer leadership to TS dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:32.254734 15677 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:18:32.255257 15677 raft_consensus.cc:606] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Received request to transfer leadership to TS 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:32.257680 15597 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:18:32.258174 15597 raft_consensus.cc:606] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:32.261056 15522 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
mode: GRACEFUL
new_leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:34862
I20260709 13:18:32.261638 15522 raft_consensus.cc:606] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Received request to transfer leadership to TS 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:32.264353 15597 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:18:32.264858 15597 raft_consensus.cc:606] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:32.266189 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:18:32.267828 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:32.285485 16815 raft_consensus.cc:1064] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:32.287569 16815 consensus_queue.cc:237] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:32.293566 16240 raft_consensus.cc:1275] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:32.295563 16787 consensus_queue.cc:1048] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:32.296758 15522 raft_consensus.cc:1275] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:32.299985 16811 consensus_queue.cc:1048] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:32.293613 15597 raft_consensus.cc:1275] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:32.305378 16787 consensus_queue.cc:1048] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:32.315883 16787 raft_consensus.cc:2955] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:32.317433 15520 raft_consensus.cc:2955] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:32.326064 15595 raft_consensus.cc:1275] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:32.328055 16811 consensus_queue.cc:1048] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:32.332168 15597 raft_consensus.cc:2955] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:32.334957 15422 catalog_manager.cc:5697] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:32.339074 16240 raft_consensus.cc:2955] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:32.349557 15677 consensus_queue.cc:237] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:32.356957 15520 raft_consensus.cc:1275] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:32.361436 16815 consensus_queue.cc:1048] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:32.362010 16778 raft_consensus.cc:993] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69: : Instructing follower 5498ef3e89a941e39c25f68cf64e6b0f to start an election
I20260709 13:18:32.362504 16813 raft_consensus.cc:1081] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Signalling peer 5498ef3e89a941e39c25f68cf64e6b0f to start an election
I20260709 13:18:32.364384 15595 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:53324
I20260709 13:18:32.365023 15595 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:32.365409 15522 raft_consensus.cc:1275] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:32.365432 15595 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:32.369845 16692 consensus_queue.cc:1048] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:32.396020 16702 raft_consensus.cc:993] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:32.396620 16702 raft_consensus.cc:1081] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Signalling peer 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:32.396116 16815 raft_consensus.cc:2955] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:32.401844 15670 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:49100
I20260709 13:18:32.402449 15670 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:32.402788 15670 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:32.406418 16702 raft_consensus.cc:993] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:32.407065 16702 raft_consensus.cc:1081] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Signalling peer 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:32.407439 15522 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:32.409046 15528 raft_consensus.cc:2955] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:32.414242 16793 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:32.415912 15677 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:49100
I20260709 13:18:32.416571 15677 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:32.417016 15677 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:32.420295 15410 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 68dd26481a1b4442b702cb5976b1fb41 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:32.421972 16238 raft_consensus.cc:1217] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:32.426206 15670 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:32.427735 15669 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:32.429374 16793 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:32.429862 15670 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:32.432631 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:32.433090 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:32.433341 15528 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:32.433672 15603 raft_consensus.cc:3055] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:32.434078 15603 raft_consensus.cc:740] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:32.434801 15603 consensus_queue.cc:260] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:32.435866 15603 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:32.440115 15528 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:32.444576 15603 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:32.446741 16236 raft_consensus.cc:1275] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:32.450194 16815 consensus_queue.cc:1048] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:32.453778 15642 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:32.455312 16692 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:32.455829 16692 raft_consensus.cc:697] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:32.456714 16692 consensus_queue.cc:237] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:32.462284 16236 raft_consensus.cc:2955] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:32.464931 15677 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:32.473379 16787 raft_consensus.cc:993] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: : Instructing follower 5498ef3e89a941e39c25f68cf64e6b0f to start an election
I20260709 13:18:32.474539 16786 raft_consensus.cc:1081] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Signalling peer 5498ef3e89a941e39c25f68cf64e6b0f to start an election
I20260709 13:18:32.487807 15595 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:32.490314 15595 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:32.491636 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:32.492260 15524 raft_consensus.cc:3055] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:32.492563 15524 raft_consensus.cc:740] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:32.492624 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:32.493191 15676 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:32.493403 15524 consensus_queue.cc:260] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:32.494575 15524 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:32.504593 15524 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:32.505728 15567 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:32.507493 16793 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:32.508286 16793 raft_consensus.cc:697] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:32.509176 16793 consensus_queue.cc:237] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:32.511674 15677 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:32.513161 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:32.513972 15524 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:32.515880 15676 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:32.524187 15524 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:32.525756 15641 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:32.526995 16692 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:32.527730 16692 raft_consensus.cc:697] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:32.533223 16692 consensus_queue.cc:237] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:32.534096 15595 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:53310
I20260709 13:18:32.534660 15595 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:32.535012 15595 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:32.538681 15424 catalog_manager.cc:5697] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:32.549325 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:32.550027 15603 raft_consensus.cc:3055] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:32.550437 15603 raft_consensus.cc:740] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:32.551277 15603 consensus_queue.cc:260] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:32.544204 15595 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:32.552924 15603 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:32.556716 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:32.557273 15528 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:32.558622 15595 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:32.569649 15528 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:32.571676 15567 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:32.572533 16793 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:32.573033 16793 raft_consensus.cc:697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:32.573911 16793 consensus_queue.cc:237] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:32.579012 15589 tablet_service.cc:1558] Processing DeleteTablet for tablet 68dd26481a1b4442b702cb5976b1fb41 with delete_type TABLET_DATA_TOMBSTONED (TS 5498ef3e89a941e39c25f68cf64e6b0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53300
I20260709 13:18:32.582445 15425 catalog_manager.cc:5697] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 1 to 2, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:32.593348 16786 raft_consensus.cc:1064] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
W20260709 13:18:32.593809 16786 raft_consensus.cc:1068] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: Unable to promote non-voter dd1a14c3173d49dc9b55bb116e3783c9: Service unavailable: leader transfer in progress
I20260709 13:18:32.594961 15603 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:32.597679 15424 catalog_manager.cc:5697] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 1 to 2, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 2 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:32.601883 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:32.602535 15676 raft_consensus.cc:3055] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:32.602805 15676 raft_consensus.cc:740] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:32.603560 15676 consensus_queue.cc:260] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:32.605033 15676 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:32.605216 16834 tablet_replica.cc:333] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:18:32.606536 16834 raft_consensus.cc:2243] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:32.607388 16834 raft_consensus.cc:2272] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:32.616878 15425 catalog_manager.cc:5697] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 1 to 2, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:32.616932 15676 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:32.620352 16834 ts_tablet_manager.cc:1916] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:32.640873 15676 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49088
I20260709 13:18:32.641525 15676 raft_consensus.cc:606] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Received request to transfer leadership
I20260709 13:18:32.641112 15422 catalog_manager.cc:5697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 1 to 2, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 2 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:32.644097 16834 ts_tablet_manager.cc:1929] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:32.644519 16834 log.cc:1199] T 68dd26481a1b4442b702cb5976b1fb41 P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-1-root/wals/68dd26481a1b4442b702cb5976b1fb41
I20260709 13:18:32.648192 15412 catalog_manager.cc:5028] TS 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): tablet 68dd26481a1b4442b702cb5976b1fb41 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:32.648209 15676 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49088
I20260709 13:18:32.648710 15676 raft_consensus.cc:606] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Received request to transfer leadership
W20260709 13:18:32.649693 15410 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 1fa44270fe124591b1e8c82c82b89efe already in progress [suppressed 4 similar messages]
I20260709 13:18:32.653582 15676 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49088
I20260709 13:18:32.654044 15676 raft_consensus.cc:606] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Received request to transfer leadership
W20260709 13:18:32.657809 15474 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:18:32.658174 15474 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 1fa44270fe124591b1e8c82c82b89efe already in progress
W20260709 13:18:32.658428 15474 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 1fa44270fe124591b1e8c82c82b89efe already in progress
I20260709 13:18:32.672420 16692 raft_consensus.cc:993] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4: : Instructing follower dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:32.672950 16815 raft_consensus.cc:1081] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Signalling peer dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:32.674561 16236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:53976
I20260709 13:18:32.675004 16236 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:32.675266 16236 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:32.680615 16236 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:32.682142 16236 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:32.682956 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:32.683099 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:32.683564 15520 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:32.683650 15676 raft_consensus.cc:3055] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:32.683995 15676 raft_consensus.cc:740] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:32.684628 15676 consensus_queue.cc:260] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:32.685413 15676 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:32.690368 15520 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:32.691359 15676 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:32.691520 16204 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:32.692346 16735 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:32.692806 16735 raft_consensus.cc:697] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:32.693761 16735 consensus_queue.cc:237] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:32.706487 15422 catalog_manager.cc:5697] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 1 to 2, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 2 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:32.742813 16702 raft_consensus.cc:1064] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:32.744629 16702 consensus_queue.cc:237] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:32.750357 16236 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:32.751688 15676 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:32.751803 16793 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:32.752583 15520 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:32.754944 16793 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:32.765056 16796 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:32.822000 16796 raft_consensus.cc:2955] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:32.824384 15676 raft_consensus.cc:2955] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:32.824970 15528 raft_consensus.cc:2955] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:32.844430 15422 catalog_manager.cc:5697] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:32.849138 16236 raft_consensus.cc:2955] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:32.877583 15595 consensus_queue.cc:237] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:32.882648 16236 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:32.883652 15670 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:32.883867 16793 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:32.884792 16794 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:32.892227 16796 raft_consensus.cc:2955] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:32.897521 15595 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:32.902935 15524 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:32.904404 16236 raft_consensus.cc:2955] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:32.905174 16815 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:32.908301 16815 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:32.908880 15670 raft_consensus.cc:1275] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:32.910218 16794 consensus_queue.cc:1048] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:32.910768 15676 raft_consensus.cc:2955] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:32.914395 15412 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b9a4c0b115544577a855e9c7383d9a48 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:32.929661 15524 raft_consensus.cc:1275] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:32.931277 16793 consensus_queue.cc:1048] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:32.963939 16238 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Advancing to term 2
I20260709 13:18:32.979005 15425 catalog_manager.cc:5697] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:32.981806 15524 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:18:32.992252 16702 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:33.007311 15514 tablet_service.cc:1558] Processing DeleteTablet for tablet b9a4c0b115544577a855e9c7383d9a48 with delete_type TABLET_DATA_TOMBSTONED (TS d3c4f5bd76e0445795bad5f6dd93ff69 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57292
I20260709 13:18:33.027866 16238 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEARNER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:18:33.029387 15677 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:18:33.031610 16702 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:33.041560 16796 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:33.070242 15520 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:18:33.081614 16786 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:33.090565 16847 tablet_replica.cc:333] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:18:33.096221 16847 raft_consensus.cc:2243] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:33.097302 16847 raft_consensus.cc:2272] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:33.104334 16847 ts_tablet_manager.cc:1916] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:33.123250 16847 ts_tablet_manager.cc:1929] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:33.124092 16847 log.cc:1199] T b9a4c0b115544577a855e9c7383d9a48 P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-0-root/wals/b9a4c0b115544577a855e9c7383d9a48
I20260709 13:18:33.126209 15411 catalog_manager.cc:5028] TS d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): tablet b9a4c0b115544577a855e9c7383d9a48 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:33.133284 16238 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Advancing to term 2
I20260709 13:18:33.155014 16811 raft_consensus.cc:993] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4: : Instructing follower d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:18:33.155572 16811 raft_consensus.cc:1081] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Signalling peer d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:18:33.159772 15595 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:18:33.161162 15528 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:57300
I20260709 13:18:33.162664 15528 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:33.163038 15528 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:33.166814 16238 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEARNER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:18:33.168442 16786 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:33.168540 15528 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:33.169797 16786 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:33.178453 16702 raft_consensus.cc:1064] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:33.180840 16702 consensus_queue.cc:237] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:33.192625 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:33.192765 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:33.194489 15670 raft_consensus.cc:3055] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:33.194535 15528 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:33.200145 15670 raft_consensus.cc:740] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:33.201148 15670 consensus_queue.cc:260] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:33.202513 15670 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:33.215458 16238 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEARNER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:18:33.217005 16796 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:33.217519 15670 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:33.218429 15669 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:18:33.222587 15520 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:18:33.224900 16702 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:33.225543 15491 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:33.231856 16796 raft_consensus.cc:2955] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:33.232887 16793 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:33.234683 16813 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:33.235217 16813 raft_consensus.cc:697] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:33.236111 16813 consensus_queue.cc:237] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:33.250958 15670 raft_consensus.cc:2955] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:33.254642 15528 raft_consensus.cc:2955] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:33.265318 16238 raft_consensus.cc:2955] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:33.268824 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:33.278434 15422 catalog_manager.cc:5697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 2 to 4, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:33.289091 15425 catalog_manager.cc:5697] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 2 to 3, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 3 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:33.291028 15669 raft_consensus.cc:1275] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:33.292611 16724 consensus_queue.cc:1048] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:33.307896 15595 consensus_queue.cc:237] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:33.314000 15528 raft_consensus.cc:1275] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:33.320387 16735 consensus_queue.cc:1048] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:33.321938 15670 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:33.326390 16793 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:33.328361 16238 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:33.334408 16796 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:33.344343 16793 raft_consensus.cc:2955] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:33.364321 15669 raft_consensus.cc:2955] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:33.367522 16238 raft_consensus.cc:2955] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:33.408666 15424 catalog_manager.cc:5697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index 4 to 5, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New cstate: current_term: 2 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:33.444278 15514 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c436bd48e6b4ad9a1bb596e2ecb15fc with delete_type TABLET_DATA_TOMBSTONED (TS d3c4f5bd76e0445795bad5f6dd93ff69 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:57292
W20260709 13:18:33.448918 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 56118.7us
I20260709 13:18:33.450721 16847 tablet_replica.cc:333] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:18:33.455353 16847 raft_consensus.cc:2243] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:33.456105 16847 raft_consensus.cc:2272] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:33.459872 16847 ts_tablet_manager.cc:1916] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:33.463327 15412 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1c436bd48e6b4ad9a1bb596e2ecb15fc with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:33.484030 16847 ts_tablet_manager.cc:1929] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:18:33.484607 16847 log.cc:1199] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-0-root/wals/1c436bd48e6b4ad9a1bb596e2ecb15fc
I20260709 13:18:33.486279 15411 catalog_manager.cc:5028] TS d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): tablet 1c436bd48e6b4ad9a1bb596e2ecb15fc (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:33.494179 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:33.530328 15520 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6139553a47b64d798a29413921f18396"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
mode: GRACEFUL
new_leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:34862
I20260709 13:18:33.530905 15520 raft_consensus.cc:606] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Received request to transfer leadership to TS 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:33.534049 15669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:18:33.534618 15669 raft_consensus.cc:606] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Received request to transfer leadership to TS dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:33.538038 15520 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
mode: GRACEFUL
new_leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:34862
I20260709 13:18:33.538590 15520 raft_consensus.cc:606] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Received request to transfer leadership to TS 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:33.540592 15595 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d481434f60dc4276831163b535fb8159"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:18:33.541038 15595 raft_consensus.cc:606] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:33.543615 15595 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4fd164d74054954b750fe9958e66778"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:18:33.544116 15595 raft_consensus.cc:606] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:33.546137 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:18:33.547609 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:33.549391 16692 raft_consensus.cc:993] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4: : Instructing follower dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:33.549907 16692 raft_consensus.cc:1081] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Signalling peer dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:33.551345 16238 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:53976
I20260709 13:18:33.551774 16238 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:33.552022 16238 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:33.556247 16238 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:33.557895 16238 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:33.558528 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:33.558919 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:33.559557 15520 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:33.560415 15669 raft_consensus.cc:3055] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:33.560736 15669 raft_consensus.cc:740] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:33.561398 15669 consensus_queue.cc:260] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:33.562515 15669 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:33.565299 15520 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:33.566361 16204 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:33.566987 16860 raft_consensus.cc:2804] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:33.567181 15669 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:33.567469 16860 raft_consensus.cc:697] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:33.568264 16860 consensus_queue.cc:237] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:33.577123 15424 catalog_manager.cc:5697] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 1 to 2, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 2 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:33.636651 16778 raft_consensus.cc:993] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69: : Instructing follower 5498ef3e89a941e39c25f68cf64e6b0f to start an election
I20260709 13:18:33.637079 16854 raft_consensus.cc:1081] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Signalling peer 5498ef3e89a941e39c25f68cf64e6b0f to start an election
I20260709 13:18:33.638388 15595 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6139553a47b64d798a29413921f18396"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:53324
I20260709 13:18:33.638828 15595 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:33.639074 15595 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:33.643381 15595 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:33.644805 15595 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:33.645582 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:33.645673 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:33.646392 16236 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:33.646427 15520 raft_consensus.cc:3055] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:33.646796 15520 raft_consensus.cc:740] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:33.647390 15520 consensus_queue.cc:260] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:33.648232 15520 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:33.650960 16236 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:33.652040 15568 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:33.652653 16702 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:33.652901 15520 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:33.653122 16702 raft_consensus.cc:697] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:33.654284 16702 consensus_queue.cc:237] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:33.655133 16236 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEARNER]: Advancing to term 3
I20260709 13:18:33.661378 15669 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:18:33.664654 16236 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEARNER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:18:33.664361 16778 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:33.665885 16854 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:33.684840 15595 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:33.691107 16854 raft_consensus.cc:1064] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:33.673804 15424 catalog_manager.cc:5697] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 1 to 2, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 2 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:33.692595 15595 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:18:33.705272 16854 consensus_queue.cc:237] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:33.711268 16813 raft_consensus.cc:1027] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: attempt to promote peer dd1a14c3173d49dc9b55bb116e3783c9: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 13:18:33.712604 16702 raft_consensus.cc:993] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:33.713100 16862 raft_consensus.cc:1081] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Signalling peer 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:33.714841 15677 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d481434f60dc4276831163b535fb8159"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:49100
I20260709 13:18:33.715369 15677 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:33.715723 15677 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:33.721613 16814 raft_consensus.cc:993] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69: : Instructing follower 5498ef3e89a941e39c25f68cf64e6b0f to start an election
I20260709 13:18:33.722313 16814 raft_consensus.cc:1081] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Signalling peer 5498ef3e89a941e39c25f68cf64e6b0f to start an election
I20260709 13:18:33.724244 15595 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:53324
I20260709 13:18:33.724761 15595 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:33.725111 15595 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:33.727805 15677 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:33.732055 16236 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEARNER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:33.733453 15677 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:33.735321 15603 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:33.737223 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:33.737461 16778 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:33.737811 15524 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:33.740018 15670 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:33.741371 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:33.741968 15603 raft_consensus.cc:3055] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:33.741873 16813 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:33.742372 15603 raft_consensus.cc:740] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:33.743103 15603 consensus_queue.cc:260] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:33.744182 15603 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:33.744997 15524 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:18:33.747617 16813 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:33.752230 15603 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:18:33.753319 15642 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:33.754139 16786 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:33.754182 15595 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:33.754737 16786 raft_consensus.cc:697] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:33.755652 16786 consensus_queue.cc:237] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:33.770555 16813 raft_consensus.cc:2955] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 2 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:33.774302 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:33.774914 15677 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:33.776417 15422 catalog_manager.cc:5697] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 3 to 4, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 4 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:33.772365 15603 raft_consensus.cc:2955] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:33.782121 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:33.786489 15524 raft_consensus.cc:3055] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:33.787658 15595 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:33.789979 15524 raft_consensus.cc:740] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:33.785456 15677 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:33.789207 15670 raft_consensus.cc:2955] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:33.791401 16238 raft_consensus.cc:2955] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:33.791036 15524 consensus_queue.cc:260] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:33.793727 15524 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:33.799114 15422 catalog_manager.cc:5697] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index 2 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:33.809880 15524 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:33.814713 15567 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:33.815945 16862 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:33.816792 16862 raft_consensus.cc:697] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:33.817641 16862 consensus_queue.cc:237] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:33.820305 15524 consensus_queue.cc:237] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:33.830965 15603 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:33.841833 16236 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:33.843107 16866 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:33.841149 15424 catalog_manager.cc:5697] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 1 to 2, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 2 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:33.855682 16866 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:33.864053 16865 raft_consensus.cc:2955] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:33.865253 16236 raft_consensus.cc:2955] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:33.869038 15603 raft_consensus.cc:2955] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:33.874598 15411 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1fa44270fe124591b1e8c82c82b89efe with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:33.880653 15424 catalog_manager.cc:5697] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 5 to 6, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New cstate: current_term: 3 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:33.899602 15660 tablet_service.cc:1558] Processing DeleteTablet for tablet 1fa44270fe124591b1e8c82c82b89efe with delete_type TABLET_DATA_TOMBSTONED (TS 3ef186ec12554a1d87814af0a0ace9a4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:49088
I20260709 13:18:33.907087 16870 tablet_replica.cc:333] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:18:33.907790 16870 raft_consensus.cc:2243] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:33.908491 16870 raft_consensus.cc:2272] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:33.911260 16870 ts_tablet_manager.cc:1916] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:33.919312 16796 raft_consensus.cc:993] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:33.919759 16862 raft_consensus.cc:1081] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Signalling peer 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:33.921070 15670 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4fd164d74054954b750fe9958e66778"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:49100
I20260709 13:18:33.921608 15670 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:33.921991 15670 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:33.926286 16870 ts_tablet_manager.cc:1929] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:18:33.926769 16870 log.cc:1199] T 1fa44270fe124591b1e8c82c82b89efe P 3ef186ec12554a1d87814af0a0ace9a4: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-2-root/wals/1fa44270fe124591b1e8c82c82b89efe
I20260709 13:18:33.928396 15670 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:33.928941 15410 catalog_manager.cc:5028] TS 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): tablet 1fa44270fe124591b1e8c82c82b89efe (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:33.930200 15670 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:33.930950 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:33.931474 15524 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:33.931432 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:33.932003 15603 raft_consensus.cc:3055] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:33.932229 15603 raft_consensus.cc:740] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:33.932821 15603 consensus_queue.cc:260] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:33.933611 15603 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:33.936371 15524 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:33.937517 15641 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:33.938704 16786 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:33.939131 15603 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:33.939196 16786 raft_consensus.cc:697] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:33.940266 16786 consensus_queue.cc:237] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:33.951390 15425 catalog_manager.cc:5697] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 1 to 2, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:33.962584 15677 raft_consensus.cc:1275] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:33.964068 16860 consensus_queue.cc:1048] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:33.977213 15520 raft_consensus.cc:1275] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:33.982089 16860 consensus_queue.cc:1048] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:34.106026 16236 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:34.107216 16796 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:34.119508 15520 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:34.120848 16876 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:34.311419 15524 raft_consensus.cc:1275] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:34.312543 16815 consensus_queue.cc:1048] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:34.315276 15603 raft_consensus.cc:1275] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:34.316437 16786 consensus_queue.cc:1048] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:34.391935 15520 raft_consensus.cc:1275] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:34.393074 16880 consensus_queue.cc:1048] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:34.396188 15603 raft_consensus.cc:1275] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:34.397692 16880 consensus_queue.cc:1048] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:34.403317 15520 raft_consensus.cc:1275] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:34.406327 16796 consensus_queue.cc:1048] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:34.409965 15677 raft_consensus.cc:1275] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:34.412770 16862 consensus_queue.cc:1048] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:34.524793 15670 consensus_queue.cc:237] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
W20260709 13:18:34.532014 15642 consensus_peers.cc:597] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38190179f6494b5591027e1c41f83176. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:34.534134 15520 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:34.536507 15595 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:34.538942 16815 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:34.540629 16815 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:34.547461 16880 raft_consensus.cc:2955] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:34.548722 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:34.549686 15595 raft_consensus.cc:2955] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:34.549489 15520 raft_consensus.cc:2955] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:34.565155 15603 consensus_queue.cc:237] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:34.565459 15425 catalog_manager.cc:5697] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
W20260709 13:18:34.589012 15568 consensus_peers.cc:597] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7dbabc834ad54c16ae25f79064932f42. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:34.591784 15677 raft_consensus.cc:1275] T 7dbabc834ad54c16ae25f79064932f42 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:34.593173 15520 raft_consensus.cc:1275] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:34.593672 16862 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:34.594806 16876 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:34.614352 16876 raft_consensus.cc:2955] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:34.617658 15677 raft_consensus.cc:2955] T 7dbabc834ad54c16ae25f79064932f42 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:34.623238 15524 raft_consensus.cc:2955] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:34.627841 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:34.638290 15603 consensus_queue.cc:237] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:34.651033 15424 catalog_manager.cc:5697] T 7dbabc834ad54c16ae25f79064932f42 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:34.665220 15670 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:34.667083 16796 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:18:34.669886 15568 consensus_peers.cc:597] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e93d4a9ac6e844da87dcd8c6bfa7549b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:34.680958 15524 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:34.685262 16796 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:34.700568 15670 raft_consensus.cc:2955] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:34.703620 15524 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
mode: GRACEFUL
new_leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:34862
I20260709 13:18:34.704236 15524 raft_consensus.cc:606] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Received request to transfer leadership to TS 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:34.708230 15669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "974a060267d24f00888c0942dfbca77e"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:18:34.708843 15669 raft_consensus.cc:606] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:34.709355 16796 raft_consensus.cc:2955] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:34.719259 15603 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:18:34.719907 15603 raft_consensus.cc:606] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:34.721175 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:18:34.722837 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:34.730199 15524 consensus_queue.cc:237] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:34.746340 16866 raft_consensus.cc:993] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69: : Instructing follower 5498ef3e89a941e39c25f68cf64e6b0f to start an election
I20260709 13:18:34.746830 16866 raft_consensus.cc:1081] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Signalling peer 5498ef3e89a941e39c25f68cf64e6b0f to start an election
I20260709 13:18:34.749217 15603 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:53324
I20260709 13:18:34.749751 15603 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:34.750100 15603 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:34.745654 15424 catalog_manager.cc:5697] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:34.752964 15670 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:34.753118 15595 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:34.754602 16865 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:18:34.755852 15493 consensus_peers.cc:597] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ced58b89726545d3af99de36a1aa1f30. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:34.756379 16866 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:34.760862 15603 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:34.763767 15603 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:34.764441 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:34.765044 15677 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:34.764997 15520 raft_consensus.cc:2955] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:34.771941 15677 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:34.773154 15566 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:34.776250 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:34.776856 15524 raft_consensus.cc:3055] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:34.777141 15524 raft_consensus.cc:740] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:34.777953 15524 consensus_queue.cc:260] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:34.778421 16894 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:34.778959 16894 raft_consensus.cc:697] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:34.779104 15524 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:34.779987 16894 consensus_queue.cc:237] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:34.784628 15595 raft_consensus.cc:2955] T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:34.778147 16866 raft_consensus.cc:2955] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:34.789846 15670 raft_consensus.cc:2955] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:34.816787 15422 catalog_manager.cc:5697] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:34.836884 15424 catalog_manager.cc:5697] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 1 to 2, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 2 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:34.845093 15524 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:34.982118 16880 raft_consensus.cc:993] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4: : Instructing follower d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:18:34.982609 16880 raft_consensus.cc:1081] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Signalling peer d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:18:34.985303 15524 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "974a060267d24f00888c0942dfbca77e"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:57300
I20260709 13:18:34.985872 15524 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:34.986232 15524 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:34.993014 15524 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:34.996956 15524 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:34.998163 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:34.998744 15603 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:34.999183 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:34.999758 15669 raft_consensus.cc:3055] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:35.000049 15669 raft_consensus.cc:740] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:35.000777 15669 consensus_queue.cc:260] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:35.001791 15669 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:35.005438 15603 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:35.006801 15493 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:35.007622 16903 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:35.008096 16903 raft_consensus.cc:697] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:35.008888 16903 consensus_queue.cc:237] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:35.010418 15669 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:35.022809 15424 catalog_manager.cc:5697] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 1 to 2, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 2 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:35.034900 16906 ts_tablet_manager.cc:933] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:35.036732 16906 tablet_copy_client.cc:323] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.67:32779
I20260709 13:18:35.038115 15687 tablet_copy_service.cc:140] P 3ef186ec12554a1d87814af0a0ace9a4: Received BeginTabletCopySession request for tablet 38190179f6494b5591027e1c41f83176 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:57710)
I20260709 13:18:35.038558 15687 tablet_copy_service.cc:161] P 3ef186ec12554a1d87814af0a0ace9a4: Beginning new tablet copy session on tablet 38190179f6494b5591027e1c41f83176 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:57710: session id = dd1a14c3173d49dc9b55bb116e3783c9-38190179f6494b5591027e1c41f83176
I20260709 13:18:35.043797 15687 tablet_copy_source_session.cc:215] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:35.047737 16906 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38190179f6494b5591027e1c41f83176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:35.063299 16906 tablet_copy_client.cc:806] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:35.063958 16906 tablet_copy_client.cc:670] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:35.068107 16906 tablet_copy_client.cc:538] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:35.077813 16906 tablet_bootstrap.cc:492] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:35.098452 16906 tablet_bootstrap.cc:492] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:35.099407 16906 tablet_bootstrap.cc:492] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:35.100087 16906 ts_tablet_manager.cc:1403] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.023s	user 0.020s	sys 0.004s
I20260709 13:18:35.103017 16906 raft_consensus.cc:359] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:35.103857 16906 raft_consensus.cc:740] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:35.104511 16906 consensus_queue.cc:260] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:35.107018 16906 ts_tablet_manager.cc:1434] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:18:35.114578 15687 tablet_copy_service.cc:342] P 3ef186ec12554a1d87814af0a0ace9a4: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-38190179f6494b5591027e1c41f83176 received from {username='slave'} at 127.0.0.1:57710
I20260709 13:18:35.115028 15687 tablet_copy_service.cc:434] P 3ef186ec12554a1d87814af0a0ace9a4: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-38190179f6494b5591027e1c41f83176 on tablet 38190179f6494b5591027e1c41f83176 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:35.126844 16906 ts_tablet_manager.cc:933] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:35.131340 16906 tablet_copy_client.cc:323] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.66:41593
I20260709 13:18:35.132890 15613 tablet_copy_service.cc:140] P 5498ef3e89a941e39c25f68cf64e6b0f: Received BeginTabletCopySession request for tablet 7dbabc834ad54c16ae25f79064932f42 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:34102)
I20260709 13:18:35.133476 15613 tablet_copy_service.cc:161] P 5498ef3e89a941e39c25f68cf64e6b0f: Beginning new tablet copy session on tablet 7dbabc834ad54c16ae25f79064932f42 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:34102: session id = dd1a14c3173d49dc9b55bb116e3783c9-7dbabc834ad54c16ae25f79064932f42
I20260709 13:18:35.141211 15613 tablet_copy_source_session.cc:215] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:35.144659 16906 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dbabc834ad54c16ae25f79064932f42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:35.160991 16906 tablet_copy_client.cc:806] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:35.161598 16906 tablet_copy_client.cc:670] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:35.165064 16906 tablet_copy_client.cc:538] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:35.168285 16909 ts_tablet_manager.cc:933] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:35.169941 16909 tablet_copy_client.cc:323] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.66:41593
I20260709 13:18:35.171543 15613 tablet_copy_service.cc:140] P 5498ef3e89a941e39c25f68cf64e6b0f: Received BeginTabletCopySession request for tablet e93d4a9ac6e844da87dcd8c6bfa7549b from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:34102)
I20260709 13:18:35.172101 15613 tablet_copy_service.cc:161] P 5498ef3e89a941e39c25f68cf64e6b0f: Beginning new tablet copy session on tablet e93d4a9ac6e844da87dcd8c6bfa7549b from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:34102: session id = dd1a14c3173d49dc9b55bb116e3783c9-e93d4a9ac6e844da87dcd8c6bfa7549b
I20260709 13:18:35.179772 15613 tablet_copy_source_session.cc:215] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:35.182632 16909 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e93d4a9ac6e844da87dcd8c6bfa7549b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:35.184584 16906 tablet_bootstrap.cc:492] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:35.204788 16796 raft_consensus.cc:993] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:35.205257 16894 raft_consensus.cc:1081] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Signalling peer 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:35.207110 15670 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:49100
I20260709 13:18:35.207641 15670 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:35.207986 15670 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:35.208271 16906 tablet_bootstrap.cc:492] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:35.211205 16906 tablet_bootstrap.cc:492] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:35.211898 16906 ts_tablet_manager.cc:1403] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.028s	user 0.023s	sys 0.001s
I20260709 13:18:35.216404 16906 raft_consensus.cc:359] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:35.217324 16906 raft_consensus.cc:740] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:35.218061 16906 consensus_queue.cc:260] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:35.220515 16906 ts_tablet_manager.cc:1434] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 13:18:35.220507 16909 tablet_copy_client.cc:806] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:35.221410 16909 tablet_copy_client.cc:670] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:35.223935 15613 tablet_copy_service.cc:342] P 5498ef3e89a941e39c25f68cf64e6b0f: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-7dbabc834ad54c16ae25f79064932f42 received from {username='slave'} at 127.0.0.1:34102
I20260709 13:18:35.224400 15613 tablet_copy_service.cc:434] P 5498ef3e89a941e39c25f68cf64e6b0f: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-7dbabc834ad54c16ae25f79064932f42 on tablet 7dbabc834ad54c16ae25f79064932f42 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:35.226544 16909 tablet_copy_client.cc:538] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:35.230226 16913 ts_tablet_manager.cc:933] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:18:35.232390 15670 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:35.237097 16909 tablet_bootstrap.cc:492] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:35.242717 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:35.243338 15603 raft_consensus.cc:3055] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:35.243630 15603 raft_consensus.cc:740] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:35.243963 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:35.244540 16236 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:35.244340 15603 consensus_queue.cc:260] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:35.245436 15603 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:35.255031 16236 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:35.257472 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:35.258347 15670 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:35.260398 16786 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:35.260962 16786 raft_consensus.cc:697] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:35.261801 16786 consensus_queue.cc:237] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:35.242332 16913 tablet_copy_client.cc:323] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.65:42481
I20260709 13:18:35.285836 15677 raft_consensus.cc:1275] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:35.286345 15603 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:35.287722 16894 consensus_queue.cc:1048] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:35.292004 15538 tablet_copy_service.cc:140] P d3c4f5bd76e0445795bad5f6dd93ff69: Received BeginTabletCopySession request for tablet ced58b89726545d3af99de36a1aa1f30 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:56840)
I20260709 13:18:35.292683 15538 tablet_copy_service.cc:161] P d3c4f5bd76e0445795bad5f6dd93ff69: Beginning new tablet copy session on tablet ced58b89726545d3af99de36a1aa1f30 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:56840: session id = dd1a14c3173d49dc9b55bb116e3783c9-ced58b89726545d3af99de36a1aa1f30
I20260709 13:18:35.290966 15424 catalog_manager.cc:5697] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 1 to 2, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:35.343313 15520 raft_consensus.cc:1275] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:35.369434 15538 tablet_copy_source_session.cc:215] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:35.370128 16876 consensus_queue.cc:1048] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:35.385501 16913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ced58b89726545d3af99de36a1aa1f30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:35.440620 16236 raft_consensus.cc:1217] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:35.446364 15603 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:35.467511 16913 tablet_copy_client.cc:806] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:35.468127 16913 tablet_copy_client.cc:670] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:35.471976 16913 tablet_copy_client.cc:538] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:35.475083 16909 tablet_bootstrap.cc:492] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:35.476024 16909 tablet_bootstrap.cc:492] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:35.476720 16909 ts_tablet_manager.cc:1403] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.240s	user 0.113s	sys 0.059s
I20260709 13:18:35.481781 16903 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:35.489854 16909 raft_consensus.cc:359] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:35.490813 16909 raft_consensus.cc:740] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:35.491488 16909 consensus_queue.cc:260] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:35.493970 16909 ts_tablet_manager.cc:1434] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.017s	user 0.002s	sys 0.004s
I20260709 13:18:35.494225 16913 tablet_bootstrap.cc:492] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:35.501574 15612 tablet_copy_service.cc:342] P 5498ef3e89a941e39c25f68cf64e6b0f: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-e93d4a9ac6e844da87dcd8c6bfa7549b received from {username='slave'} at 127.0.0.1:34102
I20260709 13:18:35.502076 15612 tablet_copy_service.cc:434] P 5498ef3e89a941e39c25f68cf64e6b0f: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-e93d4a9ac6e844da87dcd8c6bfa7549b on tablet e93d4a9ac6e844da87dcd8c6bfa7549b with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:35.518611 15669 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:35.520568 16866 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:35.535820 16236 raft_consensus.cc:1217] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:35.556520 16913 tablet_bootstrap.cc:492] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:35.564615 16913 tablet_bootstrap.cc:492] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:35.574851 16913 ts_tablet_manager.cc:1403] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.094s	user 0.020s	sys 0.011s
I20260709 13:18:35.577797 16913 raft_consensus.cc:359] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:35.578666 16913 raft_consensus.cc:740] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:35.579329 16913 consensus_queue.cc:260] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:35.581719 16913 ts_tablet_manager.cc:1434] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:18:35.583374 15538 tablet_copy_service.cc:342] P d3c4f5bd76e0445795bad5f6dd93ff69: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-ced58b89726545d3af99de36a1aa1f30 received from {username='slave'} at 127.0.0.1:56840
I20260709 13:18:35.583786 15538 tablet_copy_service.cc:434] P d3c4f5bd76e0445795bad5f6dd93ff69: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-ced58b89726545d3af99de36a1aa1f30 on tablet ced58b89726545d3af99de36a1aa1f30 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:35.672935 16236 raft_consensus.cc:1217] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:35.723847 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:35.750500 16236 raft_consensus.cc:1217] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:35.783299 15595 raft_consensus.cc:1275] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:35.784759 16786 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:35.801249 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:35.803663 16236 raft_consensus.cc:1275] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:35.813813 16918 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:35.886170 15602 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:18:35.886729 15602 raft_consensus.cc:606] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:35.891579 15669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:18:35.892187 15669 raft_consensus.cc:606] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:35.897213 15524 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
mode: GRACEFUL
new_leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:34862
I20260709 13:18:35.897791 15524 raft_consensus.cc:606] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Received request to transfer leadership to TS 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:35.899355 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:18:35.900744 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:35.924980 16896 raft_consensus.cc:1064] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:35.927453 16896 consensus_queue.cc:237] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:35.933964 16876 raft_consensus.cc:993] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:35.934485 16876 raft_consensus.cc:1081] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Signalling peer 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:35.938171 15670 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:49100
I20260709 13:18:35.938730 15670 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:35.939067 15670 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:35.939486 15524 raft_consensus.cc:1275] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:35.940821 16896 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:35.946247 15670 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:18:35.946806 15669 raft_consensus.cc:1275] T 7dbabc834ad54c16ae25f79064932f42 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:35.949071 16894 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:35.949450 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:35.950048 15524 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:35.950812 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:35.951371 15602 raft_consensus.cc:3055] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:35.951627 15602 raft_consensus.cc:740] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:35.952342 15602 consensus_queue.cc:260] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:18:35.953365 15602 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:35.955170 16236 raft_consensus.cc:1275] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:35.964969 16896 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:35.974313 15524 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:35.980540 15641 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:35.981549 16692 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:35.982050 16692 raft_consensus.cc:697] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:35.982788 16692 consensus_queue.cc:237] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:18:35.993356 16876 raft_consensus.cc:2955] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:35.995450 15669 raft_consensus.cc:2955] T 7dbabc834ad54c16ae25f79064932f42 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:35.998201 16929 raft_consensus.cc:1064] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:36.000775 16929 consensus_queue.cc:237] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:36.003708 15528 raft_consensus.cc:2955] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:36.010730 15602 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:36.012372 16236 raft_consensus.cc:2955] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:36.011261 15670 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:36.017951 15422 catalog_manager.cc:5697] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:36.030586 16238 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:36.039732 15520 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:36.044335 15603 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:36.041402 16786 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:36.047861 16880 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:36.052088 16918 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:36.055339 16866 raft_consensus.cc:993] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69: : Instructing follower 5498ef3e89a941e39c25f68cf64e6b0f to start an election
I20260709 13:18:36.055866 16866 raft_consensus.cc:1081] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Signalling peer 5498ef3e89a941e39c25f68cf64e6b0f to start an election
I20260709 13:18:36.058465 15603 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:53324
I20260709 13:18:36.059051 15603 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:36.059407 15603 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:36.060702 15602 consensus_queue.cc:237] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:36.066774 15520 raft_consensus.cc:1275] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:36.067186 16240 raft_consensus.cc:1275] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:36.068346 16894 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:36.071758 16894 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:36.073320 15425 catalog_manager.cc:5697] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 1 to 2, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:36.081071 16796 raft_consensus.cc:2955] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:36.084467 15603 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:36.091928 16236 raft_consensus.cc:2955] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:36.095008 15528 raft_consensus.cc:2955] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:36.100375 15412 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7dbabc834ad54c16ae25f79064932f42 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:36.120349 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:36.120963 15670 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:36.121721 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:36.122334 15523 raft_consensus.cc:3055] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:36.122627 15523 raft_consensus.cc:740] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:36.123360 15523 consensus_queue.cc:260] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:36.124665 15523 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:36.128564 15670 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:36.130117 15566 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:36.134473 15603 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:36.147121 16896 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:36.152550 16896 raft_consensus.cc:697] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:36.153554 16896 consensus_queue.cc:237] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:36.187474 15424 catalog_manager.cc:5697] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:36.194906 15523 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
W20260709 13:18:36.204433 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 57709.7us
I20260709 13:18:36.215813 16918 raft_consensus.cc:2955] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:36.221467 15520 raft_consensus.cc:2955] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:36.223616 15597 raft_consensus.cc:2955] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:36.226055 15660 tablet_service.cc:1558] Processing DeleteTablet for tablet 7dbabc834ad54c16ae25f79064932f42 with delete_type TABLET_DATA_TOMBSTONED (TS 3ef186ec12554a1d87814af0a0ace9a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49088
I20260709 13:18:36.229611 15426 catalog_manager.cc:5697] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 1 to 2, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 2 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:36.232259 16876 raft_consensus.cc:1064] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:36.239499 15424 catalog_manager.cc:5697] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:36.240706 16876 consensus_queue.cc:237] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:36.265973 16236 raft_consensus.cc:2955] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:36.284487 16949 tablet_replica.cc:333] T 7dbabc834ad54c16ae25f79064932f42 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:18:36.285151 16236 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:36.285640 16949 raft_consensus.cc:2243] T 7dbabc834ad54c16ae25f79064932f42 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:36.287999 16949 raft_consensus.cc:2272] T 7dbabc834ad54c16ae25f79064932f42 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.287535 15669 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:36.288503 16946 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:36.290936 16876 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:36.292274 16949 ts_tablet_manager.cc:1916] T 7dbabc834ad54c16ae25f79064932f42 P 3ef186ec12554a1d87814af0a0ace9a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:36.341190 16949 ts_tablet_manager.cc:1929] T 7dbabc834ad54c16ae25f79064932f42 P 3ef186ec12554a1d87814af0a0ace9a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:36.342784 16949 log.cc:1199] T 7dbabc834ad54c16ae25f79064932f42 P 3ef186ec12554a1d87814af0a0ace9a4: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-2-root/wals/7dbabc834ad54c16ae25f79064932f42
I20260709 13:18:36.391988 15410 catalog_manager.cc:5028] TS 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): tablet 7dbabc834ad54c16ae25f79064932f42 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:36.417795 16896 raft_consensus.cc:2955] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:36.428500 15670 consensus_queue.cc:237] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:36.440846 16240 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:36.449374 16236 raft_consensus.cc:2955] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:36.479866 16880 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:36.498518 15524 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:36.499025 15528 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:36.499117 15669 raft_consensus.cc:2955] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:36.519325 15424 catalog_manager.cc:5697] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:36.521797 15595 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:36.525527 16929 raft_consensus.cc:993] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4: : Instructing follower d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:18:36.526386 16786 raft_consensus.cc:1081] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Signalling peer d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:18:36.525350 16957 raft_consensus.cc:2955] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:36.533435 16237 raft_consensus.cc:2955] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:36.542611 16786 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:36.554570 16880 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:36.564589 16956 raft_consensus.cc:1064] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:36.569459 16956 consensus_queue.cc:237] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:36.574826 16894 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:36.582845 15424 catalog_manager.cc:5697] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:36.588816 15603 consensus_queue.cc:237] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:36.620687 15669 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:36.626060 16796 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:36.626720 15676 raft_consensus.cc:1275] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:36.632997 16240 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:36.634334 16796 consensus_queue.cc:1048] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:36.635123 15410 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 38190179f6494b5591027e1c41f83176 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:18:36.636643 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 45261.6us
I20260709 13:18:36.642071 16237 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:36.672724 15528 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:57300
I20260709 13:18:36.673691 15528 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:36.674149 15528 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:36.676790 15589 tablet_service.cc:1558] Processing DeleteTablet for tablet 38190179f6494b5591027e1c41f83176 with delete_type TABLET_DATA_TOMBSTONED (TS 5498ef3e89a941e39c25f68cf64e6b0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53300
I20260709 13:18:36.677002 16959 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:36.688258 15669 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:36.692763 16956 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:36.699049 16963 tablet_replica.cc:333] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:18:36.699152 15524 raft_consensus.cc:2955] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:36.702858 16963 raft_consensus.cc:2243] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:36.704173 16963 raft_consensus.cc:2272] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.702771 15528 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:36.707437 16876 raft_consensus.cc:2955] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:36.714458 16236 raft_consensus.cc:2955] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:36.718891 15520 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:36.754781 15522 raft_consensus.cc:1275] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:36.755828 15677 raft_consensus.cc:2955] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:36.762624 16880 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:36.763571 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:36.764241 15602 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:36.765305 15595 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:36.767086 16955 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:36.768957 15528 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:36.775754 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:36.776399 15674 raft_consensus.cc:3055] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:36.776693 15674 raft_consensus.cc:740] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:36.777549 15674 consensus_queue.cc:260] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:36.780922 16959 consensus_queue.cc:1048] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:36.778759 15674 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:36.776798 15523 raft_consensus.cc:2955] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:36.797652 15602 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:36.813481 15674 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:36.832512 16963 ts_tablet_manager.cc:1916] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:36.855222 15493 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:36.850226 16958 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:18:36.856084 16955 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:36.876140 15424 catalog_manager.cc:5697] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New cstate: current_term: 1 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:36.882781 15412 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e93d4a9ac6e844da87dcd8c6bfa7549b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:36.886178 16955 raft_consensus.cc:697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
W20260709 13:18:36.886436 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.887212 16955 consensus_queue.cc:237] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:36.891319 15514 tablet_service.cc:1558] Processing DeleteTablet for tablet e93d4a9ac6e844da87dcd8c6bfa7549b with delete_type TABLET_DATA_TOMBSTONED (TS d3c4f5bd76e0445795bad5f6dd93ff69 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57292
I20260709 13:18:36.908983 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:37.012905 16976 tablet_replica.cc:333] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:18:37.019762 16976 raft_consensus.cc:2243] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:37.020503 16976 raft_consensus.cc:2272] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:37.025568 16976 ts_tablet_manager.cc:1916] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:37.036161 16963 ts_tablet_manager.cc:1929] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:37.041683 16963 log.cc:1199] T 38190179f6494b5591027e1c41f83176 P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-1-root/wals/38190179f6494b5591027e1c41f83176
I20260709 13:18:37.076130 15412 catalog_manager.cc:5028] TS 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): tablet 38190179f6494b5591027e1c41f83176 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
W20260709 13:18:37.079701 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 99914.3us
W20260709 13:18:37.122328 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 107436us
I20260709 13:18:37.137308 16972 raft_consensus.cc:2955] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:37.137323 15424 catalog_manager.cc:5697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 2 to 3, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 3 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:37.159886 16976 ts_tablet_manager.cc:1929] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:37.158739 15596 raft_consensus.cc:2955] T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:37.164245 16976 log.cc:1199] T e93d4a9ac6e844da87dcd8c6bfa7549b P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-0-root/wals/e93d4a9ac6e844da87dcd8c6bfa7549b
I20260709 13:18:37.169565 15411 catalog_manager.cc:5028] TS d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): tablet e93d4a9ac6e844da87dcd8c6bfa7549b (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:37.169679 16235 raft_consensus.cc:2955] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:37.210179 15422 catalog_manager.cc:5697] T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:37.243829 15676 raft_consensus.cc:2955] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:37.262203 15528 consensus_queue.cc:237] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:37.269443 16235 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:37.269946 15674 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:37.272452 16866 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:37.284590 16972 raft_consensus.cc:2955] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:37.286248 16982 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:37.287604 15669 raft_consensus.cc:2955] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:37.292829 16235 raft_consensus.cc:2955] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:37.307173 15411 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ced58b89726545d3af99de36a1aa1f30 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:37.311933 15422 catalog_manager.cc:5697] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 3 to 4, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:37.325578 15589 tablet_service.cc:1558] Processing DeleteTablet for tablet ced58b89726545d3af99de36a1aa1f30 with delete_type TABLET_DATA_TOMBSTONED (TS 5498ef3e89a941e39c25f68cf64e6b0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53300
I20260709 13:18:37.326809 16963 tablet_replica.cc:333] T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:18:37.328042 16963 raft_consensus.cc:2243] T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:37.328696 16963 raft_consensus.cc:2272] T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:37.330147 15596 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:37.332018 16972 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:37.332669 16963 ts_tablet_manager.cc:1916] T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:37.343813 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:37.347513 15669 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:37.349800 16972 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:37.375106 16963 ts_tablet_manager.cc:1929] T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:37.375589 16963 log.cc:1199] T ced58b89726545d3af99de36a1aa1f30 P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-1-root/wals/ced58b89726545d3af99de36a1aa1f30
I20260709 13:18:37.377444 15412 catalog_manager.cc:5028] TS 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): tablet ced58b89726545d3af99de36a1aa1f30 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:37.427285 15596 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:18:37.427958 15596 raft_consensus.cc:606] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Received request to transfer leadership to TS dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:37.430498 15669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:18:37.430927 15669 raft_consensus.cc:606] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Received request to transfer leadership to TS dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:37.433372 15669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2afa8e112a744f218227526d861b98c3"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:18:37.433949 15669 raft_consensus.cc:606] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:37.437036 15526 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "974a060267d24f00888c0942dfbca77e"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
mode: GRACEFUL
new_leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:34862
I20260709 13:18:37.437644 15526 raft_consensus.cc:606] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Received request to transfer leadership to TS 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:37.441192 15596 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:18:37.441730 15596 raft_consensus.cc:606] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:37.442821 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:18:37.444260 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:37.505059 16894 raft_consensus.cc:993] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:37.505625 16896 raft_consensus.cc:1081] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Signalling peer dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:37.507115 16236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:57244
I20260709 13:18:37.507558 16236 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:37.507845 16236 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:37.513662 16236 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:37.515443 16236 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:37.516609 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:37.517233 15602 raft_consensus.cc:3055] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:37.517499 15602 raft_consensus.cc:740] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:37.518245 15602 consensus_queue.cc:260] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:37.519429 15602 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:37.516590 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:37.522447 15528 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:37.527182 15602 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:37.528347 16205 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:37.529131 15528 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:37.529309 16969 raft_consensus.cc:2804] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:37.529943 16969 raft_consensus.cc:697] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:37.530861 16969 consensus_queue.cc:237] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:37.540423 15425 catalog_manager.cc:5697] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 1 to 2, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 2 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:37.660930 16984 raft_consensus.cc:993] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69: : Instructing follower 5498ef3e89a941e39c25f68cf64e6b0f to start an election
I20260709 13:18:37.661415 16866 raft_consensus.cc:1081] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Signalling peer 5498ef3e89a941e39c25f68cf64e6b0f to start an election
I20260709 13:18:37.662910 15596 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "974a060267d24f00888c0942dfbca77e"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:53324
I20260709 13:18:37.663380 15596 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:37.663656 15596 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:37.669837 15596 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:37.671602 15596 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:37.672363 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:37.672569 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:37.672971 15528 raft_consensus.cc:3055] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:37.673063 15669 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:37.673230 15528 raft_consensus.cc:740] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:37.673977 15528 consensus_queue.cc:260] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:37.674873 15528 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:37.678279 15669 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:18:37.679507 15566 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:37.680325 16946 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:37.680783 16946 raft_consensus.cc:697] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:37.681681 15528 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:18:37.681411 16946 consensus_queue.cc:237] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:37.690961 15425 catalog_manager.cc:5697] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 2 to 3, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 3 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:37.730911 16896 raft_consensus.cc:993] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:37.731335 16946 raft_consensus.cc:1081] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Signalling peer 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:37.732723 15676 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:49100
I20260709 13:18:37.733158 15676 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:37.733430 15676 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:37.738289 15676 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:37.740087 15676 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:37.741299 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:37.741956 15595 raft_consensus.cc:3055] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:37.742291 15595 raft_consensus.cc:740] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:37.741305 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:37.743364 16236 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:37.743058 15595 consensus_queue.cc:260] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:37.744232 15595 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:37.750254 16236 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:37.751106 15595 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:37.751593 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:37.752354 16786 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:37.752717 16786 raft_consensus.cc:697] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:37.753505 16786 consensus_queue.cc:237] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:37.762918 15425 catalog_manager.cc:5697] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 1 to 2, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:37.772907 16980 raft_consensus.cc:993] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4: : Instructing follower dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:37.773331 16786 raft_consensus.cc:1081] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Signalling peer dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:37.774899 16235 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:53976
I20260709 13:18:37.775442 16235 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:37.775722 16235 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:37.780541 16980 raft_consensus.cc:993] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4: : Instructing follower d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:18:37.781018 16983 raft_consensus.cc:1081] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Signalling peer d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:18:37.782605 16235 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:37.783232 15528 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2afa8e112a744f218227526d861b98c3"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:57300
I20260709 13:18:37.783903 15528 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:37.784274 15528 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:37.785683 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:37.786319 15526 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:37.793273 15528 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:37.795168 15528 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:37.796581 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:37.797547 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:37.798136 15595 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:37.798467 15669 raft_consensus.cc:3055] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:37.798859 15669 raft_consensus.cc:740] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:37.799721 15669 consensus_queue.cc:260] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:37.802238 15669 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:37.805981 15595 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:37.807526 15669 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:37.807948 15493 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:37.808439 15526 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:37.808743 16984 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:37.809279 16984 raft_consensus.cc:697] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:37.809597 16235 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:37.810963 16984 consensus_queue.cc:237] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:37.811005 16204 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:37.814698 16969 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:37.815243 16969 raft_consensus.cc:697] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:37.816179 16969 consensus_queue.cc:237] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:37.818912 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:37.819497 15669 raft_consensus.cc:3055] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:37.819716 15669 raft_consensus.cc:740] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:37.820509 15669 consensus_queue.cc:260] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:37.825243 15669 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:37.826992 15425 catalog_manager.cc:5697] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 1 to 2, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 2 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:37.829954 15426 catalog_manager.cc:5697] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 1 to 2, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 2 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:37.831944 15669 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:38.033622 15595 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:38.034757 16969 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:38.045953 15528 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:38.047583 16969 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:38.128719 15526 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:38.130028 16946 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:38.132228 15669 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:38.133533 16946 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:38.246223 15602 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:38.247473 16786 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:38.257161 16235 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:38.259022 16980 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:38.263393 15669 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:38.264909 16969 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:38.278716 15528 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:38.280627 16969 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:38.298964 15602 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:38.301333 16866 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:38.319756 15669 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:38.326722 16972 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:38.397024 15602 consensus_queue.cc:237] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:38.402128 15528 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:18:38.402128 15669 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:18:38.403651 16894 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:18:38.404814 15568 consensus_peers.cc:597] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 974a060267d24f00888c0942dfbca77e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:38.405843 16894 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:38.410019 16896 raft_consensus.cc:2955] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:38.412011 15528 raft_consensus.cc:2955] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:38.416139 15669 raft_consensus.cc:2955] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:38.427686 15425 catalog_manager.cc:5697] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index -1 to 4, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 3 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:38.429886 15669 consensus_queue.cc:237] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:38.454509 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:38.475740 15602 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:38.475732 15526 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:18:38.476068 15642 consensus_peers.cc:597] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3502bb7169aa41bb9ebdaeddf5068af9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:38.478613 16983 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:38.481101 16983 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:38.493476 16786 raft_consensus.cc:2955] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:38.503873 15526 raft_consensus.cc:2955] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:38.513249 15669 consensus_queue.cc:237] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:38.508507 15602 raft_consensus.cc:2955] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:38.524195 15422 catalog_manager.cc:5697] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:38.533053 15528 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:18:38.535296 16983 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:38.537407 15602 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:18:38.539018 16983 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:18:38.542361 15642 consensus_peers.cc:597] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d6670c20b1954b36a4920b44d1a8209d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:38.562026 16983 raft_consensus.cc:2955] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:38.571640 15602 raft_consensus.cc:2955] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:38.577000 15526 raft_consensus.cc:2955] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:38.578313 15595 consensus_queue.cc:237] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:38.593565 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:38.590698 15425 catalog_manager.cc:5697] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index -1 to 3, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 2 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
W20260709 13:18:38.606057 15568 consensus_peers.cc:597] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b2c57bed49da41d795c8b21ed4461f1f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:38.606245 15674 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:18:38.611088 15526 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:18:38.611881 16896 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:38.613746 16896 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:38.632014 17001 raft_consensus.cc:2955] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:38.633347 15526 raft_consensus.cc:2955] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:38.637722 15674 raft_consensus.cc:2955] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:38.661695 15426 catalog_manager.cc:5697] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index -1 to 4, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 3 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:38.688841 15526 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
mode: GRACEFUL
new_leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:34862
I20260709 13:18:38.689407 15526 raft_consensus.cc:606] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Received request to transfer leadership to TS 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:38.693868 15602 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:18:38.694386 15602 raft_consensus.cc:606] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:38.695484 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:18:38.696564 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:38.741953 17001 raft_consensus.cc:993] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:38.742596 16896 raft_consensus.cc:1081] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Signalling peer 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:38.743974 15674 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:49100
I20260709 13:18:38.744554 15674 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:38.744979 15674 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:38.752360 15674 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:38.756107 15674 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:38.757195 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:38.757176 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:38.757882 15602 raft_consensus.cc:3055] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:38.758054 16235 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:38.758213 15602 raft_consensus.cc:740] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:38.759008 15602 consensus_queue.cc:260] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:38.759997 15602 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:38.765570 16235 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:38.766745 15602 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:38.768061 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:38.768702 16980 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:38.769168 16980 raft_consensus.cc:697] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:38.771641 16980 consensus_queue.cc:237] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:38.800103 15425 catalog_manager.cc:5697] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 1 to 2, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:38.865088 17024 ts_tablet_manager.cc:933] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:38.867019 17024 tablet_copy_client.cc:323] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.67:32779
I20260709 13:18:38.868986 15687 tablet_copy_service.cc:140] P 3ef186ec12554a1d87814af0a0ace9a4: Received BeginTabletCopySession request for tablet 3502bb7169aa41bb9ebdaeddf5068af9 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:57710)
I20260709 13:18:38.869540 15687 tablet_copy_service.cc:161] P 3ef186ec12554a1d87814af0a0ace9a4: Beginning new tablet copy session on tablet 3502bb7169aa41bb9ebdaeddf5068af9 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:57710: session id = dd1a14c3173d49dc9b55bb116e3783c9-3502bb7169aa41bb9ebdaeddf5068af9
I20260709 13:18:38.878109 15687 tablet_copy_source_session.cc:215] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:38.881716 17024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3502bb7169aa41bb9ebdaeddf5068af9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.899300 17024 tablet_copy_client.cc:806] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:38.899936 17024 tablet_copy_client.cc:670] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:38.905189 17024 tablet_copy_client.cc:538] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:38.911912 17024 tablet_bootstrap.cc:492] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:38.932938 17024 tablet_bootstrap.cc:492] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:38.933907 17024 tablet_bootstrap.cc:492] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:38.934623 17024 ts_tablet_manager.cc:1403] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.011s
I20260709 13:18:38.937677 17024 raft_consensus.cc:359] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:38.938539 17024 raft_consensus.cc:740] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:38.939213 17024 consensus_queue.cc:260] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:38.942073 17024 ts_tablet_manager.cc:1434] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:18:38.944007 15687 tablet_copy_service.cc:342] P 3ef186ec12554a1d87814af0a0ace9a4: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-3502bb7169aa41bb9ebdaeddf5068af9 received from {username='slave'} at 127.0.0.1:57710
I20260709 13:18:38.944504 15687 tablet_copy_service.cc:434] P 3ef186ec12554a1d87814af0a0ace9a4: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-3502bb7169aa41bb9ebdaeddf5068af9 on tablet 3502bb7169aa41bb9ebdaeddf5068af9 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:38.948443 17024 ts_tablet_manager.cc:933] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:38.950067 17024 tablet_copy_client.cc:323] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.66:41593
I20260709 13:18:38.953092 15612 tablet_copy_service.cc:140] P 5498ef3e89a941e39c25f68cf64e6b0f: Received BeginTabletCopySession request for tablet 974a060267d24f00888c0942dfbca77e from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:34102)
I20260709 13:18:38.953648 15612 tablet_copy_service.cc:161] P 5498ef3e89a941e39c25f68cf64e6b0f: Beginning new tablet copy session on tablet 974a060267d24f00888c0942dfbca77e from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:34102: session id = dd1a14c3173d49dc9b55bb116e3783c9-974a060267d24f00888c0942dfbca77e
I20260709 13:18:38.961581 15612 tablet_copy_source_session.cc:215] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:38.967007 17024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 974a060267d24f00888c0942dfbca77e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.983664 17024 tablet_copy_client.cc:806] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:38.984225 17024 tablet_copy_client.cc:670] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:38.989495 17024 tablet_copy_client.cc:538] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:38.993108 17027 ts_tablet_manager.cc:933] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:39.002334 17024 tablet_bootstrap.cc:492] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:39.006757 17027 tablet_copy_client.cc:323] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.67:32779
I20260709 13:18:39.046460 15687 tablet_copy_service.cc:140] P 3ef186ec12554a1d87814af0a0ace9a4: Received BeginTabletCopySession request for tablet d6670c20b1954b36a4920b44d1a8209d from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:57710)
I20260709 13:18:39.047046 15687 tablet_copy_service.cc:161] P 3ef186ec12554a1d87814af0a0ace9a4: Beginning new tablet copy session on tablet d6670c20b1954b36a4920b44d1a8209d from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:57710: session id = dd1a14c3173d49dc9b55bb116e3783c9-d6670c20b1954b36a4920b44d1a8209d
I20260709 13:18:39.055691 15687 tablet_copy_source_session.cc:215] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:39.059487 17027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6670c20b1954b36a4920b44d1a8209d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.081346 17027 tablet_copy_client.cc:806] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:39.081943 17027 tablet_copy_client.cc:670] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:39.087934 17027 tablet_copy_client.cc:538] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:39.097592 17024 tablet_bootstrap.cc:492] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:39.097939 17027 tablet_bootstrap.cc:492] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:39.098558 17024 tablet_bootstrap.cc:492] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:39.099339 17024 ts_tablet_manager.cc:1403] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.097s	user 0.029s	sys 0.006s
I20260709 13:18:39.108613 17024 raft_consensus.cc:359] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:39.109257 17024 raft_consensus.cc:740] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:39.109779 17024 consensus_queue.cc:260] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:39.112391 17024 ts_tablet_manager.cc:1434] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:18:39.114820 15612 tablet_copy_service.cc:342] P 5498ef3e89a941e39c25f68cf64e6b0f: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-974a060267d24f00888c0942dfbca77e received from {username='slave'} at 127.0.0.1:34102
I20260709 13:18:39.115275 15612 tablet_copy_service.cc:434] P 5498ef3e89a941e39c25f68cf64e6b0f: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-974a060267d24f00888c0942dfbca77e on tablet 974a060267d24f00888c0942dfbca77e with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:39.137892 17027 tablet_bootstrap.cc:492] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:39.138852 17027 tablet_bootstrap.cc:492] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:39.139495 17027 ts_tablet_manager.cc:1403] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.042s	user 0.027s	sys 0.009s
I20260709 13:18:39.142068 17027 raft_consensus.cc:359] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:39.142688 17027 raft_consensus.cc:740] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:39.143325 17027 consensus_queue.cc:260] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:39.145802 17027 ts_tablet_manager.cc:1434] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:18:39.148352 15687 tablet_copy_service.cc:342] P 3ef186ec12554a1d87814af0a0ace9a4: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-d6670c20b1954b36a4920b44d1a8209d received from {username='slave'} at 127.0.0.1:57710
I20260709 13:18:39.148813 15687 tablet_copy_service.cc:434] P 3ef186ec12554a1d87814af0a0ace9a4: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-d6670c20b1954b36a4920b44d1a8209d on tablet d6670c20b1954b36a4920b44d1a8209d with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:39.176424 17027 ts_tablet_manager.cc:933] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:39.178404 17027 tablet_copy_client.cc:323] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.66:41593
I20260709 13:18:39.179966 15612 tablet_copy_service.cc:140] P 5498ef3e89a941e39c25f68cf64e6b0f: Received BeginTabletCopySession request for tablet b2c57bed49da41d795c8b21ed4461f1f from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:34102)
I20260709 13:18:39.180434 15612 tablet_copy_service.cc:161] P 5498ef3e89a941e39c25f68cf64e6b0f: Beginning new tablet copy session on tablet b2c57bed49da41d795c8b21ed4461f1f from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:34102: session id = dd1a14c3173d49dc9b55bb116e3783c9-b2c57bed49da41d795c8b21ed4461f1f
I20260709 13:18:39.185806 15612 tablet_copy_source_session.cc:215] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:39.189050 17027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2c57bed49da41d795c8b21ed4461f1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.208094 17027 tablet_copy_client.cc:806] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:39.208680 17027 tablet_copy_client.cc:670] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:39.213668 17027 tablet_copy_client.cc:538] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:39.220391 16984 raft_consensus.cc:993] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69: : Instructing follower 5498ef3e89a941e39c25f68cf64e6b0f to start an election
I20260709 13:18:39.221426 16972 raft_consensus.cc:1081] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Signalling peer 5498ef3e89a941e39c25f68cf64e6b0f to start an election
I20260709 13:18:39.223575 15602 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:53324
I20260709 13:18:39.224226 15602 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:39.224583 15602 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.223560 17027 tablet_bootstrap.cc:492] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:39.231706 15602 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:39.234912 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:39.235560 16235 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.236086 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:39.236686 15526 raft_consensus.cc:3055] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:39.236989 15526 raft_consensus.cc:740] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:39.237766 15526 consensus_queue.cc:260] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:39.238896 15526 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.250859 15526 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:39.252457 15567 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:39.253062 15602 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:39.253482 16946 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:39.252782 16235 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:39.254158 16946 raft_consensus.cc:697] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:39.256636 16946 consensus_queue.cc:237] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:39.275640 17027 tablet_bootstrap.cc:492] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:39.276935 17027 tablet_bootstrap.cc:492] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:39.277773 17027 ts_tablet_manager.cc:1403] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.054s	user 0.019s	sys 0.015s
I20260709 13:18:39.281265 17027 raft_consensus.cc:359] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:39.282269 17027 raft_consensus.cc:740] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:39.282994 17027 consensus_queue.cc:260] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:39.292966 17027 ts_tablet_manager.cc:1434] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:18:39.295693 15612 tablet_copy_service.cc:342] P 5498ef3e89a941e39c25f68cf64e6b0f: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-b2c57bed49da41d795c8b21ed4461f1f received from {username='slave'} at 127.0.0.1:34102
I20260709 13:18:39.296160 15612 tablet_copy_service.cc:434] P 5498ef3e89a941e39c25f68cf64e6b0f: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-b2c57bed49da41d795c8b21ed4461f1f on tablet b2c57bed49da41d795c8b21ed4461f1f with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:39.307247 15602 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:39.310259 15426 catalog_manager.cc:5697] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 1 to 2, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 2 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:39.331698 17037 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:39.342832 16236 raft_consensus.cc:1217] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:39.365185 16236 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:39.368367 16983 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:39.439343 16235 raft_consensus.cc:1217] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 13:18:39.448012 16235 raft_consensus.cc:1217] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:18:39.622967 16235 raft_consensus.cc:1217] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 13:18:39.697496 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:39.739423 15523 raft_consensus.cc:1275] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:39.740836 17001 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:39.750329 16235 raft_consensus.cc:1275] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:39.751992 16946 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:39.848524 17039 raft_consensus.cc:1064] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:39.850948 17039 consensus_queue.cc:237] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:39.858882 16235 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEARNER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:18:39.861016 15523 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:18:39.862298 15595 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:18:39.862576 17039 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:39.864773 16983 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:39.865754 17036 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:39.889142 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:39.898912 17036 raft_consensus.cc:1064] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:39.901808 17036 consensus_queue.cc:237] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:39.902889 17037 raft_consensus.cc:2955] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:39.908931 15523 raft_consensus.cc:2955] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:39.914214 15595 raft_consensus.cc:2955] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:39.932898 15523 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:39.934881 16983 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:39.933352 15422 catalog_manager.cc:5697] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index 3 to 4, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:39.947232 16235 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:39.952033 16235 raft_consensus.cc:2955] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:39.953626 15595 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:39.963398 16896 raft_consensus.cc:1064] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:39.965724 16896 consensus_queue.cc:237] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:39.970470 17036 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:39.987511 17055 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:39.991881 15674 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:39.993857 16946 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:40.013502 15523 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:40.018635 17001 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:40.031347 16235 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEARNER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:40.033139 16946 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:40.045042 15602 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:18:40.045643 15602 raft_consensus.cc:606] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:40.051997 15676 consensus_queue.cc:237] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:40.057042 15669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "50726e0d396948b79be43aece292acb5"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:18:40.057674 15669 raft_consensus.cc:606] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:40.059378 16237 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:40.060595 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:18:40.060892 17055 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:40.062237 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:40.076514 17056 raft_consensus.cc:2955] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.082880 16896 raft_consensus.cc:1064] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:40.084906 15528 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:40.089327 16237 raft_consensus.cc:2955] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.092136 17055 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:40.092053 16896 consensus_queue.cc:237] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:40.098140 15528 raft_consensus.cc:2955] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.135574 15424 catalog_manager.cc:5697] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 4 to 5, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New cstate: current_term: 2 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:18:40.138814 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 85609.7us
I20260709 13:18:40.142901 16946 raft_consensus.cc:2955] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.158733 15589 tablet_service.cc:1558] Processing DeleteTablet for tablet d6670c20b1954b36a4920b44d1a8209d with delete_type TABLET_DATA_TOMBSTONED (TS 5498ef3e89a941e39c25f68cf64e6b0f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:53300
I20260709 13:18:40.163573 15410 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d6670c20b1954b36a4920b44d1a8209d with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:40.178170 17039 raft_consensus.cc:2955] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.181448 17070 tablet_replica.cc:333] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:18:40.182513 17070 raft_consensus.cc:2243] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:40.189110 17070 raft_consensus.cc:2272] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:40.190208 15595 raft_consensus.cc:2955] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.192241 16236 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEARNER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:40.195202 15528 raft_consensus.cc:2955] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.195382 15669 raft_consensus.cc:2955] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.203214 17062 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:40.204203 15674 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:40.206864 17001 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:40.210867 15526 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:40.212790 15523 raft_consensus.cc:2955] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.216413 15426 catalog_manager.cc:5697] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.229321 17001 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:40.185099 16237 raft_consensus.cc:2955] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.199437 17070 ts_tablet_manager.cc:1916] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:40.247601 15426 catalog_manager.cc:5697] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 4 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:40.253644 16234 raft_consensus.cc:2955] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.262830 15676 raft_consensus.cc:2955] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.265134 17068 raft_consensus.cc:2955] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.288796 15596 consensus_queue.cc:237] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:40.313160 15669 consensus_queue.cc:237] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:40.315145 15528 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:40.315678 16234 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:40.317018 17070 ts_tablet_manager.cc:1929] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:18:40.322746 15424 catalog_manager.cc:5697] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 4 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:40.323787 17070 log.cc:1199] T d6670c20b1954b36a4920b44d1a8209d P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-1-root/wals/d6670c20b1954b36a4920b44d1a8209d
I20260709 13:18:40.326483 16235 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:40.325506 17073 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:18:40.338078 15528 raft_consensus.cc:2955] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.338343 16237 raft_consensus.cc:2955] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.332929 16983 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:18:40.352180 16983 raft_consensus.cc:2955] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.357419 16236 raft_consensus.cc:2955] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.363858 15410 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3502bb7169aa41bb9ebdaeddf5068af9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:40.373272 15424 catalog_manager.cc:5697] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:40.378211 17072 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:40.385941 15596 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:40.387259 16946 raft_consensus.cc:993] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:18:40.388052 17001 raft_consensus.cc:1081] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Signalling peer d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:18:40.387827 16946 raft_consensus.cc:2955] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.398640 17056 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:40.400735 15412 catalog_manager.cc:5028] TS 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): tablet d6670c20b1954b36a4920b44d1a8209d (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:40.406945 15514 tablet_service.cc:1558] Processing DeleteTablet for tablet 3502bb7169aa41bb9ebdaeddf5068af9 with delete_type TABLET_DATA_TOMBSTONED (TS d3c4f5bd76e0445795bad5f6dd93ff69 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57292
I20260709 13:18:40.408478 15519 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:57310
I20260709 13:18:40.409154 15519 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:40.412328 15528 raft_consensus.cc:2955] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.413615 17078 tablet_replica.cc:333] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:18:40.414741 17078 raft_consensus.cc:2243] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:40.419451 15412 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 974a060267d24f00888c0942dfbca77e with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:40.409597 15519 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:40.416267 16236 raft_consensus.cc:2955] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.417384 15595 raft_consensus.cc:2955] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.419840 17078 raft_consensus.cc:2272] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:40.442371 17056 raft_consensus.cc:993] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4: : Instructing follower d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:18:40.442938 17056 raft_consensus.cc:1081] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Signalling peer d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:18:40.444470 15523 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "50726e0d396948b79be43aece292acb5"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:57300
I20260709 13:18:40.444986 15523 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:40.445292 15523 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:40.450347 15425 catalog_manager.cc:5697] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 5 to 6, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New cstate: current_term: 3 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:40.452643 15523 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:40.456766 15523 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:40.457418 17078 ts_tablet_manager.cc:1916] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:40.470831 15660 tablet_service.cc:1558] Processing DeleteTablet for tablet 974a060267d24f00888c0942dfbca77e with delete_type TABLET_DATA_TOMBSTONED (TS 3ef186ec12554a1d87814af0a0ace9a4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:49088
I20260709 13:18:40.471072 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:40.471697 15674 raft_consensus.cc:3055] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:40.472028 15674 raft_consensus.cc:740] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:40.472784 15674 consensus_queue.cc:260] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:40.474849 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:40.475453 15597 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:40.480916 17078 ts_tablet_manager.cc:1929] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:40.481411 17078 log.cc:1199] T 3502bb7169aa41bb9ebdaeddf5068af9 P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-0-root/wals/3502bb7169aa41bb9ebdaeddf5068af9
I20260709 13:18:40.483126 15411 catalog_manager.cc:5028] TS d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): tablet 3502bb7169aa41bb9ebdaeddf5068af9 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:40.483924 15674 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:40.486424 15596 consensus_queue.cc:237] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:40.483983 15597 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:40.492373 15674 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:40.496024 15491 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:40.497514 16984 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:40.498150 16984 raft_consensus.cc:697] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:40.499106 16984 consensus_queue.cc:237] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:40.524650 17079 tablet_replica.cc:333] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:18:40.483378 15519 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:40.528317 15519 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:40.533612 15519 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:40.535282 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:40.535938 16236 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:40.536720 17079 raft_consensus.cc:2243] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:40.540591 17079 raft_consensus.cc:2272] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:40.541515 16237 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:40.542692 16236 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:40.544044 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:40.545015 16866 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:40.545445 16866 raft_consensus.cc:697] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:40.543151 15422 catalog_manager.cc:5697] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 2 to 3, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 3 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:40.546212 16866 consensus_queue.cc:237] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:40.537420 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:40.538221 17073 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:40.549353 15596 raft_consensus.cc:3055] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:40.549499 17079 ts_tablet_manager.cc:1916] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:40.553105 17073 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:40.557406 17071 raft_consensus.cc:2955] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.562606 15596 raft_consensus.cc:740] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:40.566428 16237 raft_consensus.cc:2955] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.566644 15519 raft_consensus.cc:2955] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:40.577422 15596 consensus_queue.cc:260] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:40.578549 15412 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b2c57bed49da41d795c8b21ed4461f1f with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:40.580111 17079 ts_tablet_manager.cc:1929] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:18:40.580542 17079 log.cc:1199] T 974a060267d24f00888c0942dfbca77e P 3ef186ec12554a1d87814af0a0ace9a4: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-2-root/wals/974a060267d24f00888c0942dfbca77e
I20260709 13:18:40.581012 15596 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:40.580807 15422 catalog_manager.cc:5697] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 5 to 6, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New cstate: current_term: 3 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:40.582666 15410 catalog_manager.cc:5028] TS 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): tablet 974a060267d24f00888c0942dfbca77e (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:40.596727 15660 tablet_service.cc:1558] Processing DeleteTablet for tablet b2c57bed49da41d795c8b21ed4461f1f with delete_type TABLET_DATA_TOMBSTONED (TS 3ef186ec12554a1d87814af0a0ace9a4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:49088
I20260709 13:18:40.597586 15596 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:40.600553 17079 tablet_replica.cc:333] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:18:40.599577 15422 catalog_manager.cc:5697] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 1 to 2, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 2 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:40.601673 17079 raft_consensus.cc:2243] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:40.602596 17079 raft_consensus.cc:2272] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:40.607470 17079 ts_tablet_manager.cc:1916] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:40.623566 17079 ts_tablet_manager.cc:1929] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:18:40.624027 17079 log.cc:1199] T b2c57bed49da41d795c8b21ed4461f1f P 3ef186ec12554a1d87814af0a0ace9a4: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-2-root/wals/b2c57bed49da41d795c8b21ed4461f1f
I20260709 13:18:40.625419 15410 catalog_manager.cc:5028] TS 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): tablet b2c57bed49da41d795c8b21ed4461f1f (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:40.977195 15596 raft_consensus.cc:1275] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:40.978534 16866 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:40.983786 16237 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:40.986109 16866 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:40.991750 15674 raft_consensus.cc:1275] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:40.993201 17084 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:41.012503 15595 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:41.013818 17086 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:41.063159 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:41.356458 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:41.391582 15676 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:18:41.392184 15676 raft_consensus.cc:606] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Received request to transfer leadership to TS 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:41.395148 15523 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
mode: GRACEFUL
new_leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:34862
I20260709 13:18:41.395743 15523 raft_consensus.cc:606] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Received request to transfer leadership to TS dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:41.399605 15519 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5243f722506441269fac66aa989ce36d"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:34862
I20260709 13:18:41.400182 15519 raft_consensus.cc:606] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:41.401471 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:18:41.402580 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:41.657724 15676 consensus_queue.cc:237] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
W20260709 13:18:41.667744 15642 consensus_peers.cc:597] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4b73eb6a0096455b87f6109b059cf0ee. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:41.670910 15519 raft_consensus.cc:1275] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:41.671720 15595 raft_consensus.cc:1275] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:41.672434 17095 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:41.673418 17056 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:41.682047 17056 raft_consensus.cc:2955] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:41.683856 15595 raft_consensus.cc:2955] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:41.692461 15519 raft_consensus.cc:2955] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:41.696735 15523 consensus_queue.cc:237] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:41.703835 15424 catalog_manager.cc:5697] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:41.715523 15674 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:18:41.715893 15493 consensus_peers.cc:597] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 40f71e7bfe1a4575846bcddbb0b9fd55. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:41.715986 15595 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:18:41.719079 17086 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:41.719988 17084 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:41.733038 17084 raft_consensus.cc:2955] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:41.735101 15676 raft_consensus.cc:2955] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:41.743685 15595 raft_consensus.cc:2955] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:41.754340 15523 consensus_queue.cc:237] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:41.762554 15595 raft_consensus.cc:1275] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:41.762863 15674 raft_consensus.cc:1275] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:41.756403 15425 catalog_manager.cc:5697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index -1 to 4, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 3 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:41.764103 17086 consensus_queue.cc:1048] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:18:41.764955 15493 consensus_peers.cc:597] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8153c7c094654fb492f5c47863ad170d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:41.764992 17084 consensus_queue.cc:1048] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:41.777033 17084 raft_consensus.cc:993] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69: : Instructing follower 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:41.778486 17086 raft_consensus.cc:1081] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Signalling peer 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:41.783406 15676 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5243f722506441269fac66aa989ce36d"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:49094
I20260709 13:18:41.784057 15676 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:41.784533 15676 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:41.789825 15676 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:41.792117 15676 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:41.790846 16984 raft_consensus.cc:2955] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:41.793709 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:41.794313 15596 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:41.796977 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:41.797550 15523 raft_consensus.cc:3055] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:41.798609 15523 raft_consensus.cc:740] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:41.799347 15523 consensus_queue.cc:260] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:41.800542 15523 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:41.800400 15676 raft_consensus.cc:2955] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:41.802591 15596 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:41.803774 15642 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:41.804404 17056 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:41.804874 17056 raft_consensus.cc:697] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:41.805691 17056 consensus_queue.cc:237] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:41.815193 15595 raft_consensus.cc:2955] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:41.827239 15523 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:41.840931 15523 consensus_queue.cc:237] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
W20260709 13:18:41.858671 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 46251.6us
I20260709 13:18:41.865391 15424 catalog_manager.cc:5697] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 1 to 2, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:41.857894 15425 catalog_manager.cc:5697] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
W20260709 13:18:41.877367 15493 consensus_peers.cc:597] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2afa8e112a744f218227526d861b98c3. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:41.880038 15596 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:18:41.883538 15674 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:18:41.885172 17095 raft_consensus.cc:993] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4: : Instructing follower 5498ef3e89a941e39c25f68cf64e6b0f to start an election
I20260709 13:18:41.885699 17095 raft_consensus.cc:1081] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Signalling peer 5498ef3e89a941e39c25f68cf64e6b0f to start an election
I20260709 13:18:41.886718 16984 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:41.890515 15596 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:53310
I20260709 13:18:41.891144 15596 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:41.891597 15596 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:41.894035 16984 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:41.900447 15596 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:18:41.904134 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:41.905386 15674 raft_consensus.cc:3055] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:41.904495 17109 raft_consensus.cc:2955] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:41.906862 15595 raft_consensus.cc:2955] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:41.906463 15674 raft_consensus.cc:740] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:41.909284 15674 consensus_queue.cc:260] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:18:41.910934 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:41.911196 15674 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:41.915585 15523 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:41.930297 15425 catalog_manager.cc:5697] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index -1 to 3, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 2 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:41.908036 15596 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:18:41.933557 15523 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:18:41.930406 15674 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:18:41.935901 15567 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:41.936728 16946 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:41.937156 16946 raft_consensus.cc:697] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:41.937911 16946 consensus_queue.cc:237] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:18:41.940694 15676 raft_consensus.cc:2955] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:41.954527 15422 catalog_manager.cc:5697] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 2 to 3, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 3 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:41.997604 16984 raft_consensus.cc:993] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69: : Instructing follower dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:41.998131 17109 raft_consensus.cc:1081] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Signalling peer dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:41.999570 16236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:57258
I20260709 13:18:41.999994 16236 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:42.000236 16236 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:42.005692 16236 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:42.007773 16236 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:42.008520 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:42.009052 15526 raft_consensus.cc:3055] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:42.009174 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:42.009491 15526 raft_consensus.cc:740] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:42.009951 15676 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:42.010290 15526 consensus_queue.cc:260] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:42.011586 15526 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:42.016520 15676 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:42.017895 15526 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:42.017884 16203 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:42.018926 17076 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:42.019397 17076 raft_consensus.cc:697] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:42.020182 17076 consensus_queue.cc:237] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:42.030421 15424 catalog_manager.cc:5697] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 1 to 2, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 2 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:42.168520 17116 ts_tablet_manager.cc:933] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:42.170488 17116 tablet_copy_client.cc:323] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.67:32779
I20260709 13:18:42.172127 15687 tablet_copy_service.cc:140] P 3ef186ec12554a1d87814af0a0ace9a4: Received BeginTabletCopySession request for tablet 4b73eb6a0096455b87f6109b059cf0ee from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:57710)
I20260709 13:18:42.172694 15687 tablet_copy_service.cc:161] P 3ef186ec12554a1d87814af0a0ace9a4: Beginning new tablet copy session on tablet 4b73eb6a0096455b87f6109b059cf0ee from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:57710: session id = dd1a14c3173d49dc9b55bb116e3783c9-4b73eb6a0096455b87f6109b059cf0ee
I20260709 13:18:42.183590 15687 tablet_copy_source_session.cc:215] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:42.188272 17116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b73eb6a0096455b87f6109b059cf0ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.213658 17116 tablet_copy_client.cc:806] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:42.214251 17116 tablet_copy_client.cc:670] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:42.218223 17116 tablet_copy_client.cc:538] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:42.226272 17116 tablet_bootstrap.cc:492] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:42.235694 15526 raft_consensus.cc:1275] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:42.237844 16983 consensus_queue.cc:1048] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:42.249361 15596 raft_consensus.cc:1275] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:42.250543 17116 tablet_bootstrap.cc:492] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:42.251530 17116 tablet_bootstrap.cc:492] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:42.252276 17116 ts_tablet_manager.cc:1403] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.005s
I20260709 13:18:42.256101 17116 raft_consensus.cc:359] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:42.257006 17116 raft_consensus.cc:740] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:42.257675 17116 consensus_queue.cc:260] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:42.260674 17116 ts_tablet_manager.cc:1434] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:18:42.269369 17056 consensus_queue.cc:1048] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:42.273088 15687 tablet_copy_service.cc:342] P 3ef186ec12554a1d87814af0a0ace9a4: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-4b73eb6a0096455b87f6109b059cf0ee received from {username='slave'} at 127.0.0.1:57710
I20260709 13:18:42.273543 15687 tablet_copy_service.cc:434] P 3ef186ec12554a1d87814af0a0ace9a4: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-4b73eb6a0096455b87f6109b059cf0ee on tablet 4b73eb6a0096455b87f6109b059cf0ee with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:42.277196 17116 ts_tablet_manager.cc:933] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:18:42.291014 17122 ts_tablet_manager.cc:933] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:18:42.295002 17123 ts_tablet_manager.cc:933] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:18:42.314561 17123 tablet_copy_client.cc:323] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.65:42481
I20260709 13:18:42.316820 15538 tablet_copy_service.cc:140] P d3c4f5bd76e0445795bad5f6dd93ff69: Received BeginTabletCopySession request for tablet 8153c7c094654fb492f5c47863ad170d from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:56840)
I20260709 13:18:42.317384 15538 tablet_copy_service.cc:161] P d3c4f5bd76e0445795bad5f6dd93ff69: Beginning new tablet copy session on tablet 8153c7c094654fb492f5c47863ad170d from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:56840: session id = dd1a14c3173d49dc9b55bb116e3783c9-8153c7c094654fb492f5c47863ad170d
I20260709 13:18:42.326234 15538 tablet_copy_source_session.cc:215] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:42.327865 17116 tablet_copy_client.cc:323] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.65:42481
I20260709 13:18:42.329396 17122 tablet_copy_client.cc:323] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.65:42481
I20260709 13:18:42.329898 15538 tablet_copy_service.cc:140] P d3c4f5bd76e0445795bad5f6dd93ff69: Received BeginTabletCopySession request for tablet 2afa8e112a744f218227526d861b98c3 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:56840)
I20260709 13:18:42.330482 15538 tablet_copy_service.cc:161] P d3c4f5bd76e0445795bad5f6dd93ff69: Beginning new tablet copy session on tablet 2afa8e112a744f218227526d861b98c3 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:56840: session id = dd1a14c3173d49dc9b55bb116e3783c9-2afa8e112a744f218227526d861b98c3
I20260709 13:18:42.331019 15537 tablet_copy_service.cc:140] P d3c4f5bd76e0445795bad5f6dd93ff69: Received BeginTabletCopySession request for tablet 40f71e7bfe1a4575846bcddbb0b9fd55 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:56840)
I20260709 13:18:42.331562 15537 tablet_copy_service.cc:161] P d3c4f5bd76e0445795bad5f6dd93ff69: Beginning new tablet copy session on tablet 40f71e7bfe1a4575846bcddbb0b9fd55 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:56840: session id = dd1a14c3173d49dc9b55bb116e3783c9-40f71e7bfe1a4575846bcddbb0b9fd55
I20260709 13:18:42.331722 17123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8153c7c094654fb492f5c47863ad170d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.339000 15538 tablet_copy_source_session.cc:215] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:42.339915 15537 tablet_copy_source_session.cc:215] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:42.344344 17116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2afa8e112a744f218227526d861b98c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.349750 17122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40f71e7bfe1a4575846bcddbb0b9fd55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.357599 17123 tablet_copy_client.cc:806] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:42.358368 17123 tablet_copy_client.cc:670] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:42.372395 17116 tablet_copy_client.cc:806] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:42.373107 17116 tablet_copy_client.cc:670] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:42.384371 17122 tablet_copy_client.cc:806] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:42.386027 17123 tablet_copy_client.cc:538] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:42.398947 17116 tablet_copy_client.cc:538] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:42.400624 17123 tablet_bootstrap.cc:492] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:42.403369 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:42.409233 17116 tablet_bootstrap.cc:492] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:42.385416 17122 tablet_copy_client.cc:670] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:42.414541 17122 tablet_copy_client.cc:538] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:42.430982 15526 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:42.432654 17122 tablet_bootstrap.cc:492] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:42.433633 17076 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:42.439905 17123 tablet_bootstrap.cc:492] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:42.441187 17123 tablet_bootstrap.cc:492] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:42.441998 17123 ts_tablet_manager.cc:1403] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.042s	user 0.022s	sys 0.004s
I20260709 13:18:42.448797 17123 raft_consensus.cc:359] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:42.451386 15676 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:42.454123 17073 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:42.470242 17123 raft_consensus.cc:740] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:42.475165 17123 consensus_queue.cc:260] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:42.477658 17123 ts_tablet_manager.cc:1434] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.035s	user 0.004s	sys 0.001s
I20260709 13:18:42.483356 15537 tablet_copy_service.cc:342] P d3c4f5bd76e0445795bad5f6dd93ff69: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-8153c7c094654fb492f5c47863ad170d received from {username='slave'} at 127.0.0.1:56840
I20260709 13:18:42.483886 15537 tablet_copy_service.cc:434] P d3c4f5bd76e0445795bad5f6dd93ff69: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-8153c7c094654fb492f5c47863ad170d on tablet 8153c7c094654fb492f5c47863ad170d with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:42.496086 15526 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:42.501889 17073 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:42.533012 17122 tablet_bootstrap.cc:492] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:42.540885 17122 tablet_bootstrap.cc:492] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:42.541730 17122 ts_tablet_manager.cc:1403] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.110s	user 0.025s	sys 0.014s
W20260709 13:18:42.554602 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 44482.2us
W20260709 13:18:42.562144 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 55755.8us
I20260709 13:18:42.642020 17122 raft_consensus.cc:359] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:42.645004 17122 raft_consensus.cc:740] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:42.646834 17122 consensus_queue.cc:260] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:42.651171 17122 ts_tablet_manager.cc:1434] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.109s	user 0.018s	sys 0.006s
I20260709 13:18:42.656983 16237 raft_consensus.cc:1217] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:42.667290 15537 tablet_copy_service.cc:342] P d3c4f5bd76e0445795bad5f6dd93ff69: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-40f71e7bfe1a4575846bcddbb0b9fd55 received from {username='slave'} at 127.0.0.1:56840
I20260709 13:18:42.667796 15537 tablet_copy_service.cc:434] P d3c4f5bd76e0445795bad5f6dd93ff69: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-40f71e7bfe1a4575846bcddbb0b9fd55 on tablet 40f71e7bfe1a4575846bcddbb0b9fd55 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:42.675149 15669 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:42.685837 16235 raft_consensus.cc:1217] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:42.690160 17093 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:42.711755 17116 tablet_bootstrap.cc:492] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:42.726511 17116 tablet_bootstrap.cc:492] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:42.746747 17116 ts_tablet_manager.cc:1403] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 0.338s	user 0.021s	sys 0.007s
I20260709 13:18:42.762349 17116 raft_consensus.cc:359] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:42.765859 17116 raft_consensus.cc:740] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:42.774044 17116 consensus_queue.cc:260] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:42.776676 17116 ts_tablet_manager.cc:1434] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.029s	user 0.003s	sys 0.002s
I20260709 13:18:42.788863 15537 tablet_copy_service.cc:342] P d3c4f5bd76e0445795bad5f6dd93ff69: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-2afa8e112a744f218227526d861b98c3 received from {username='slave'} at 127.0.0.1:56840
I20260709 13:18:42.789300 15537 tablet_copy_service.cc:434] P d3c4f5bd76e0445795bad5f6dd93ff69: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-2afa8e112a744f218227526d861b98c3 on tablet 2afa8e112a744f218227526d861b98c3 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:42.809808 16237 raft_consensus.cc:1217] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 13:18:43.025353 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:43.069185 15596 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:18:43.070367 15596 raft_consensus.cc:606] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:43.072985 15674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4fd164d74054954b750fe9958e66778"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:18:43.073575 15674 raft_consensus.cc:606] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:43.074903 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:18:43.076562 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:43.092913 16237 raft_consensus.cc:1217] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:18:43.134671 17139 raft_consensus.cc:993] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:43.135246 17073 raft_consensus.cc:1081] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Signalling peer 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:43.136863 15674 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:49100
I20260709 13:18:43.137311 15674 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:43.137599 15674 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:43.142246 15674 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:18:43.144052 15674 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:43.145013 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:43.145601 15596 raft_consensus.cc:3055] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:43.145864 15596 raft_consensus.cc:740] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:43.146552 15596 consensus_queue.cc:260] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:18:43.145012 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:43.147559 15528 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:43.147609 15596 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:43.154593 15596 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:18:43.157415 15642 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:43.155198 15528 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:18:43.160755 17056 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:43.161506 17056 raft_consensus.cc:697] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:43.162343 17056 consensus_queue.cc:237] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:18:43.173254 15424 catalog_manager.cc:5697] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 3 to 4, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 4 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:43.200810 16983 raft_consensus.cc:1064] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:43.203202 16983 consensus_queue.cc:237] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:43.222595 15596 raft_consensus.cc:1275] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:43.228564 17111 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:43.230489 17084 raft_consensus.cc:1064] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:43.230589 15526 raft_consensus.cc:1275] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:43.233274 16237 raft_consensus.cc:1275] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:43.233355 17084 consensus_queue.cc:237] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:43.235312 17095 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:43.243444 17056 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:43.252666 16984 raft_consensus.cc:1064] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:43.255617 16984 consensus_queue.cc:237] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:43.311736 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 68805.3us
W20260709 13:18:43.326103 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 59579.6us
I20260709 13:18:43.341648 17095 raft_consensus.cc:993] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4: : Instructing follower d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:18:43.342253 17095 raft_consensus.cc:1081] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Signalling peer d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:18:43.349644 17095 raft_consensus.cc:2955] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.352331 16237 raft_consensus.cc:1275] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:43.354051 15528 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4fd164d74054954b750fe9958e66778"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:57300
I20260709 13:18:43.354475 15674 raft_consensus.cc:1275] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:43.354625 15528 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:43.354980 15528 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:43.358239 15597 raft_consensus.cc:2955] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.359380 17086 consensus_queue.cc:1048] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:43.361968 15528 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:43.364311 15528 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:43.367802 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:43.368345 15596 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:43.370661 17086 consensus_queue.cc:1048] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:43.372859 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:43.373458 15669 raft_consensus.cc:3055] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:43.373742 15669 raft_consensus.cc:740] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:43.374511 15669 consensus_queue.cc:260] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:43.375636 15669 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:43.376819 15596 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:43.383049 15669 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:43.383006 15595 raft_consensus.cc:1275] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:43.385831 17153 raft_consensus.cc:2955] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.388530 15493 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:43.389317 17084 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:43.390802 15674 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:43.390733 17084 raft_consensus.cc:697] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:43.392186 17150 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:43.392587 16237 raft_consensus.cc:2955] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.395129 17084 consensus_queue.cc:237] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:43.415740 15422 catalog_manager.cc:5697] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:43.419005 15676 raft_consensus.cc:2955] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.426690 17086 consensus_queue.cc:1048] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:43.433718 16237 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEARNER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:43.433511 15426 catalog_manager.cc:5697] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 2 to 3, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 3 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:43.434798 15597 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:43.442514 15526 raft_consensus.cc:2955] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.438910 15676 consensus_queue.cc:237] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:43.447379 16237 raft_consensus.cc:2955] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.456553 16235 raft_consensus.cc:1275] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:43.459424 17095 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:43.462358 15597 raft_consensus.cc:1275] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:43.465909 17095 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:43.472965 17150 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:43.472555 17146 raft_consensus.cc:2955] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.475907 16235 raft_consensus.cc:2955] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.482062 15597 raft_consensus.cc:2955] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.479662 17086 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:43.495402 15595 raft_consensus.cc:2955] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.496361 15424 catalog_manager.cc:5697] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index 2 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.531518 17084 raft_consensus.cc:1064] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:18:43.533159 15410 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4b73eb6a0096455b87f6109b059cf0ee with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:43.533901 17084 consensus_queue.cc:237] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:43.540390 17109 raft_consensus.cc:2955] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.546712 15422 catalog_manager.cc:5697] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 3 to 4, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.554924 15595 raft_consensus.cc:2955] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.554682 16237 raft_consensus.cc:2955] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.560597 15514 tablet_service.cc:1558] Processing DeleteTablet for tablet 4b73eb6a0096455b87f6109b059cf0ee with delete_type TABLET_DATA_TOMBSTONED (TS d3c4f5bd76e0445795bad5f6dd93ff69 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57292
I20260709 13:18:43.568996 17164 tablet_replica.cc:333] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:18:43.570639 17164 raft_consensus.cc:2243] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:43.571425 17164 raft_consensus.cc:2272] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:43.576435 17164 ts_tablet_manager.cc:1916] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:43.604422 15669 raft_consensus.cc:2955] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.622464 17164 ts_tablet_manager.cc:1929] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:43.622786 15597 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:18:43.622998 17164 log.cc:1199] T 4b73eb6a0096455b87f6109b059cf0ee P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-0-root/wals/4b73eb6a0096455b87f6109b059cf0ee
I20260709 13:18:43.624467 17146 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:43.624686 15411 catalog_manager.cc:5028] TS d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): tablet 4b73eb6a0096455b87f6109b059cf0ee (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:43.640301 15421 catalog_manager.cc:5697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 4 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.652666 15596 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:18:43.653244 15526 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:18:43.655998 15669 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:18:43.657342 17111 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:43.658728 17109 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:43.664669 16237 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEARNER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:18:43.670797 17086 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:43.676783 15528 consensus_queue.cc:237] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:43.684891 16237 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:43.685281 15676 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:43.686956 17153 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:43.673615 17086 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:18:43.695586 17150 raft_consensus.cc:2955] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.701169 17149 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:43.704368 15674 raft_consensus.cc:2955] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.720896 16234 raft_consensus.cc:2955] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.724854 16237 raft_consensus.cc:2955] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.725630 17149 raft_consensus.cc:2955] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.733520 15411 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 40f71e7bfe1a4575846bcddbb0b9fd55 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:43.744192 15424 catalog_manager.cc:5697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: config changed from index 5 to 6, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New cstate: current_term: 3 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.769629 15597 raft_consensus.cc:2955] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.785398 15674 raft_consensus.cc:2955] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.795569 15597 raft_consensus.cc:1275] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:43.797037 17149 consensus_queue.cc:1048] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:43.799216 15589 tablet_service.cc:1558] Processing DeleteTablet for tablet 40f71e7bfe1a4575846bcddbb0b9fd55 with delete_type TABLET_DATA_TOMBSTONED (TS 5498ef3e89a941e39c25f68cf64e6b0f not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:53300
I20260709 13:18:43.812619 15421 catalog_manager.cc:5697] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index 3 to 4, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:43.820688 17173 tablet_replica.cc:333] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:18:43.821105 15674 raft_consensus.cc:1275] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:43.821797 17173 raft_consensus.cc:2243] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:43.822486 17150 consensus_queue.cc:1048] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:43.830721 15519 consensus_queue.cc:237] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:43.822732 17173 raft_consensus.cc:2272] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:43.837738 17173 ts_tablet_manager.cc:1916] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:43.842765 15676 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:43.845144 17149 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:43.855317 16237 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:43.867503 17150 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:18:43.876636 17150 raft_consensus.cc:2955] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.878993 16237 raft_consensus.cc:2955] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.879938 17173 ts_tablet_manager.cc:1929] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:18:43.880558 17173 log.cc:1199] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-1-root/wals/40f71e7bfe1a4575846bcddbb0b9fd55
I20260709 13:18:43.882367 15412 catalog_manager.cc:5028] TS 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): tablet 40f71e7bfe1a4575846bcddbb0b9fd55 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:43.886725 15676 raft_consensus.cc:2955] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.902103 15519 consensus_queue.cc:237] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:43.911352 15411 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2afa8e112a744f218227526d861b98c3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:43.907862 15423 catalog_manager.cc:5697] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index 4 to 5, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New cstate: current_term: 2 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.913470 16234 raft_consensus.cc:1275] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:43.912024 15597 raft_consensus.cc:1275] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:43.915820 17149 consensus_queue.cc:1048] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:43.917197 17109 consensus_queue.cc:1048] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:43.927217 17109 raft_consensus.cc:2955] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.932798 16234 raft_consensus.cc:2955] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.933817 15597 raft_consensus.cc:2955] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:43.941537 15589 tablet_service.cc:1558] Processing DeleteTablet for tablet 2afa8e112a744f218227526d861b98c3 with delete_type TABLET_DATA_TOMBSTONED (TS 5498ef3e89a941e39c25f68cf64e6b0f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:53300
I20260709 13:18:43.942512 17173 tablet_replica.cc:333] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:18:43.943392 17173 raft_consensus.cc:2243] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:43.944068 17173 raft_consensus.cc:2272] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:43.956207 15411 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8153c7c094654fb492f5c47863ad170d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:43.972055 17173 ts_tablet_manager.cc:1916] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:44.002578 15424 catalog_manager.cc:5697] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 3 to 4, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:44.014694 15660 tablet_service.cc:1558] Processing DeleteTablet for tablet 8153c7c094654fb492f5c47863ad170d with delete_type TABLET_DATA_TOMBSTONED (TS 3ef186ec12554a1d87814af0a0ace9a4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49088
I20260709 13:18:44.016315 17177 tablet_replica.cc:333] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:18:44.017360 17177 raft_consensus.cc:2243] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:44.018355 17177 raft_consensus.cc:2272] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:44.018535 17173 ts_tablet_manager.cc:1929] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:18:44.018944 17173 log.cc:1199] T 2afa8e112a744f218227526d861b98c3 P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-1-root/wals/2afa8e112a744f218227526d861b98c3
I20260709 13:18:44.020181 15412 catalog_manager.cc:5028] TS 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): tablet 2afa8e112a744f218227526d861b98c3 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:44.020853 17177 ts_tablet_manager.cc:1916] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:44.033780 17177 ts_tablet_manager.cc:1929] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:44.034135 17177 log.cc:1199] T 8153c7c094654fb492f5c47863ad170d P 3ef186ec12554a1d87814af0a0ace9a4: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-2-root/wals/8153c7c094654fb492f5c47863ad170d
I20260709 13:18:44.035473 15410 catalog_manager.cc:5028] TS 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): tablet 8153c7c094654fb492f5c47863ad170d (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:18:44.077405 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:45.098803 15676 consensus_queue.cc:237] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:45.113704 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:18:45.114959 15642 consensus_peers.cc:597] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 582343936346420b87ecf2eebd00dc25. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:45.115724 15595 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:45.116499 15519 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:45.117544 17095 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:45.118506 17179 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:45.126906 17095 raft_consensus.cc:2955] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:45.128474 15519 raft_consensus.cc:2955] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:45.130784 15596 raft_consensus.cc:2955] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:45.146486 15423 catalog_manager.cc:5697] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:45.154258 15519 consensus_queue.cc:237] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:45.163403 15674 raft_consensus.cc:1275] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:45.165050 17109 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:45.167116 15596 raft_consensus.cc:1275] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:18:45.167482 15493 consensus_peers.cc:597] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bd36c2cfd99c45cea9e48096d54bed1a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:45.183940 15519 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:34862
I20260709 13:18:45.184628 15519 raft_consensus.cc:606] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:45.184592 17109 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:45.188367 15674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:18:45.188962 15674 raft_consensus.cc:606] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Received request to transfer leadership to TS dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:45.190495 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:18:45.193375 17086 raft_consensus.cc:993] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69: : Instructing follower 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:45.194237 17109 raft_consensus.cc:1081] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Signalling peer 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:18:45.192273 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:45.197029 15676 raft_consensus.cc:2955] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:45.198791 15674 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:49094
I20260709 13:18:45.202689 15595 raft_consensus.cc:2955] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:45.206416 15674 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:45.206955 15674 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:45.211205 17086 raft_consensus.cc:2955] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:45.217602 15674 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:45.223148 15424 catalog_manager.cc:5697] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:45.227524 15676 consensus_queue.cc:237] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:45.231232 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:45.231910 15519 raft_consensus.cc:3055] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:45.232231 15519 raft_consensus.cc:740] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:45.231276 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:45.233152 15595 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:45.232988 15519 consensus_queue.cc:260] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
W20260709 13:18:45.235033 15642 consensus_peers.cc:597] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b99eaeae24b44aa78cfa8c90caca288e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:45.235423 15519 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:45.231351 15674 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:45.239473 15596 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:18:45.240851 15528 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:18:45.241266 17179 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:45.241225 15595 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:45.244130 17186 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:45.244145 15642 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:45.250072 17186 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:45.250696 17186 raft_consensus.cc:697] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:45.251559 17186 consensus_queue.cc:237] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:45.265504 15424 catalog_manager.cc:5697] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 1 to 2, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:45.266954 15519 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:45.266424 16983 raft_consensus.cc:2955] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:45.274412 15528 raft_consensus.cc:2955] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:45.275197 15597 raft_consensus.cc:2955] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:45.290680 15421 catalog_manager.cc:5697] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index -1 to 3, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 2 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:45.294551 15519 consensus_queue.cc:237] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:45.301620 15676 raft_consensus.cc:1275] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:45.301908 15597 raft_consensus.cc:1275] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:18:45.302160 15493 consensus_peers.cc:597] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0af5649725244ae599dfc6ed7aeac5f1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:45.303443 17109 consensus_queue.cc:1048] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:45.305253 17086 consensus_queue.cc:1048] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:45.315529 17194 raft_consensus.cc:2955] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:45.319082 15674 raft_consensus.cc:2955] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:45.337626 15424 catalog_manager.cc:5697] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New cstate: current_term: 1 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:45.344923 15674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49088
I20260709 13:18:45.345660 15674 raft_consensus.cc:606] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Received request to transfer leadership
I20260709 13:18:45.352370 15676 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49088
I20260709 13:18:45.352978 15676 raft_consensus.cc:606] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Received request to transfer leadership
W20260709 13:18:45.355537 15410 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for bd36c2cfd99c45cea9e48096d54bed1a already in progress
I20260709 13:18:45.356117 17186 raft_consensus.cc:993] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4: : Instructing follower dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:45.356459 15597 raft_consensus.cc:2955] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:18:45.356607 17186 raft_consensus.cc:1081] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Signalling peer dd1a14c3173d49dc9b55bb116e3783c9 to start an election
I20260709 13:18:45.364197 16237 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:53976
I20260709 13:18:45.364769 16237 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:45.365142 16237 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:45.372468 16237 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:45.375357 16237 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:45.382540 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:45.383144 15676 raft_consensus.cc:3055] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:45.383487 15676 raft_consensus.cc:740] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:45.384250 15676 consensus_queue.cc:260] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:45.385468 15676 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:45.388805 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:45.389377 15597 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:45.393888 15674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49088
I20260709 13:18:45.394553 15674 raft_consensus.cc:606] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Received request to transfer leadership
I20260709 13:18:45.396687 15597 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:18:45.397882 16205 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:45.398801 17134 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:45.399312 17134 raft_consensus.cc:697] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:45.400174 17134 consensus_queue.cc:237] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:45.400689 15676 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
W20260709 13:18:45.405536 15474 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:18:45.406013 15474 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for bd36c2cfd99c45cea9e48096d54bed1a already in progress
W20260709 13:18:45.406301 15474 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for bd36c2cfd99c45cea9e48096d54bed1a already in progress
I20260709 13:18:45.420414 15423 catalog_manager.cc:5697] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 2 to 3, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 3 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:18:46.552052 15549 diagnostics_log.cc:230] Unable to collect stacks to diagnostics log: Incomplete: not collecting stack trace since debugger or strace is attached
I20260709 13:18:46.648082 17207 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:46.648825 17207 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:46.649266 17181 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:46.649875 17181 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.651443 17207 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:46.652195 17181 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:46.652786 17181 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:46.653268 17181 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.681684 17186 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:46.684473 17186 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.687032 17186 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:46.688503 17134 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:46.697880 17182 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:46.698568 17182 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.700784 17182 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:46.689169 17134 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.725850 17134 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:46.726406 17134 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:46.726851 17134 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.729851 17207 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:46.730540 17207 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.732749 17207 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:46.820174 17134 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:46.834076 17186 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:46.770416 17182 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:46.836228 17186 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:46.853605 17186 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:46.854421 17182 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.856890 17182 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:46.850915 17181 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:46.886879 17181 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:46.837591 17134 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:46.887248 17216 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:46.953313 17216 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.956765 17216 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:46.945837 17186 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:46.968840 17216 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:46.968914 17186 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.969357 17224 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:46.970412 17223 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:46.971141 17186 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:46.970948 17223 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.889446 17181 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.972983 17223 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:46.973174 17181 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:46.961843 17215 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:46.891501 17182 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:46.967612 17221 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:46.946374 17134 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.969616 17216 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.974165 17182 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.974341 17221 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.975925 17134 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:46.970043 17224 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.976297 17182 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:46.976462 17221 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:46.968217 17220 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:46.977217 17220 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.976109 17216 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:46.977906 17224 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:46.974092 17215 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.979447 17220 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:46.980284 17215 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:46.982000 17215 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:46.982219 17224 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:46.982640 17215 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.982969 17216 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:46.983434 17221 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:46.983958 17220 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:46.983659 17216 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.984768 17215 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:46.984462 17220 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.985270 17221 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.982770 17224 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:46.987478 17221 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:46.987573 17220 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:46.985905 17216 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:46.987520 17224 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:47.163069 17225 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.164835 17225 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:47.308825 17225 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:47.312939 17225 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:47.328979 17225 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.386572 17186 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:47.387785 17205 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:47.398916 17205 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.388727 17207 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:47.402683 17207 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:47.389781 17212 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:47.391845 17182 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:47.393874 17215 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:47.395793 17217 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:47.396778 17218 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:47.397693 17219 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:47.406877 17212 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:47.417259 17182 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.420648 17182 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:47.420800 17212 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:47.421495 17215 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.424634 17215 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:47.430042 17231 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:47.425727 17218 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.414088 17227 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:47.428447 17207 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:47.439024 17221 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:47.438868 17220 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:47.440116 17221 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.440392 17220 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.440896 17216 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:47.441980 17216 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.445040 17216 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:47.444615 17220 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:47.446167 17186 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.446794 17220 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:47.446931 17205 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.447578 17220 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.449281 17186 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.434172 17217 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:47.450732 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 281) took 1794 ms. Trace:
I20260709 13:18:47.451325 15519 rpcz_store.cc:276] 0709 13:18:45.641411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.424265 (+1782854us) service_pool.cc:224] Handling call
0709 13:18:47.425161 (+   896us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:47.425627 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:47.425663 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:47.425730 (+    67us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:47.425774 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:47.425841 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:47.435331 (+  9490us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:47.452834 17217 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:18:47.453379 15567 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.008s	sys 0.006s
W20260709 13:18:47.453964 15567 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.008s	sys 0.006s
I20260709 13:18:47.433689 17219 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:47.449313 17205 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:47.458808 17205 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:47.460049 17219 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:47.464856 17225 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.467085 17205 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:18:47.467646 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 310445us
I20260709 13:18:47.469002 17218 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.471894 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 283) took 1527 ms. Trace:
I20260709 13:18:47.472323 15526 rpcz_store.cc:276] 0709 13:18:45.935246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.456034 (+1520788us) service_pool.cc:224] Handling call
0709 13:18:47.456571 (+   537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:47.462052 (+  5481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:47.462088 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:47.462131 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:47.462175 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:47.462232 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:47.462366 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:47.467211 17221 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:47.471151 17217 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:47.484179 17226 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:47.480325 17212 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:47.480827 17219 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:47.485704 17217 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.488979 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:47.490154 17229 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.470403 17231 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.491590 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:47.493716 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:47.499723 15673 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:47.499840 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:47.493825 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:47.494009 17218 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:47.501426 15670 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:47.501629 17229 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.501389 17218 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.507557 17218 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.496086 17205 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:47.501909 17226 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.511096 17205 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:47.513376 17229 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.511440 17231 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.512578 17226 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.513442 17205 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:47.514129 17219 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.517455 17219 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.523622 17217 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.515208 17212 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:47.526930 17212 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:47.527580 17231 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:47.527737 17220 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:47.533640 17229 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:47.528985 17231 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:47.534466 17229 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:47.535238 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:47.536021 15676 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:47.536218 17231 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:47.536852 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:47.537222 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:47.537461 17206 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:47.538081 15677 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:47.537132 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:47.538367 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:47.537587 15674 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:18:47.538318 17206 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.538800 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:47.541738 17206 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.543535 17207 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:47.544430 17207 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.547465 17207 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.550747 17217 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:47.551564 17217 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:47.536841 17229 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:47.483978 17227 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.563746 17211 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:47.553045 17219 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:47.566948 17226 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:47.565670 17217 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:47.567613 17226 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.566999 17208 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:47.568439 17207 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:47.558622 17186 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:47.557709 17218 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:47.574241 17226 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.578905 17219 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:47.576611 17207 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:47.619333 17230 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:47.620349 17207 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:47.619208 17234 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:47.564532 17211 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.569536 17208 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:47.621625 17219 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:47.645699 17234 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.647853 15596 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:47.655138 17227 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.655894 17225 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:47.656670 17225 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.658762 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:47.659170 17211 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.623114 17218 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.660548 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:47.660776 17208 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:47.661159 15597 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:18:47.661787 17218 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.659983 17225 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.620150 17230 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:47.670993 17230 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:47.673475 17236 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:47.674659 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:18:47.676195 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 119965us
I20260709 13:18:47.675724 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:47.676750 15566 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:47.678851 17225 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:47.657321 17231 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:47.683097 15566 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:47.683398 17231 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.648391 17186 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.685758 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:47.671335 17208 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:47.690277 17225 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:18:47.692302 17231 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.692215 17208 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.694509 17208 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.690951 17227 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:47.695120 17227 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:47.695565 17227 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:47.686308 15597 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:47.686525 17211 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:47.698138 17211 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:47.698626 17211 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:47.691566 17230 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:47.674367 17236 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.678131 17218 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:47.712430 17230 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:47.718842 17225 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:47.686496 17232 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:47.727108 15566 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:47.727766 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:47.718868 17234 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.723932 17232 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:47.714865 17207 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:47.734874 15566 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:47.736837 17238 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:47.737375 17238 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:47.737754 17238 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:47.743090 15566 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:47.746644 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:47.747308 17208 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:47.747718 17208 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:47.748045 17208 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:47.748716 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:47.750115 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:47.750339 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:47.746251 17207 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.751513 15596 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:47.739226 17218 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:18:47.750715 15603 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 1.
I20260709 13:18:47.747352 15595 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 1.
I20260709 13:18:47.752056 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:47.750392 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:47.752506 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:47.750658 17219 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:47.770150 17235 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:47.743736 17231 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:47.752277 17237 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:47.771041 17235 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:47.829867 17212 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:47.834885 17212 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.835434 17214 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.762734 17219 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:47.839937 17232 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:47.836289 17214 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:47.829999 17237 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.837646 17230 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:47.835307 17233 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:47.846328 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:47.845431 17241 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:47.849289 17237 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.842092 17219 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:47.846396 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:47.751823 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:47.853590 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:47.856208 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:47.854367 17207 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.854889 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:47.858350 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:47.858529 17212 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.859764 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:47.860459 15676 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:47.876461 15566 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:47.878710 17226 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:47.879184 17226 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:47.879714 17226 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:47.884141 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:47.873513 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:47.903169 17245 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:47.903956 17245 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:47.910033 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:47.912038 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:47.864588 17207 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:47.912673 15600 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 1.
I20260709 13:18:47.914487 15677 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:18:47.846292 17233 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.873239 17225 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:47.872833 17234 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:47.921332 15566 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:47.926191 15597 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 1.
I20260709 13:18:47.870553 17231 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.931435 17233 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.933631 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:47.927104 17236 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.939584 17233 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:47.942523 15599 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:47.919525 17241 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.956025 17233 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:47.958634 17233 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:47.951673 17218 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:18:47.952800 17234 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:47.952972 17225 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.987247 17211 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:47.954046 17207 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:47.990540 17242 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:47.994206 17238 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:47.997433 17225 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:47.963853 17245 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:48.002763 17214 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:48.004756 17238 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:48.018967 17220 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:48.022481 17238 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:48.023729 17241 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:48.023603 17238 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:48.024718 17231 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:48.026216 17238 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:48.027946 17219 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.029996 17246 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:48.030668 17246 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:48.031476 17207 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:48.032881 17246 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:48.023088 16205 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:48.034775 17221 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:48.035234 17221 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:48.035511 16205 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:48.037163 17220 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:48.037581 17220 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:48.038491 17234 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:48.015012 17237 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:48.040083 17237 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.042048 17242 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:48.042352 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:48.035650 17221 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:48.017807 17225 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:48.054626 17225 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.056260 17237 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:48.056875 17225 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:48.057574 17237 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:48.062678 17237 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.065016 17237 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:48.068457 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.071916 17211 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:48.037981 17220 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:48.072553 17208 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.044973 17242 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:47.972298 17227 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:48.076292 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.076974 17227 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:48.045686 17234 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:48.077901 17234 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.081043 17234 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:48.081602 17208 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:48.083521 17233 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:48.086827 17235 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:48.087098 17226 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.092003 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:48.092694 15677 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:48.094832 17226 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:48.096446 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:48.097739 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:48.098379 15668 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:48.104034 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.105302 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.106488 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:48.107468 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.108188 16240 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:48.109102 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.109597 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:48.110750 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.111244 16235 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:48.111442 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.109730 16240 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:48.114645 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.114764 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.115239 16238 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:48.116447 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.117134 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.117470 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.118314 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.117134 15523 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:48.119418 15528 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:18:48.119800 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.121346 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.122596 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.123062 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.123225 15519 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 1.
I20260709 13:18:48.125823 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.126578 15523 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:48.129379 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.130074 15523 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 1.
I20260709 13:18:48.132726 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.085801 17233 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:48.136133 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:48.136909 15670 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:18:48.138166 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.112680 16237 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:48.139533 17219 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:48.151355 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:48.153674 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.150609 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:48.155120 15669 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:48.152388 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.154346 15526 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:48.159756 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.158880 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.160472 15523 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 1.
I20260709 13:18:48.162532 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.163276 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:48.163889 15672 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:48.164788 16205 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:48.167979 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:48.176074 16237 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:48.183203 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:48.184175 15677 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:48.056883 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:48.184867 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:48.189428 15675 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:48.195242 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.198463 17253 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:48.198729 17234 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:48.199115 17253 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:48.199327 17234 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:48.199405 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:48.202157 17255 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:48.194043 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:48.203827 15670 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:48.204998 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.205669 15526 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:18:48.206979 17186 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:48.213589 15602 raft_consensus.cc:1240] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:48.217130 16205 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:48.217692 17186 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:48.217990 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.219430 17186 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.221597 17186 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:48.198125 17252 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:48.222863 17252 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:48.202227 17234 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:48.217828 17258 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:48.225337 15675 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:18:48.225656 17258 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:48.226095 17258 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:48.202504 17253 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:48.226276 17250 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:48.216789 16238 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:48.230508 15669 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:18:48.232270 15676 raft_consensus.cc:1240] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:48.232901 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:48.233397 15674 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:48.233651 15566 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:48.225319 17252 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:48.226847 17250 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.236773 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.239069 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:48.245837 15566 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:48.246523 16205 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:48.247670 17259 raft_consensus.cc:2804] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:48.248670 17259 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:48.249097 17259 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:48.246888 15602 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:48.249366 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.252256 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:48.253197 17235 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:48.237231 17208 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:48.255163 17235 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:48.255604 17235 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:48.256084 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:48.258239 17208 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:48.258752 17208 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:48.262279 15668 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:48.263594 15568 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:48.264394 16205 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:48.268112 15677 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:18:48.269872 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.270802 15566 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:48.273109 15674 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:18:48.274055 17227 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:18:48.276749 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 44088.9us
I20260709 13:18:48.277104 17227 raft_consensus.cc:697] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:48.274667 15566 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:48.275339 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:48.278687 15602 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:48.278097 17227 consensus_queue.cc:237] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.280524 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:48.280552 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:48.282696 15672 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:18:48.283913 15671 raft_consensus.cc:1240] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:48.283900 15675 raft_consensus.cc:1240] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:48.269824 17257 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:48.285584 17219 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:48.286492 17211 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:48.290023 17224 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:48.291318 17224 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:48.291743 17224 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:48.229604 17255 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.290558 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.292974 15566 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:48.290633 17219 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:48.295763 17219 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:48.296411 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:48.297518 15602 raft_consensus.cc:1240] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:48.298024 15566 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:48.298141 17205 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:48.298542 17211 raft_consensus.cc:697] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:48.298729 17205 raft_consensus.cc:697] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:48.298314 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.303378 17213 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:48.303985 17213 raft_consensus.cc:697] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:48.302330 15566 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:48.305125 15566 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:48.303544 17205 consensus_queue.cc:237] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:48.285225 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.310585 17228 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:48.306679 16205 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:48.311546 15675 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:48.311950 16205 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:48.314507 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:48.304906 17213 consensus_queue.cc:237] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:48.294648 17186 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:48.320541 17186 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.321089 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.315610 17134 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:48.322371 17134 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:48.322747 17134 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:48.315948 17258 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.319257 17238 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:48.313795 17228 raft_consensus.cc:697] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:48.335754 17238 raft_consensus.cc:697] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:48.323040 17186 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:48.341388 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.299757 17257 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:48.331310 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:48.353032 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.353457 17258 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:48.353600 16236 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:48.362488 17262 ts_tablet_manager.cc:933] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:18:48.362599 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 84884.1us
I20260709 13:18:48.364524 15568 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:48.365837 15568 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:48.367146 15568 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:48.368264 15568 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:48.382058 15670 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:48.382326 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:48.383018 15674 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 1.
I20260709 13:18:48.383284 15566 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:48.383581 15568 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:48.395741 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.355801 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.398139 17210 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:48.322677 17259 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.398921 16233 raft_consensus.cc:1240] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:48.398937 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:48.356559 17257 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:48.390524 17212 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:48.386618 17227 raft_consensus.cc:2804] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:48.386830 17218 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:18:48.403162 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 90365.1us
I20260709 13:18:48.403601 17258 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:48.390293 17233 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:48.413481 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.355860 17250 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:48.394434 17232 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:48.426433 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.381695 15598 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:18:48.434587 17255 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:48.352191 17211 consensus_queue.cc:237] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.443686 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.424554 17212 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:48.440310 15568 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:48.441064 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.428135 17227 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:48.445477 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:48.427978 17218 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:48.447494 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:48.443660 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 60496.9us
I20260709 13:18:48.430599 17210 raft_consensus.cc:697] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:48.436306 17258 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:48.438218 17233 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:48.439884 17232 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:48.452654 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.454649 15523 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:48.455528 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.458048 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:48.447685 17212 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:48.449142 17227 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:48.450670 17218 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:48.464135 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.454571 17258 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:48.452798 17214 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:48.465955 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:48.454454 17233 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:48.470027 17262 tablet_copy_client.cc:323] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.67:32779
I20260709 13:18:48.456400 17232 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:48.463723 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.485559 17259 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:48.485427 16237 raft_consensus.cc:1240] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:48.487389 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.445490 17228 consensus_queue.cc:237] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.463207 17224 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.474225 17214 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:48.495949 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:48.542254 15670 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:48.451781 17238 consensus_queue.cc:237] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.543939 17263 ts_tablet_manager.cc:933] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:18:48.471973 17220 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.545238 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.543347 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.543954 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.551506 15522 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
I20260709 13:18:48.553344 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:48.555794 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.555334 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.543067 17214 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:48.576020 16240 raft_consensus.cc:2393] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 3: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
W20260709 13:18:48.566653 17221 consensus_meta.cc:287] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.000s
I20260709 13:18:48.580758 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:48.477787 17210 consensus_queue.cc:237] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:48.645627 15603 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:48.650336 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:48.630739 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.650775 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:48.708854 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:48.768536 15597 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:48.698801 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.652369 16239 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:48.769704 15523 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:18:48.631161 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.630254 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.774681 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.770788 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.629851 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.767283 15522 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:48.641850 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:48.654331 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:48.648056 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 200177us
I20260709 13:18:48.682125 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:18:48.667732 17264 log.cc:927] Time spent T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 13:18:48.766695 17208 consensus_meta.cc:287] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.508s	user 0.006s	sys 0.003s
I20260709 13:18:48.665798 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.798194 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:48.799371 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.799021 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:48.647985 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:48.631485 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.805190 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:48.799423 15520 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:48.648218 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:48.599040 15424 catalog_manager.cc:5697] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 2 to 3, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 3 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:48.810151 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:48.697906 15668 raft_consensus.cc:1240] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:48.697479 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:48.667176 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.741508 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.835147 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:48.835835 15676 raft_consensus.cc:1240] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:48.631842 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:48.682427 17224 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:48.742583 17134 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.770828 17258 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.666944 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.840449 15602 raft_consensus.cc:1240] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:18:48.840792 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:48.595549 17221 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.844854 16236 raft_consensus.cc:1240] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 1. Current term is 2. Ops: []
W20260709 13:18:48.840924 17219 consensus_meta.cc:287] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.545s	user 0.010s	sys 0.000s
I20260709 13:18:48.807216 17208 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:48.851027 16235 raft_consensus.cc:2393] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:18:48.851645 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:48.834319 15594 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:18:48.846974 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:48.855631 17263 tablet_copy_client.cc:323] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.65:42481
I20260709 13:18:48.856173 17275 ts_tablet_manager.cc:933] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:48.841341 17261 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:18:48.857537 15595 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:48.837942 15521 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:48.849066 15598 raft_consensus.cc:1240] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
W20260709 13:18:48.898083 17235 consensus_meta.cc:287] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.642s	user 0.006s	sys 0.005s
I20260709 13:18:49.464915 17235 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:48.897378 17261 consensus_queue.cc:1243] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer 5498ef3e89a941e39c25f68cf64e6b0f log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:18:48.851027 17219 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.924496 17212 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:49.282914 17232 consensus_meta.cc:287] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.808s	user 0.007s	sys 0.004s
I20260709 13:18:49.468381 17232 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:49.282464 17218 consensus_meta.cc:287] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.804s	user 0.001s	sys 0.010s
I20260709 13:18:48.931149 17208 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:48.890137 17221 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:18:49.283260 17214 consensus_meta.cc:287] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.673s	user 0.007s	sys 0.003s
I20260709 13:18:49.471208 17214 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:49.473413 17214 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:18:48.990209 17233 consensus_meta.cc:287] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.510s	user 0.007s	sys 0.002s
I20260709 13:18:49.474330 17233 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.911585 17134 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:49.477670 17218 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:49.480010 17218 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:48.890733 17257 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.922111 17258 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:49.046382 15600 raft_consensus.cc:1240] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:49.300329 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:49.540483 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:49.541767 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:49.542953 15601 raft_consensus.cc:2393] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 2: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:49.564481 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:49.567804 15601 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:49.558537 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:49.569911 15524 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:18:49.571112 17233 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:49.571406 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:49.560352 17232 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:49.501720 17257 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:49.468966 17212 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:49.555228 16238 raft_consensus.cc:1240] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:49.468999 17219 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:18:49.576548 17275 thread.cc:641] tablet-download-pool [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.003s	sys 0.011s
W20260709 13:18:49.576977 17275 thread.cc:608] tablet-download-pool [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.003s	sys 0.011s
I20260709 13:18:49.577558 17275 tablet_copy_client.cc:323] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.67:32779
I20260709 13:18:49.584491 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:49.589603 15601 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:18:49.618495 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:49.624457 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:48.986979 17220 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:49.633301 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:49.637992 15601 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:49.641624 15601 raft_consensus.cc:1240] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:48.893226 17227 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:49.675804 17227 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:49.693831 15599 raft_consensus.cc:1240] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:49.694857 17235 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:48.908154 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:49.706138 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:49.707508 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:48.892971 16240 raft_consensus.cc:1240] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:48.891419 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:18:49.870131 17279 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.004s	sys 0.010s
W20260709 13:18:49.870744 17279 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.830s	user 0.004s	sys 0.010s
I20260709 13:18:49.945638 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:49.947165 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:49.947813 15671 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:49.948771 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:49.949373 15671 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:50.021600 17285 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:50.022307 17285 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:50.024706 17285 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:18:50.045116 16205 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.023s	sys 0.006s
W20260709 13:18:50.045666 16205 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.023s	sys 0.006s
I20260709 13:18:50.213482 15600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 223) took 1572 ms. Trace:
I20260709 13:18:50.213967 15600 rpcz_store.cc:276] 0709 13:18:48.640647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.640772 (+   125us) service_pool.cc:224] Handling call
0709 13:18:48.666207 (+ 25435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:49.671494 (+1005287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.213121 (+541627us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b540011e4d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:18:50.213361 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":873841024}
I20260709 13:18:50.215900 15600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 230) took 1412 ms. Trace:
I20260709 13:18:50.234352 17287 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.395809 17287 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:48.892596 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 118316us
I20260709 13:18:50.304571 17284 consensus_queue.cc:1059] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:50.404466 17284 consensus_queue.cc:1243] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer 5498ef3e89a941e39c25f68cf64e6b0f log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:18:50.405167 17284 raft_consensus.cc:3055] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:50.405576 17284 raft_consensus.cc:740] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:50.406569 17284 consensus_queue.cc:260] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:50.408012 17284 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:48.980026 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:48.892987 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:50.410705 15527 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
W20260709 13:18:50.411405 15527 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1451) took 1521 ms (client timeout 1534 ms). Trace:
W20260709 13:18:50.411497 16203 connection.cc:256] RPC call timeout handler was delayed by -1.651059117999921s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
W20260709 13:18:50.411954 15527 rpcz_store.cc:269] 0709 13:18:48.889627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.892708 (+  3081us) service_pool.cc:224] Handling call
0709 13:18:50.411309 (+1518601us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:50.410733 15595 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53324 (request call id 1534) took 1761 ms (client timeout 1980 ms). Trace:
W20260709 13:18:50.420488 15595 rpcz_store.cc:269] 0709 13:18:48.649434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.979701 (+330267us) service_pool.cc:224] Handling call
0709 13:18:50.410646 (+1430945us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:50.395601 15600 rpcz_store.cc:276] 0709 13:18:48.803975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.215073 (+1411098us) service_pool.cc:224] Handling call
0709 13:18:50.215380 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:50.215551 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:50.215574 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:50.215648 (+    74us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:50.215680 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.215740 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:50.215847 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:50.433810 15596 raft_consensus.cc:2393] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 2: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:50.375399 15597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 1459) took 1518 ms. Trace:
I20260709 13:18:50.434158 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:48.914642 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:50.435192 15600 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:50.435730 15674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 1541) took 1526 ms. Trace:
I20260709 13:18:50.436062 15674 rpcz_store.cc:276] 0709 13:18:48.909034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.914396 (+  5362us) service_pool.cc:224] Handling call
0709 13:18:50.435645 (+1521249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:50.436947 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:50.437187 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1442) took 1755 ms. Trace:
W20260709 13:18:50.437443 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 86657.6us
W20260709 13:18:50.437906 16203 connection.cc:256] RPC call timeout handler was delayed by -1.641159117999905s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1526s for a response
W20260709 13:18:50.438305 16203 connection.cc:256] RPC call timeout handler was delayed by -1.6218625169999541s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21389999999999998s for a response
W20260709 13:18:50.438592 16203 connection.cc:256] RPC call timeout handler was delayed by -1.5952591179998308s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1577s for a response
W20260709 13:18:50.438856 16203 connection.cc:256] RPC call timeout handler was delayed by -1.5383113249999951s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15710000000000002s for a response
W20260709 13:18:50.439126 16203 connection.cc:256] RPC call timeout handler was delayed by -1.4197591179997744s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
W20260709 13:18:50.439469 16203 connection.cc:256] RPC call timeout handler was delayed by -1.4152591179997671s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1777s for a response
I20260709 13:18:48.858194 16234 raft_consensus.cc:1240] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:50.440306 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:50.440984 15595 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:50.441100 15687 tablet_copy_service.cc:140] P 3ef186ec12554a1d87814af0a0ace9a4: Received BeginTabletCopySession request for tablet b99eaeae24b44aa78cfa8c90caca288e from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:57710)
I20260709 13:18:50.441756 15687 tablet_copy_service.cc:161] P 3ef186ec12554a1d87814af0a0ace9a4: Beginning new tablet copy session on tablet b99eaeae24b44aa78cfa8c90caca288e from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:57710: session id = dd1a14c3173d49dc9b55bb116e3783c9-b99eaeae24b44aa78cfa8c90caca288e
W20260709 13:18:50.439764 16203 connection.cc:256] RPC call timeout handler was delayed by -1.3216591179998431s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18810000000000002s for a response
W20260709 13:18:50.442492 16203 connection.cc:256] RPC call timeout handler was delayed by -1.3054855609998413s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17390000000000003s for a response
W20260709 13:18:50.442770 16203 connection.cc:256] RPC call timeout handler was delayed by -1.2586222839997845s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1889s for a response
W20260709 13:18:50.443051 16203 connection.cc:256] RPC call timeout handler was delayed by -1.1659591179998188s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2054s for a response
W20260709 13:18:50.443315 16203 connection.cc:256] RPC call timeout handler was delayed by -1.1632591179998144s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2057s for a response
W20260709 13:18:50.443563 16203 connection.cc:256] RPC call timeout handler was delayed by -1.1101591179999559s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2116s for a response
W20260709 13:18:50.443827 16203 connection.cc:256] RPC call timeout handler was delayed by -1.022822283999858s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21509999999999999s for a response
I20260709 13:18:50.445950 15526 rpcz_store.cc:276] 0709 13:18:48.682013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.804192 (+122179us) service_pool.cc:224] Handling call
0709 13:18:50.436508 (+1632316us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b5800003698. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99e4fc0 00007f5ce99cf929 00007f5ce99cf3d2 00007f5ce2f46bd4
0709 13:18:50.436695 (+   187us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:50.436874 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:50.436898 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:50.436938 (+    40us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:50.436966 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.437014 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:50.437115 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5512064}
I20260709 13:18:50.447651 15674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 1553) took 1449 ms. Trace:
I20260709 13:18:50.448024 15674 rpcz_store.cc:276] 0709 13:18:48.998604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.436704 (+1438100us) service_pool.cc:224] Handling call
0709 13:18:50.447590 (+ 10886us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:50.448393 17220 consensus_queue.cc:1059] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:50.439258 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:50.449541 15671 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:50.450241 15671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 1554) took 1451 ms. Trace:
I20260709 13:18:50.452683 15671 rpcz_store.cc:276] 0709 13:18:48.998911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.438636 (+1439725us) service_pool.cc:224] Handling call
0709 13:18:50.450181 (+ 11545us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:50.451736 17284 raft_consensus.cc:3055] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:50.434561 15597 rpcz_store.cc:276] 0709 13:18:48.857499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.374396 (+1516897us) service_pool.cc:224] Handling call
0709 13:18:50.374810 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:50.375047 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:50.375073 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:50.375130 (+    57us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:50.375159 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.375212 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:50.375332 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:50.448941 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:50.456537 15674 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:50.458124 15674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 1556) took 1458 ms. Trace:
I20260709 13:18:50.458890 15674 rpcz_store.cc:276] 0709 13:18:48.999481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.448668 (+1449187us) service_pool.cc:224] Handling call
0709 13:18:50.458071 (+  9403us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:50.460610 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:50.463835 15674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57710 (request call id 344) took 1434 ms. Trace:
I20260709 13:18:50.459848 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:50.455648 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:50.455782 17284 raft_consensus.cc:740] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:50.465170 15671 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:50.465951 15671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 1552) took 1467 ms. Trace:
I20260709 13:18:50.465716 17284 consensus_queue.cc:260] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:50.464650 15597 raft_consensus.cc:2393] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 2: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:50.467743 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:50.454697 15493 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:48.982687 16231 raft_consensus.cc:1240] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:50.524056 15600 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:18:50.530838 15687 tablet_copy_source_session.cc:215] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:50.547931 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:50.548532 15597 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 3: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:50.549336 15597 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:50.545343 15602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 221) took 1905 ms. Trace:
I20260709 13:18:50.549779 17285 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:50.549675 15602 rpcz_store.cc:276] 0709 13:18:48.640014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.640138 (+   124us) service_pool.cc:224] Handling call
0709 13:18:48.640467 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:48.842011 (+201544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.545047 (+1703036us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b540016ba54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:18:50.545276 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":453050624}
I20260709 13:18:50.550459 17285 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:50.550935 17285 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:50.551136 16205 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:50.551855 17290 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:50.552383 17290 raft_consensus.cc:697] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:50.552480 15674 rpcz_store.cc:276] 0709 13:18:49.029048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.460386 (+1431338us) service_pool.cc:224] Handling call
0709 13:18:50.463774 (+  3388us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:50.553748 17290 consensus_queue.cc:237] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:50.463018 15595 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
W20260709 13:18:50.556376 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 101826us
I20260709 13:18:50.556881 16205 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:50.557440 17287 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:50.549118 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:50.553815 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:50.558491 15674 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:50.558521 15600 raft_consensus.cc:2393] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 3: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:48.921015 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:50.559974 15672 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 1542) took 1650 ms (client timeout 1891 ms). Trace:
I20260709 13:18:50.466391 15671 rpcz_store.cc:276] 0709 13:18:48.998180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.455374 (+1457194us) service_pool.cc:224] Handling call
0709 13:18:50.465860 (+ 10486us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:50.561260 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:50.561790 17285 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:50.562539 15521 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1441) took 1895 ms (client timeout 2086 ms). Trace:
W20260709 13:18:50.562935 15521 rpcz_store.cc:269] 0709 13:18:48.666737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.666902 (+   165us) service_pool.cc:224] Handling call
0709 13:18:50.562478 (+1895576us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:50.563344 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:18:50.544478 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 98840.1us
I20260709 13:18:50.564142 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:18:50.564890 15520 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1448) took 1764 ms (client timeout 1541 ms). Trace:
W20260709 13:18:50.565260 15520 rpcz_store.cc:269] 0709 13:18:48.809737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.809861 (+   124us) service_pool.cc:224] Handling call
0709 13:18:50.564004 (+1754143us) spinlock_profiling.cc:243] Waited 2.75 ms on lock 0x7b5800003698. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99d10a0 00007f5ce99d0984 00007f5ce2f4b6d4 00007f5ce2f4b662
0709 13:18:50.564824 (+   820us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6335744}
W20260709 13:18:50.560345 15672 rpcz_store.cc:269] 0709 13:18:48.909387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.920743 (+ 11356us) service_pool.cc:224] Handling call
0709 13:18:50.559914 (+1639171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:50.571934 15686 tablet_copy_service.cc:140] P 3ef186ec12554a1d87814af0a0ace9a4: Received BeginTabletCopySession request for tablet 582343936346420b87ecf2eebd00dc25 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:57710)
I20260709 13:18:50.572129 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:50.572543 15686 tablet_copy_service.cc:161] P 3ef186ec12554a1d87814af0a0ace9a4: Beginning new tablet copy session on tablet 582343936346420b87ecf2eebd00dc25 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:57710: session id = dd1a14c3173d49dc9b55bb116e3783c9-582343936346420b87ecf2eebd00dc25
I20260709 13:18:50.573726 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:50.573848 17293 consensus_queue.cc:1059] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:50.574441 15672 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:50.575160 17292 consensus_queue.cc:1059] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:50.563953 17285 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:50.578210 17285 raft_consensus.cc:3055] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:50.579439 17285 raft_consensus.cc:740] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:50.580504 15686 tablet_copy_source_session.cc:215] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:50.580288 17221 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:50.581055 17221 consensus_queue.cc:1243] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer 5498ef3e89a941e39c25f68cf64e6b0f log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:18:50.581530 17221 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:50.582509 17221 raft_consensus.cc:697] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:50.583493 17221 consensus_queue.cc:237] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:48.920113 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:50.585120 15670 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 1547) took 1674 ms (client timeout 2132 ms). Trace:
W20260709 13:18:50.585458 15670 rpcz_store.cc:269] 0709 13:18:48.910966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.913669 (+  2703us) service_pool.cc:224] Handling call
0709 13:18:50.585060 (+1671391us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:48.920439 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:50.587936 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:50.588577 15669 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 1548) took 1677 ms (client timeout 2232 ms). Trace:
I20260709 13:18:50.588716 15671 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 1.
W20260709 13:18:50.588943 15669 rpcz_store.cc:269] 0709 13:18:48.911309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.913761 (+  2452us) service_pool.cc:224] Handling call
0709 13:18:50.588517 (+1674756us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:50.589869 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:50.590284 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:50.591828 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:48.912036 15668 raft_consensus.cc:1240] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:50.595341 15668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49094 (request call id 1543) took 1685 ms. Trace:
I20260709 13:18:50.595697 15668 rpcz_store.cc:276] 0709 13:18:48.909713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.911435 (+  1722us) service_pool.cc:224] Handling call
0709 13:18:48.911803 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:50.595075 (+1683272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.595261 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:50.592727 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:48.895632 16204 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:18:50.597795 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 1.70217e+06us
I20260709 13:18:50.599058 15671 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:50.599586 16204 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:50.571089 15674 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:18:50.581492 15672 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:50.603044 17292 consensus_queue.cc:1243] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer 5498ef3e89a941e39c25f68cf64e6b0f log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:18:50.603566 17221 consensus_queue.cc:1059] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:50.604377 17292 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:50.580302 17285 consensus_queue.cc:260] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:50.606529 17285 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:50.607882 17208 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:50.609066 15566 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:50.610306 17294 consensus_queue.cc:1059] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:50.610246 17227 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:50.611449 17227 raft_consensus.cc:697] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:50.612545 17296 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:50.612290 17227 consensus_queue.cc:237] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:50.610728 15566 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:50.613658 17219 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:18:48.887912 15698 debug-util.cc:398] Leaking SignalData structure 0x7b080032c400 after lost signal to thread 14432
I20260709 13:18:50.614264 17219 raft_consensus.cc:697] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:50.614445 16203 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:18:50.614957 15698 debug-util.cc:398] Leaking SignalData structure 0x7b08002c49e0 after lost signal to thread 15460
I20260709 13:18:50.615458 17219 consensus_queue.cc:237] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:50.616257 15698 debug-util.cc:398] Leaking SignalData structure 0x7b08001179a0 after lost signal to thread 15552
I20260709 13:18:50.617293 17292 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:48.921741 15567 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
W20260709 13:18:50.619875 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 1.6981e+06us
I20260709 13:18:50.621119 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:50.621605 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:50.622481 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:50.623126 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:50.623739 16234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57258 (request call id 517) took 2254 ms. Trace:
I20260709 13:18:50.624101 16234 rpcz_store.cc:276] 0709 13:18:48.369263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.392425 (+ 23162us) service_pool.cc:224] Handling call
0709 13:18:48.434326 (+ 41901us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:50.440056 (+2005730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.623484 (+183428us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b54005703d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:18:50.623651 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":954604288}
I20260709 13:18:48.911044 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:50.625882 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:50.612962 17296 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:50.626541 17296 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:50.625909 15675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 1546) took 1715 ms (client timeout 1603 ms). Trace:
I20260709 13:18:50.626487 15524 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
W20260709 13:18:50.627192 15675 rpcz_store.cc:269] 0709 13:18:48.910675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.910784 (+   109us) service_pool.cc:224] Handling call
0709 13:18:50.625847 (+1715063us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:50.628137 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:50.630394 17235 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:50.630820 17235 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:50.631166 17235 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:48.895280 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:18:50.632057 15519 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1452) took 1742 ms (client timeout 1683 ms). Trace:
W20260709 13:18:50.632382 15519 rpcz_store.cc:269] 0709 13:18:48.890025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.891171 (+  1146us) service_pool.cc:224] Handling call
0709 13:18:50.631973 (+1740802us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:50.634630 15596 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34102 (request call id 228) took 1992 ms (client timeout 1736 ms). Trace:
W20260709 13:18:50.635020 15596 rpcz_store.cc:269] 0709 13:18:48.646627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.650040 (+  3413us) service_pool.cc:224] Handling call
0709 13:18:50.634435 (+1984395us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b540043c954. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:50.634573 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3833867136}
I20260709 13:18:50.637187 17257 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:50.638170 15601 raft_consensus.cc:1240] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 1. Current term is 2. Ops: []
W20260709 13:18:50.640978 15567 connection.cc:256] RPC call timeout handler was delayed by -1.6583291819999886s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1533s for a response
W20260709 13:18:50.641369 15567 connection.cc:256] RPC call timeout handler was delayed by -1.6475291819999711s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1545s for a response
W20260709 13:18:50.641642 15567 connection.cc:256] RPC call timeout handler was delayed by -1.6304291819999435s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1564s for a response
I20260709 13:18:48.893340 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:50.642652 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:50.642954 15522 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 1.
W20260709 13:18:50.643409 15522 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1453) took 1753 ms (client timeout 1518 ms). Trace:
W20260709 13:18:50.643711 15522 rpcz_store.cc:269] 0709 13:18:48.890369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.891250 (+   881us) service_pool.cc:224] Handling call
0709 13:18:50.643352 (+1752102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:50.643859 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:48.933037 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:50.645179 15603 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
W20260709 13:18:50.645596 15603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53324 (request call id 1528) took 1997 ms (client timeout 1891 ms). Trace:
W20260709 13:18:50.645897 15603 rpcz_store.cc:269] 0709 13:18:48.647960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.932755 (+284795us) service_pool.cc:224] Handling call
0709 13:18:50.645540 (+1712785us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:50.646207 15602 raft_consensus.cc:1240] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:50.646809 17235 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:50.642277 15567 connection.cc:256] RPC call timeout handler was delayed by -1.580029181999862s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16200000000000003s for a response
I20260709 13:18:50.647408 15602 raft_consensus.cc:1240] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 2. Current term is 3. Ops: []
W20260709 13:18:50.647734 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 46428.5us
I20260709 13:18:50.648869 15602 raft_consensus.cc:1240] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:50.649097 16204 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:18:50.617328 15698 debug-util.cc:398] Leaking SignalData structure 0x7b08001924c0 after lost signal to thread 15627
I20260709 13:18:50.651903 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:50.652482 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:50.652643 15598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 222) took 2012 ms. Trace:
W20260709 13:18:50.651877 15698 debug-util.cc:398] Leaking SignalData structure 0x7b0800237a40 after lost signal to thread 15701
I20260709 13:18:48.986277 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:50.653861 15676 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
W20260709 13:18:50.654111 15698 debug-util.cc:398] Leaking SignalData structure 0x7b0800383d60 after lost signal to thread 16266
I20260709 13:18:50.654165 15601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 1531) took 2005 ms. Trace:
I20260709 13:18:50.654706 15601 rpcz_store.cc:276] 0709 13:18:48.648722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.646927 (+998205us) service_pool.cc:224] Handling call
0709 13:18:50.638002 (+991075us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:50.638570 (+   568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.653966 (+ 15396us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b540043c950. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:50.654081 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2278545664}
I20260709 13:18:48.896039 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:18:50.654364 15676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 1549) took 1699 ms (client timeout 1980 ms). Trace:
W20260709 13:18:50.655838 16236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 321) took 1761 ms (client timeout 2002 ms). Trace:
I20260709 13:18:50.656404 15675 raft_consensus.cc:1275] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:50.656955 17257 raft_consensus.cc:2804] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:50.657364 17257 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:50.647492 15597 raft_consensus.cc:1275] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:18:50.657723 17257 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:50.656052 15676 rpcz_store.cc:269] 0709 13:18:48.955277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.965185 (+  9908us) service_pool.cc:224] Handling call
0709 13:18:50.654313 (+1689128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:50.659713 15598 rpcz_store.cc:276] 0709 13:18:48.640322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.642576 (+  2254us) service_pool.cc:224] Handling call
0709 13:18:48.642939 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:50.554642 (+1911703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.652388 (+ 97746us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b540016ab54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:18:50.652582 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":458066816}
W20260709 13:18:50.662039 15424 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.739s	user 0.018s	sys 0.012s
I20260709 13:18:50.661871 17134 consensus_queue.cc:1059] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:48.913507 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:50.662612 15424 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.739s	user 0.018s	sys 0.013s
I20260709 13:18:50.662739 15677 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
W20260709 13:18:50.662899 15491 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.203s	user 0.024s	sys 0.022s
W20260709 13:18:50.663306 15491 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.203s	user 0.024s	sys 0.022s
W20260709 13:18:50.663576 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 1.2038e+06us
W20260709 13:18:50.665215 15491 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.719s (SENT) [suppressed 3 similar messages]
I20260709 13:18:50.666059 17287 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:50.666610 17287 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:50.668602 17287 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:50.669060 17287 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:50.658381 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:50.669502 17287 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:50.670150 15677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 1545) took 1759 ms (client timeout 1988 ms). Trace:
W20260709 13:18:50.671010 15677 rpcz_store.cc:269] 0709 13:18:48.910379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.913263 (+  2884us) service_pool.cc:224] Handling call
0709 13:18:50.670091 (+1756828us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:50.662627 17134 consensus_queue.cc:1243] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Peer 5498ef3e89a941e39c25f68cf64e6b0f log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:18:50.656217 16236 rpcz_store.cc:269] 0709 13:18:48.894227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.895780 (+  1553us) service_pool.cc:224] Handling call
0709 13:18:50.655761 (+1759981us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:50.672046 17287 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:50.675889 17287 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:50.676364 17287 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:50.679193 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:50.679948 16234 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:50.681496 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:50.682608 16236 raft_consensus.cc:2393] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 3: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:18:50.683717 15603 raft_consensus.cc:1240] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:50.684674 17235 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:50.685307 17235 consensus_queue.cc:1048] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:48.893769 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:18:50.686474 15528 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1454) took 1795 ms (client timeout 1876 ms). Trace:
I20260709 13:18:50.686507 17287 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:18:50.686857 15528 rpcz_store.cc:269] 0709 13:18:48.890672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.891311 (+   639us) service_pool.cc:224] Handling call
0709 13:18:50.686420 (+1795109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:50.687526 15599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 1520) took 2399 ms. Trace:
I20260709 13:18:50.688287 15675 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:50.669525 17257 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:50.690634 17257 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:48.998274 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:50.691399 17287 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:50.691704 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:50.691859 17287 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:50.687954 15599 rpcz_store.cc:276] 0709 13:18:48.287702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.288896 (+  1194us) service_pool.cc:224] Handling call
0709 13:18:48.291441 (+  2545us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:49.694383 (+1402942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.687243 (+992860us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54001108d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:18:50.687439 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3224836992}
I20260709 13:18:48.891131 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:50.698422 17218 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:50.698498 16232 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:50.697641 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:18:50.703248 15567 connection.cc:256] RPC call timeout handler was delayed by -1.5319801630000711s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1569s for a response
W20260709 13:18:50.703653 15567 connection.cc:256] RPC call timeout handler was delayed by -1.4592217229999278s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15760000000000002s for a response
W20260709 13:18:50.703931 15567 connection.cc:256] RPC call timeout handler was delayed by -1.45672918199989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1757s for a response
W20260709 13:18:50.704299 15567 connection.cc:256] RPC call timeout handler was delayed by -1.4243291819998376s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17930000000000001s for a response
W20260709 13:18:50.704703 15567 connection.cc:256] RPC call timeout handler was delayed by -1.3790906740000537s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
W20260709 13:18:50.704989 15567 connection.cc:256] RPC call timeout handler was delayed by -1.3496291819999442s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1876s for a response
W20260709 13:18:50.705262 15567 connection.cc:256] RPC call timeout handler was delayed by -1.2830291819998365s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.195s for a response
W20260709 13:18:50.705506 15567 connection.cc:256] RPC call timeout handler was delayed by -1.2317291819999809s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20070000000000002s for a response
W20260709 13:18:50.705751 15567 connection.cc:256] RPC call timeout handler was delayed by -1.2115801630000078s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1925s for a response
W20260709 13:18:50.706032 15567 connection.cc:256] RPC call timeout handler was delayed by -1.160629181999866s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2086s for a response
W20260709 13:18:50.706405 15567 connection.cc:256] RPC call timeout handler was delayed by -1.1214172520001284s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15180000000000002s for a response
I20260709 13:18:50.706604 17310 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:50.707258 17310 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:50.708244 17307 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.708758 17307 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:50.712740 15424 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49140 (request call id 358) took 2274 ms. Trace:
I20260709 13:18:50.715014 16239 raft_consensus.cc:3055] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:50.715392 16239 raft_consensus.cc:740] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:50.716177 16239 consensus_queue.cc:260] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:50.717391 16239 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:50.713124 15424 rpcz_store.cc:276] 0709 13:18:48.438304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.438442 (+   138us) service_pool.cc:224] Handling call
0709 13:18:50.662953 (+2224511us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:50.662981 (+    28us) write_op.cc:454] Released schema lock
0709 13:18:50.712645 (+ 49664us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:49.037761 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:49.038543 (+   782us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:49.038595 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:18:49.038627 (+    32us) tablet.cc:662] Decoding operations
0709 13:18:49.039020 (+   393us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:49.039119 (+    99us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:49.039148 (+    29us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:18:49.039457 (+   309us) tablet.cc:701] Row locks acquired
0709 13:18:49.039507 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:18:49.039666 (+   159us) write_op.cc:270] Start()
0709 13:18:49.039811 (+   145us) write_op.cc:276] Timestamp: P: 1783603129039602 usec, L: 0
0709 13:18:49.039848 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 13:18:49.040573 (+   725us) log.cc:844] Serialized 529 byte log entry
0709 13:18:50.240182 (+1199609us) op_driver.cc:464] REPLICATION: finished
0709 13:18:50.379263 (+139081us) write_op.cc:301] APPLY: starting
0709 13:18:50.379369 (+   106us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:50.379508 (+   139us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:50.379676 (+   168us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:50.379975 (+   299us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:50.380285 (+   310us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:18:50.380351 (+    66us) write_op.cc:312] APPLY: finished
0709 13:18:50.380658 (+   307us) log.cc:844] Serialized 23 byte log entry
0709 13:18:50.380873 (+   215us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:50.380961 (+    88us) write_op.cc:454] Released schema lock
0709 13:18:50.381021 (+    60us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":138753,"apply.run_cpu_time_us":2118,"apply.run_wall_time_us":2118,"num_ops":1,"prepare.queue_time_us":114323,"prepare.run_cpu_time_us":15108,"prepare.run_wall_time_us":841925,"raft.queue_time_us":287338,"raft.run_cpu_time_us":1652,"raft.run_wall_time_us":141913,"replication_time_us":1200151,"thread_start_us":3124414,"threads_started":4,"wal-append.queue_time_us":909809}]]}
W20260709 13:18:50.691992 15673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 1550) took 1694 ms (client timeout 1724 ms). Trace:
I20260709 13:18:50.650759 16204 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:18:50.726379 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 61184.3us
W20260709 13:18:50.724659 15673 rpcz_store.cc:269] 0709 13:18:48.997357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.997960 (+   603us) service_pool.cc:224] Handling call
0709 13:18:50.691930 (+1693970us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:50.726976 15491 connection.cc:256] RPC call timeout handler was delayed by -1.1775195660000009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1986s for a response
I20260709 13:18:50.727082 17258 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:50.727520 17258 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:18:50.727731 15491 connection.cc:256] RPC call timeout handler was delayed by -1.0632195660000434s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21130000000000002s for a response
I20260709 13:18:50.727873 17258 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:50.698504 15668 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:18:50.728579 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 77909.2us
W20260709 13:18:50.706688 15567 connection.cc:256] RPC call timeout handler was delayed by -1.1070172520001051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1534s for a response
W20260709 13:18:50.730458 15567 connection.cc:256] RPC call timeout handler was delayed by -1.0607291819999318s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2197s for a response
W20260709 13:18:50.730749 15567 connection.cc:256] RPC call timeout handler was delayed by -1.0548636660000739s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1816s for a response
W20260709 13:18:50.731005 15567 connection.cc:256] RPC call timeout handler was delayed by -1.0252801630001613s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21320000000000003s for a response
I20260709 13:18:50.733889 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:50.739320 16204 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:18:50.726668 16232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 319) took 1868 ms (client timeout 2004 ms). Trace:
I20260709 13:18:50.739738 16235 raft_consensus.cc:2393] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 2: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:18:50.747289 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:50.755214 16235 raft_consensus.cc:2393] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 2: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:18:50.759186 17311 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:50.760005 17311 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:50.763278 16204 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:50.763562 17287 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:50.763562 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:50.770750 16233 raft_consensus.cc:2393] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 2: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:18:50.764640 17307 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:50.767329 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:50.773811 16237 raft_consensus.cc:2393] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 2: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:18:50.767045 17310 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:50.618875 17299 consensus_queue.cc:1059] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:18:50.740293 16232 rpcz_store.cc:269] 0709 13:18:48.858287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.860008 (+  1721us) service_pool.cc:224] Handling call
0709 13:18:50.726589 (+1866581us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:50.778641 16204 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:50.779038 17257 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:50.781782 15567 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:50.787053 15567 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:50.617726 17292 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:50.767062 17220 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:18:50.785624 15642 connection.cc:256] RPC call timeout handler was delayed by -1.0123335430000679s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1502s for a response
I20260709 13:18:50.794183 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:50.788264 15567 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:50.785720 16204 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:50.791791 16238 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53976 (request call id 314) took 2090 ms. Trace:
I20260709 13:18:50.796595 15524 raft_consensus.cc:1275] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:18:50.798117 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:50.795425 16238 rpcz_store.cc:276] 0709 13:18:48.696766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.696884 (+   118us) service_pool.cc:224] Handling call
0709 13:18:48.697891 (+  1007us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:49.575114 (+877223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.787211 (+1212097us) spinlock_profiling.cc:243] Waited 857 ms on lock 0x7b54004cf4d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:18:50.787397 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1971188096}
I20260709 13:18:50.818950 15519 raft_consensus.cc:1240] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:50.817965 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:50.818279 17232 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:50.814251 17233 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:50.792443 17220 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:50.818264 17218 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:18:50.819638 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 205255us
I20260709 13:18:50.826463 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:50.825608 15567 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:50.820751 15526 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
W20260709 13:18:50.826095 15641 connection.cc:256] RPC call timeout handler was delayed by -1.0822485340002004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1502s for a response
I20260709 13:18:50.786093 17257 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:50.840714 16203 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:18:50.837302 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 51661.5us
I20260709 13:18:50.798399 17292 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:50.817871 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:50.837232 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:50.844419 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:50.847627 16236 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 3: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:18:50.846664 15524 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:50.849203 17290 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:50.824595 17220 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:50.826164 17310 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.821236 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:50.825973 17306 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:50.826380 17218 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:50.821480 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:50.799448 17214 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:18:50.861853 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:50.819531 17301 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:50.845901 17257 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:50.866168 15567 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:50.845846 17134 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:50.828852 17258 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:50.874094 16203 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:50.871706 17227 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:50.886605 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:50.873174 17311 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:50.868161 17284 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:50.878916 17306 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:50.884044 15567 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:50.867559 17218 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:50.867476 15521 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:50.888245 15528 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:50.865236 16231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57258 (request call id 509) took 2572 ms. Trace:
I20260709 13:18:50.920919 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:50.921666 15676 raft_consensus.cc:1275] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:18:50.903306 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 58185us
W20260709 13:18:50.904281 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 47203us
I20260709 13:18:50.908700 17258 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:50.872184 17301 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:50.927459 17301 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:50.915840 15567 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:50.929193 15567 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:50.921628 15520 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:50.916543 17235 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:50.864271 17310 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:50.882216 17134 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:50.927968 16204 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:50.887914 17227 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:50.939975 17227 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:50.940215 17308 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:18:50.942006 16263 debug-util.cc:398] Leaking SignalData structure 0x7b080030c060 after lost signal to thread 14432
I20260709 13:18:50.906133 17316 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:18:50.918953 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 44961.1us
I20260709 13:18:50.919412 17290 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:50.937973 16239 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:50.943998 16203 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:18:50.936086 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 60654.1us
I20260709 13:18:50.945883 16239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 736) took 2155 ms. Trace:
I20260709 13:18:50.942704 17316 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:18:50.946501 17224 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:18:50.953873 17316 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:50.942966 16263 debug-util.cc:398] Leaking SignalData structure 0x7b080030c740 after lost signal to thread 15460
I20260709 13:18:50.954035 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:50.947221 16239 rpcz_store.cc:276] 0709 13:18:48.790606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.822476 (+ 31870us) service_pool.cc:224] Handling call
0709 13:18:48.822805 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:50.714913 (+1892108us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b5400721620. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4982dc0 00007f5ce4a0d1c0 00007f5ce4a0e231 00007f5ce4a0ad8f 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c
0709 13:18:50.945581 (+230668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.945774 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4241637248}
I20260709 13:18:50.931954 17235 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:50.933013 17214 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:50.932957 17232 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:50.956588 16203 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:50.943209 17134 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:50.963922 15520 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:50.968622 17320 consensus_queue.cc:1059] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:18:50.969522 15642 consensus_peers.cc:597] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bd36c2cfd99c45cea9e48096d54bed1a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:50.968878 15493 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:50.945374 17290 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:50.945130 15668 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:50.953967 17298 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:50.954562 15673 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:50.954694 17258 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:50.917812 17296 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:50.920786 16231 rpcz_store.cc:276] 0709 13:18:48.289441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.289863 (+   422us) service_pool.cc:224] Handling call
0709 13:18:48.982455 (+692592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:50.547097 (+1564642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.850938 (+303841us) spinlock_profiling.cc:243] Waited 692 ms on lock 0x7b5400532850. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:50.861594 (+ 10656us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1591908224}
I20260709 13:18:50.960807 17235 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:50.983845 15493 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:50.977038 17290 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:50.990581 16240 raft_consensus.cc:1240] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: []
W20260709 13:18:50.990209 16263 debug-util.cc:398] Leaking SignalData structure 0x7b080036ede0 after lost signal to thread 15552
I20260709 13:18:50.989275 17221 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:51.004889 17315 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:18:51.008226 16263 debug-util.cc:398] Leaking SignalData structure 0x7b08000d0c20 after lost signal to thread 15627
W20260709 13:18:51.022544 16263 debug-util.cc:398] Leaking SignalData structure 0x7b08003a6080 after lost signal to thread 15701
I20260709 13:18:51.008167 17316 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:50.983516 17311 consensus_queue.cc:1059] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:18:51.018883 17315 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:51.033555 15599 raft_consensus.cc:1240] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:18:51.035840 17258 consensus_queue.cc:1059] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:51.036688 17258 consensus_queue.cc:1243] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer 5498ef3e89a941e39c25f68cf64e6b0f log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:18:51.031852 17315 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:50.998427 17294 consensus_queue.cc:1059] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
W20260709 13:18:51.041572 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 110340us
W20260709 13:18:51.040717 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 130558us
I20260709 13:18:51.029381 17295 raft_consensus.cc:3055] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:51.028508 17317 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:51.057430 16235 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:51.046859 15568 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:18:51.043015 16263 debug-util.cc:398] Leaking SignalData structure 0x7b08003a6980 after lost signal to thread 16266
I20260709 13:18:51.063508 17318 raft_consensus.cc:3055] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:51.073534 17318 raft_consensus.cc:740] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:51.080278 17287 raft_consensus.cc:3055] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:51.084347 17323 raft_consensus.cc:3055] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:51.084874 17323 raft_consensus.cc:740] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:51.086095 17323 consensus_queue.cc:260] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:51.087761 17323 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:51.074589 16263 debug-util.cc:398] Leaking SignalData structure 0x7b080004fea0 after lost signal to thread 16267
I20260709 13:18:51.051923 17295 raft_consensus.cc:740] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:51.097286 17227 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:51.103320 17220 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.104228 17227 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:51.106618 17235 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:51.110486 17285 consensus_meta.cc:287] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.504s	user 0.010s	sys 0.000s
I20260709 13:18:51.111332 17296 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:51.082371 17318 consensus_queue.cc:260] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.083905 17287 raft_consensus.cc:740] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:51.105657 17220 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:51.123435 17295 consensus_queue.cc:260] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.083069 17306 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:51.133077 17235 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:51.101377 17134 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.113205 17214 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:18:51.138315 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 171855us
W20260709 13:18:51.131273 16263 debug-util.cc:398] Leaking SignalData structure 0x7b08002e76e0 after lost signal to thread 17262
I20260709 13:18:51.151335 17290 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.112732 17257 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:51.144272 16263 debug-util.cc:398] Leaking SignalData structure 0x7b08002f5c00 after lost signal to thread 17263
I20260709 13:18:51.153249 15491 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:51.160885 17292 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.176995 17299 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:51.177511 17299 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:51.177883 17299 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:51.177052 15491 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:51.180276 15491 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:51.182060 15491 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:51.174439 17318 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:51.188337 16234 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
W20260709 13:18:51.179661 16263 debug-util.cc:398] Leaking SignalData structure 0x7b08002c6060 after lost signal to thread 17264
I20260709 13:18:51.179523 17295 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:51.189993 15491 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:51.182773 17294 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:51.187093 17316 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:51.182305 17304 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:18:51.195042 16263 debug-util.cc:398] Leaking SignalData structure 0x7b080036dca0 after lost signal to thread 17273
I20260709 13:18:51.193063 17317 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:51.187327 17290 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:51.231580 15491 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:51.232586 15491 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:51.224248 17304 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:51.236037 17304 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:51.235803 17320 consensus_queue.cc:1059] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:51.206776 17294 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:51.209873 17292 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:51.230183 17285 raft_consensus.cc:2804] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:51.255309 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:51.255501 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:51.230710 17317 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:51.192600 17315 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:51.265724 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:51.266800 15423 catalog_manager.cc:5697] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 2 to 3, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 3 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:51.268635 15423 catalog_manager.cc:5697] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 2 to 3, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 3 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:18:51.270046 15423 catalog_manager.cc:5697] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 1 to 2, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 2 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:18:51.252883 17294 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:51.257288 17318 raft_consensus.cc:2804] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:51.199105 17287 consensus_queue.cc:260] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.292388 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:51.291360 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:51.292042 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:51.256945 17257 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:51.257726 17285 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:51.296413 16234 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:51.265017 17317 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:51.302574 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:51.303694 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:51.296510 16235 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:51.268464 17310 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:51.309048 17300 raft_consensus.cc:3055] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:51.309521 17300 raft_consensus.cc:740] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:51.310424 17300 consensus_queue.cc:260] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:51.309906 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 329732us
I20260709 13:18:51.312139 17330 consensus_queue.cc:1059] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:51.313258 17311 raft_consensus.cc:3055] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:51.313648 17311 raft_consensus.cc:740] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:51.314569 17311 consensus_queue.cc:260] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.315846 17311 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:51.306617 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:51.318681 16238 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:51.319752 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:51.311959 17300 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.325969 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:51.330243 15524 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.295531 17318 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:51.302160 17285 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:51.300717 17134 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:18:51.338969 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 382301us
I20260709 13:18:51.349517 16203 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:51.356630 16203 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:18:51.360157 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 51238us
W20260709 13:18:51.363005 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 380885us
I20260709 13:18:51.363823 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:51.337646 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:51.340791 17318 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:51.365036 15493 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:51.368898 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:51.346557 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:51.375886 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:51.376579 15522 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:51.338335 17330 consensus_queue.cc:1059] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:18:51.362519 15537 tablet_copy_service.cc:140] P d3c4f5bd76e0445795bad5f6dd93ff69: Received BeginTabletCopySession request for tablet 0af5649725244ae599dfc6ed7aeac5f1 from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:56840)
I20260709 13:18:51.355038 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:51.377862 15493 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:51.362998 17287 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.363039 17134 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:51.368322 17323 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:51.386701 16203 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:51.368660 17284 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:51.387949 16203 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:51.387583 15493 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:51.393481 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:51.394085 15528 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.394273 16203 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:51.378548 17320 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:18:51.425006 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.425801 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:51.429616 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:18:51.442591 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 48734.4us
I20260709 13:18:51.443591 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:51.390380 17323 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:51.361199 17301 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.445640 15522 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:18:51.443676 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:51.390101 17293 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:51.462518 15524 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:51.451819 15520 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:51.396559 15537 tablet_copy_service.cc:161] P d3c4f5bd76e0445795bad5f6dd93ff69: Beginning new tablet copy session on tablet 0af5649725244ae599dfc6ed7aeac5f1 from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:56840: session id = dd1a14c3173d49dc9b55bb116e3783c9-0af5649725244ae599dfc6ed7aeac5f1
I20260709 13:18:51.453505 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:51.461994 17299 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:51.454254 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:51.387063 17329 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:51.394614 17292 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:51.395370 16231 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
W20260709 13:18:51.441134 17218 consensus_meta.cc:287] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.513s	user 0.005s	sys 0.005s
I20260709 13:18:51.452732 16203 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:51.468178 17331 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:51.457906 17323 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:51.444823 17296 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:51.393410 17227 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:18:51.482368 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:51.484205 17315 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:51.488408 17275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 582343936346420b87ecf2eebd00dc25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:51.484869 16203 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:51.491079 16240 raft_consensus.cc:2410] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:18:51.496560 15521 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:51.499076 15528 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:51.486750 17295 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:51.499828 16203 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:51.501807 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:51.503144 17292 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:51.474426 17218 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:51.475437 17329 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:51.503710 16203 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:51.476393 17293 consensus_queue.cc:1059] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:18:51.505147 15519 raft_consensus.cc:2393] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 2: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:18:51.506811 17220 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:51.507630 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:51.509446 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:51.498493 17299 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:51.514178 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:51.510855 15528 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:51.504026 17296 raft_consensus.cc:2804] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:51.514277 15491 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:51.517871 15526 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:18:51.519721 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:51.520344 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:51.521139 17317 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.521988 17336 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:18:51.523306 15675 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.028s	sys 0.087s
I20260709 13:18:51.508846 17329 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:51.529814 15675 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.028s	sys 0.087s
I20260709 13:18:51.529234 15668 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:51.518930 17296 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:51.542426 17331 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:51.548803 15528 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
W20260709 13:18:51.554723 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:51.560269 17335 log.cc:927] Time spent T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:18:51.534904 17285 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.561965 15677 raft_consensus.cc:1240] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:18:51.562891 17310 raft_consensus.cc:3055] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:51.563282 17310 raft_consensus.cc:740] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:51.563777 17304 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.564163 17310 consensus_queue.cc:260] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.565439 17310 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:51.550009 17296 raft_consensus.cc:3060] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.569221 17328 raft_consensus.cc:3055] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:51.569679 17328 raft_consensus.cc:740] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:51.574234 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:51.576716 16204 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:51.550820 17331 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:51.563480 17285 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:51.580363 16204 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:51.586478 17262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b99eaeae24b44aa78cfa8c90caca288e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:51.587594 15668 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:18:51.554948 17329 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:51.569855 17218 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:18:51.596994 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.565883 17304 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:51.603873 15676 raft_consensus.cc:1240] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:51.603410 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:51.612636 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:51.589623 16204 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:51.579859 17292 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:51.625710 16204 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:51.588594 17301 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:51.630323 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:51.565304 17323 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:51.580190 17331 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:51.634269 15596 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:18:51.604805 17224 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:51.570680 17328 consensus_queue.cc:260] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:51.627279 17317 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:51.627420 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:51.627562 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:51.624143 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:51.627882 17336 raft_consensus.cc:2804] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:51.621160 15673 raft_consensus.cc:1240] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:51.649726 17311 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:51.629859 16204 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:51.650143 17311 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:51.633373 17221 raft_consensus.cc:2804] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:51.646878 16203 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:51.637251 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:51.650547 17311 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:51.646525 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:51.653435 15671 raft_consensus.cc:1240] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:51.654453 17224 raft_consensus.cc:697] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:51.655247 17328 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.656256 15642 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:51.659847 16204 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:51.663403 17295 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:51.663951 17295 raft_consensus.cc:697] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:51.664799 17295 consensus_queue.cc:237] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.666066 17221 raft_consensus.cc:697] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:51.666591 17336 raft_consensus.cc:697] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:51.668290 15519 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:51.667091 17221 consensus_queue.cc:237] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.658010 15668 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:51.670792 17220 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:51.671653 16203 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:51.672463 17220 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:51.673360 16203 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:51.675066 17296 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.677306 17296 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:51.655406 17224 consensus_queue.cc:237] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.683676 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:51.684372 15674 raft_consensus.cc:3060] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.627305 17327 raft_consensus.cc:3055] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:51.648809 17329 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:18:51.687693 15568 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.031s	sys 0.036s
W20260709 13:18:51.685621 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 294209us
I20260709 13:18:51.629142 17294 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:51.688019 15568 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.031s	sys 0.036s
I20260709 13:18:51.680140 17287 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:51.699092 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:51.686885 17220 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:18:51.703949 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 622880us
I20260709 13:18:51.698299 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:51.698647 15493 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:51.697768 15674 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:51.677788 17318 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:51.706338 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 310082us
I20260709 13:18:51.726938 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:51.722841 15493 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:51.728953 15493 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:51.692325 17284 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:51.708467 16203 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:51.735632 15669 raft_consensus.cc:1240] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:51.735436 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:51.719400 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:51.690361 17327 raft_consensus.cc:740] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:51.726537 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:51.738823 16239 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:51.708007 15568 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:51.753964 15526 raft_consensus.cc:1240] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:51.712546 15537 tablet_copy_source_session.cc:215] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:51.763495 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:51.768018 15523 raft_consensus.cc:1240] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:51.750308 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:51.763650 15520 raft_consensus.cc:3055] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:51.769253 15520 raft_consensus.cc:740] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:51.769456 16235 raft_consensus.cc:1240] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:51.767899 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:51.765858 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:51.770754 16239 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:51.754106 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:51.767534 15526 raft_consensus.cc:1240] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:51.765488 15642 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:51.772753 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:51.774178 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:51.775060 16238 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
W20260709 13:18:51.775471 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 67546.4us
I20260709 13:18:51.768901 16237 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:51.776199 15568 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:51.777007 15493 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:51.777856 15568 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:18:51.775003 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 47601.8us
I20260709 13:18:51.718596 17220 raft_consensus.cc:697] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:51.663408 17319 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:18:51.780707 15519 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:18:51.738116 17300 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:51.792042 15674 raft_consensus.cc:1275] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:18:51.738166 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:51.796465 17323 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:51.667536 17336 consensus_queue.cc:237] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.797360 15520 consensus_queue.cc:260] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.798379 15520 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:51.736336 17317 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:51.703233 17287 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:51.788137 15528 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:51.749617 17296 raft_consensus.cc:2804] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:51.780409 17218 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:51.691530 17320 consensus_queue.cc:1059] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.071s
I20260709 13:18:51.793517 15642 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:51.786226 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:51.766073 17284 raft_consensus.cc:697] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:51.789196 17329 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:51.749781 17304 consensus_queue.cc:1059] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:18:51.810073 15642 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:51.812669 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:51.785874 17235 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:51.785701 17299 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:51.793834 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:51.786031 17219 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:51.811518 16235 raft_consensus.cc:1240] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:51.749831 17311 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.791347 17300 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:51.819501 17300 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.820562 17311 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:51.806227 17287 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.826284 15568 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:51.826462 17284 consensus_queue.cc:237] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.829160 15521 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:18:51.842516 17311 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:51.843020 17311 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:51.843465 17311 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:51.844439 17318 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:51.816586 17296 raft_consensus.cc:697] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:51.778105 15493 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:51.794337 17134 consensus_queue.cc:1059] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:18:51.859676 17294 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:51.864715 17336 consensus_queue.cc:1059] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:51.813570 17330 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:51.798761 17315 consensus_queue.cc:1059] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:18:51.829856 17307 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:51.802543 17306 consensus_queue.cc:1059] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:18:51.831566 16232 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:51.842684 17340 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:18:51.889209 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.816983 17218 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:18:51.808517 17326 consensus_queue.cc:1059] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
W20260709 13:18:51.891048 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 78447.9us
I20260709 13:18:51.879693 17233 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:51.897044 17309 raft_consensus.cc:2764] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:18:51.897238 17340 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:51.880053 17330 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
W20260709 13:18:51.901217 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 82218.3us
I20260709 13:18:51.924165 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:18:51.901691 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 243980us
I20260709 13:18:51.905479 15520 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:18:51.885381 17307 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
W20260709 13:18:51.897126 17332 log.cc:927] Time spent T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:18:51.879437 17219 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:18:51.883020 17343 consensus_queue.cc:1059] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:18:51.909837 17257 raft_consensus.cc:3055] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:51.930100 15599 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:51.908933 17233 raft_consensus.cc:697] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:51.842724 17327 consensus_queue.cc:260] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:51.910513 17308 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.027s	sys 0.041s
I20260709 13:18:51.930461 17315 raft_consensus.cc:3055] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:18:51.935701 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 157667us
I20260709 13:18:51.969723 15493 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:51.970983 15493 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:18:51.974761 17349 log.cc:927] Time spent T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:18:51.908871 17340 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:51.968770 17308 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.028s	sys 0.042s
I20260709 13:18:51.931290 17307 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.969339 17315 raft_consensus.cc:740] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:51.982144 17300 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:51.984431 16237 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:51.966089 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:51.985507 16235 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:51.890964 17220 consensus_queue.cc:237] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.888051 17296 consensus_queue.cc:237] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.926815 17290 raft_consensus.cc:3055] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:51.938302 17304 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:51.991297 17304 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:51.990891 17315 consensus_queue.cc:260] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.991001 17290 raft_consensus.cc:740] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:51.991941 17300 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:51.992241 17315 raft_consensus.cc:3060] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.991688 17304 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:51.977257 15493 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:51.994431 17287 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:51.979036 16239 raft_consensus.cc:1240] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:51.995404 15675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 1589) took 1024 ms. Trace:
I20260709 13:18:51.996495 17287 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:51.910377 17330 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:51.939817 17257 raft_consensus.cc:740] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:51.988632 16238 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:18:51.997323 16238 pending_rounds.cc:85] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9: Aborting all ops after (but not including) 5
I20260709 13:18:51.994755 15493 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:51.995771 15675 rpcz_store.cc:276] 0709 13:18:50.971415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.971900 (+   485us) service_pool.cc:224] Handling call
0709 13:18:50.972274 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:50.979140 (+  6866us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:50.979168 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:50.985878 (+  6710us) log.cc:844] Serialized 26 byte log entry
0709 13:18:51.531009 (+545131us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:51.531048 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:51.531113 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:51.995182 (+464069us) raft_consensus.cc:1685] finished
0709 13:18:51.995200 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:51.995331 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4440,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":5428,"thread_start_us":543801,"threads_started":1,"wal-append.queue_time_us":703621}
W20260709 13:18:51.928200 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 107674us
I20260709 13:18:51.993588 17134 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:52.003351 15493 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:52.005203 15493 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:52.014011 16237 raft_consensus.cc:1240] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:52.015556 16237 raft_consensus.cc:1240] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:52.016124 17275 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:53976 (request call id 310) took 3471 ms. Trace:
I20260709 13:18:52.016600 17275 rpcz_store.cc:276] 0709 13:18:48.544613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.565506 (+ 20893us) service_pool.cc:224] Handling call
0709 13:18:48.863064 (+297558us) ts_tablet_manager.cc:934] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
0709 13:18:51.486165 (+2623101us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:18:51.692007 (+205842us) tablet_metadata.cc:665] Metadata flushed
0709 13:18:51.707054 (+ 15047us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:18:52.015243 (+308189us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:18:52.015611 (+   368us) tablet_replica.cc:206] Creating consensus instance
0709 13:18:52.015954 (+   343us) ts_tablet_manager.cc:989] Replica registered.
0709 13:18:52.016020 (+    66us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":261,"tablet-copy.queue_time_us":277622,"thread_start_us":948802,"threads_started":2}
I20260709 13:18:51.984592 17328 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:51.997566 16238 pending_rounds.cc:107] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9: Aborting uncommitted NO_OP operation due to leader change: 3.6
W20260709 13:18:52.021812 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 120328us
I20260709 13:18:51.990345 17348 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:52.017668 16237 raft_consensus.cc:1240] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:52.023118 16238 raft_consensus.cc:2889] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:52.035028 15493 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:52.012760 17299 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:52.018184 17275 tablet_copy_client.cc:806] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:52.012416 17300 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:52.019809 17285 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:52.030984 17320 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:52.056289 17345 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:52.056777 17345 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:52.057130 17345 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:52.038002 17299 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:52.061621 17290 consensus_queue.cc:260] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:52.057729 17328 consensus_queue.cc:1059] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 13:18:52.059617 17233 consensus_queue.cc:237] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:52.042488 17300 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:52.042438 15668 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:52.073292 15523 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:52.047039 17320 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:18:52.062484 15493 connection.cc:256] RPC call timeout handler was delayed by -1.1961192930000379s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21650000000000003s for a response
I20260709 13:18:52.081741 15676 raft_consensus.cc:1240] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:52.082894 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:52.040925 17348 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:52.064709 17299 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:52.090790 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:52.091074 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:52.079169 17300 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:52.073983 17275 tablet_copy_client.cc:670] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:52.078703 17284 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.038856 17257 consensus_queue.cc:260] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:52.091349 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:52.085223 17320 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:52.100196 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:52.104301 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:52.107759 15522 raft_consensus.cc:2393] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 3: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:18:52.091106 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:52.106400 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:52.081203 17292 consensus_queue.cc:1059] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
W20260709 13:18:52.109171 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 46825.4us
I20260709 13:18:52.093587 17348 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:52.112005 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:52.118378 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:52.115346 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:52.106794 17304 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:52.123262 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.123219 17304 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:52.125713 17331 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:52.120541 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:52.106633 17290 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:52.131704 17329 raft_consensus.cc:3055] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:52.134378 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:52.110472 17257 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:52.137413 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:52.143530 15520 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:52.136857 17329 raft_consensus.cc:740] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:52.143563 16235 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
W20260709 13:18:52.137058 17350 log.cc:927] Time spent T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:18:52.103780 17284 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:52.149809 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:52.144289 15525 raft_consensus.cc:2393] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 2: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:18:52.152848 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:52.153146 16204 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:52.158396 17353 ts_tablet_manager.cc:933] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:18:52.162000 17358 log.cc:927] Time spent T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:18:52.172179 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:52.165714 17311 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:52.176790 15526 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:52.177879 15519 raft_consensus.cc:1275] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:18:52.182250 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:52.184404 15526 raft_consensus.cc:1240] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:52.186058 15526 raft_consensus.cc:1240] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:52.187209 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:52.189043 15526 raft_consensus.cc:2393] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 3: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:18:52.188179 17361 consensus_queue.cc:1059] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
W20260709 13:18:52.192741 17310 consensus_meta.cc:287] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.621s	user 0.010s	sys 0.000s
I20260709 13:18:52.199368 15527 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:52.204816 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:18:52.202962 17362 log.cc:927] Time spent T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:18:52.203627 17363 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 13:18:52.205734 15526 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:52.198444 17310 raft_consensus.cc:3055] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:52.148136 17306 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:18:52.181557 17318 consensus_queue.cc:1059] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:18:52.214735 17310 raft_consensus.cc:740] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:52.147359 17329 consensus_queue.cc:260] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:52.193856 17284 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:52.199576 17311 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:52.210091 17331 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:52.236380 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:52.237188 16233 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
I20260709 13:18:52.237357 17323 raft_consensus.cc:3055] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:52.237895 17323 raft_consensus.cc:740] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:52.238951 17323 consensus_queue.cc:260] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:52.240939 17323 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:52.246556 17310 consensus_queue.cc:260] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:52.250479 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:52.251498 15521 raft_consensus.cc:2393] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 3: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:18:52.251830 16204 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:52.252470 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:52.242057 17354 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:52.242630 17357 raft_consensus.cc:3055] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:52.232395 17263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0af5649725244ae599dfc6ed7aeac5f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:52.228379 17284 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:52.223803 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:52.257380 17224 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:52.263089 17224 raft_consensus.cc:697] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:52.264015 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:52.265290 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:52.266016 15524 raft_consensus.cc:2393] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 3: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:18:52.271627 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:52.253903 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:18:52.278221 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.286262 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:52.287050 15524 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:52.288065 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:52.288817 15524 raft_consensus.cc:2393] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 3: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:18:52.289616 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:52.260803 17357 raft_consensus.cc:740] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:52.298887 15527 raft_consensus.cc:2393] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 3: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:18:52.300933 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:52.316053 15527 raft_consensus.cc:1275] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:52.317324 15527 raft_consensus.cc:1240] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:52.273826 17284 raft_consensus.cc:697] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:52.354792 15676 raft_consensus.cc:1275] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:18:52.321447 15527 raft_consensus.cc:1240] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: []
W20260709 13:18:52.356550 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.264074 17224 consensus_queue.cc:237] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:52.357684 17300 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:52.368499 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:52.319237 17307 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:52.405062 15524 raft_consensus.cc:1240] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:52.266390 17354 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:52.413198 17284 consensus_queue.cc:237] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:52.415375 15672 raft_consensus.cc:1275] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:52.376178 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:52.418895 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:52.429382 15526 raft_consensus.cc:2393] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 2: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
W20260709 13:18:52.423916 15493 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.426381 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:52.426781 17336 raft_consensus.cc:3055] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:52.399504 17304 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:18:52.434602 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:52.437417 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:52.438877 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:52.439146 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:52.440816 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:52.368122 17299 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:52.448902 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:52.452488 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:52.454509 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:52.456543 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:52.459870 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:52.463585 17308 consensus_queue.cc:1059] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:52.464399 17308 consensus_queue.cc:1243] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer dd1a14c3173d49dc9b55bb116e3783c9 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:18:52.464845 17308 raft_consensus.cc:3055] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:52.465167 17308 raft_consensus.cc:740] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:18:52.462250 15568 consensus_peers.cc:597] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0af5649725244ae599dfc6ed7aeac5f1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:52.461176 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:52.432193 17336 raft_consensus.cc:740] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:52.450939 17363 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.452342 17343 consensus_queue.cc:1059] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:18:52.495623 15669 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:18:52.494906 17363 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:52.498962 15524 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:52.450557 17227 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:18:52.455811 17235 consensus_queue.cc:1059] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:18:52.502882 15524 raft_consensus.cc:1240] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:52.413813 17340 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:52.451570 17304 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:52.507823 15493 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.469769 17357 consensus_queue.cc:260] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:52.511439 17218 raft_consensus.cc:3055] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:52.511854 17218 raft_consensus.cc:740] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:52.512697 17218 consensus_queue.cc:260] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:52.478787 17219 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:18:52.515403 17235 raft_consensus.cc:3055] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:52.515843 17235 raft_consensus.cc:740] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:52.516314 15675 raft_consensus.cc:1240] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:52.516727 17235 consensus_queue.cc:260] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:52.518695 17285 raft_consensus.cc:3055] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:52.519132 17285 raft_consensus.cc:740] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:52.519981 17285 consensus_queue.cc:260] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:52.521104 17285 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:52.524190 17300 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:18:52.525115 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.528383 15671 raft_consensus.cc:1240] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:52.466495 17308 consensus_queue.cc:260] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:52.534364 17354 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:52.473656 17208 consensus_queue.cc:1059] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:18:52.513813 17218 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:52.548532 15519 raft_consensus.cc:1240] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:52.550025 15519 raft_consensus.cc:1240] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:52.551432 17322 consensus_queue.cc:1048] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:18:52.521116 17304 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:52.535805 17343 consensus_queue.cc:1243] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer dd1a14c3173d49dc9b55bb116e3783c9 log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:18:52.562680 17212 raft_consensus.cc:3055] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:52.559412 17336 consensus_queue.cc:260] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:52.559898 15526 raft_consensus.cc:1240] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:52.534803 17235 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:52.544612 17299 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:52.565022 17212 raft_consensus.cc:740] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:52.568686 17134 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:52.558871 17308 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:52.565837 17322 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.566591 17219 consensus_queue.cc:1048] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 13:18:52.582984 17307 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:18:52.601090 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 488839us
I20260709 13:18:52.595938 17363 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:52.599722 17336 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:52.624182 15641 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:52.586570 17322 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:52.628700 17287 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:52.629179 17287 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:52.629546 17287 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:52.630012 17365 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:52.630934 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:52.622241 17330 consensus_meta.cc:287] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.625s	user 0.012s	sys 0.000s
I20260709 13:18:52.620884 17311 consensus_queue.cc:1059] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:18:52.630551 17365 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:52.636425 17320 consensus_meta.cc:287] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.524s	user 0.008s	sys 0.000s
I20260709 13:18:52.636843 17340 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:52.615166 17307 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
W20260709 13:18:52.624289 17345 consensus_meta.cc:287] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.560s	user 0.006s	sys 0.005s
I20260709 13:18:52.626327 15641 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:52.645763 17353 tablet_copy_client.cc:323] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Beginning tablet copy session from remote peer at address 127.14.23.67:32779
W20260709 13:18:52.688804 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 62566.1us
I20260709 13:18:52.689872 15641 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:52.691674 15641 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:52.629385 17322 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:52.693732 15596 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:52.613139 17212 consensus_queue.cc:260] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:52.680354 17296 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.003s	sys 0.018s
I20260709 13:18:52.709403 17212 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:52.710103 17311 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:52.709555 17360 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:52.711236 17360 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:52.711620 17360 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:52.680536 17329 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:52.723306 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 177550us
I20260709 13:18:52.640148 17330 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:52.694465 17367 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:52.681830 17320 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:52.727877 17322 consensus_queue.cc:1048] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:18:52.745272 17365 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:52.745791 17287 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:52.748129 17287 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:52.692883 17345 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:52.756935 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:52.728662 17296 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.004s	sys 0.018s
I20260709 13:18:52.762112 15528 raft_consensus.cc:2393] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 3: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:18:52.766182 16231 raft_consensus.cc:1240] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:52.768266 16239 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:18:52.730561 17367 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:52.779259 16239 raft_consensus.cc:1240] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: []
W20260709 13:18:52.770956 17348 consensus_meta.cc:287] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.651s	user 0.010s	sys 0.004s
W20260709 13:18:52.773636 15642 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.008s	sys 0.011s
W20260709 13:18:52.785189 15642 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.008s	sys 0.011s
W20260709 13:18:52.795655 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.797580 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:52.797752 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:52.798372 15600 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:52.809432 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.773466 15521 raft_consensus.cc:1275] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:18:52.810329 15567 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:52.831162 15601 raft_consensus.cc:1275] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:52.831825 15597 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:18:52.812484 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 697398us
I20260709 13:18:52.813804 15526 raft_consensus.cc:1275] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:52.814149 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:52.772496 17367 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:52.839704 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:52.840507 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:52.843158 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:52.847337 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:52.849467 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:52.842061 15599 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:18:52.788093 17348 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:18:52.862525 15520 raft_consensus.cc:2393] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:18:52.862707 15526 raft_consensus.cc:1275] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:52.864149 15526 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:18:52.866300 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:18:52.868201 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.871260 15670 raft_consensus.cc:1240] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:52.871941 15600 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:52.780655 17212 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
W20260709 13:18:52.878566 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 49696.9us
I20260709 13:18:52.889353 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:52.786526 17232 consensus_queue.cc:1059] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
W20260709 13:18:52.892854 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:52.908097 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.878566 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:52.858923 17296 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:18:52.925732 15491 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:18:52.930303 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.914239 17212 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:18:52.892930 17220 consensus_queue.cc:1059] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:18:52.932405 17323 raft_consensus.cc:2804] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:52.933079 17323 raft_consensus.cc:697] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
W20260709 13:18:52.934103 17262 consensus_meta.cc:287] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.612s	user 0.005s	sys 0.005s
I20260709 13:18:52.935323 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:52.936163 16239 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:52.936779 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:52.931583 17224 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:52.937199 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:18:52.949891 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 47298.2us
I20260709 13:18:52.937826 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:52.937798 16239 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:52.950987 16234 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:52.951341 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:52.951792 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.954668 15601 raft_consensus.cc:1240] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 1. Current term is 2. Ops: []
W20260709 13:18:52.954811 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.957799 15597 raft_consensus.cc:1240] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:52.960345 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:52.956884 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:52.960659 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:52.966126 17380 consensus_queue.cc:1048] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:18:52.935326 17262 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:53976 (request call id 303) took 4629 ms. Trace:
W20260709 13:18:52.944053 17374 log.cc:927] Time spent T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
W20260709 13:18:52.933344 17387 log.cc:927] Time spent T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:18:52.932660 17220 raft_consensus.cc:3055] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:18:52.971768 17373 log.cc:927] Time spent T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 13:18:52.963215 17340 consensus_queue.cc:1059] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:18:53.002200 17299 raft_consensus.cc:3055] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:52.964073 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:18:52.980121 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:53.003770 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.004276 15601 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:52.944712 17330 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:52.960345 17320 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:52.934122 17323 consensus_queue.cc:237] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:53.007221 15523 raft_consensus.cc:1275] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:52.921131 17284 consensus_queue.cc:1048] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:18:52.938475 17360 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:52.995427 15493 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.974053 17262 rpcz_store.cc:276] 0709 13:18:48.313483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.315768 (+  2285us) service_pool.cc:224] Handling call
0709 13:18:48.362961 (+ 47193us) ts_tablet_manager.cc:934] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
0709 13:18:51.497568 (+3134607us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:18:52.212955 (+715387us) tablet_metadata.cc:665] Metadata flushed
0709 13:18:52.217278 (+  4323us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:18:52.934565 (+717287us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:18:52.934869 (+   304us) tablet_replica.cc:206] Creating consensus instance
0709 13:18:52.935191 (+   322us) ts_tablet_manager.cc:989] Replica registered.
0709 13:18:52.935248 (+    57us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":31131,"tablet-copy.queue_time_us":41461,"thread_start_us":110999,"threads_started":2}
I20260709 13:18:53.002645 17299 raft_consensus.cc:740] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:53.006215 17376 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:53.018838 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.017681 17262 tablet_copy_client.cc:806] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:53.018162 17360 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:53.018728 17376 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:52.989516 17220 raft_consensus.cc:740] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:53.026533 17262 tablet_copy_client.cc:670] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:53.015419 17263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:57258 (request call id 516) took 4673 ms. Trace:
I20260709 13:18:53.024500 17218 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:18:53.023010 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 212756us
I20260709 13:18:53.035564 15567 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:53.029567 17299 consensus_queue.cc:260] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:53.053687 15493 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.092427 15601 raft_consensus.cc:1240] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:53.093180 16234 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:53.135361 15642 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:53.136229 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:18:53.038475 17221 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.017s	sys 0.020s
W20260709 13:18:53.139088 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.138953 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:53.139532 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:53.142225 15601 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:53.140445 17307 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.143095 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:53.145079 17307 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:53.146309 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 110804us
I20260709 13:18:53.147213 17307 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:53.147864 15567 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:53.151572 17345 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:53.153342 17227 raft_consensus.cc:3055] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:53.153784 17227 raft_consensus.cc:740] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:53.154716 17227 consensus_queue.cc:260] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:53.155289 17299 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:53.156065 17227 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:53.154745 17220 consensus_queue.cc:260] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:53.159142 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:53.159561 17220 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:53.161789 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 134013us
I20260709 13:18:53.162492 17376 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:18:53.141070 17221 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.018s	sys 0.021s
I20260709 13:18:53.165933 17376 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:53.173420 15601 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:53.174160 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1482) took 1039 ms. Trace:
I20260709 13:18:53.187307 15523 rpcz_store.cc:276] 0709 13:18:52.135110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.399800 (+264690us) service_pool.cc:224] Handling call
0709 13:18:52.400158 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:53.007599 (+607441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.157782 (+150183us) spinlock_profiling.cc:243] Waited 607 ms on lock 0x7b54003cd2d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:18:53.174084 (+ 16302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1395354368}
I20260709 13:18:53.151209 15567 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:53.180734 15677 raft_consensus.cc:1240] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:53.182924 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:53.135259 17348 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:53.191596 15567 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:53.195097 15567 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:53.197219 15567 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:53.200350 15567 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:53.205237 15567 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:53.241683 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:53.237982 17227 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:53.248416 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:53.255383 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:53.255154 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 119714us
I20260709 13:18:53.257908 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:53.184641 17233 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:53.237057 17403 log.cc:927] Time spent T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:18:53.190311 17376 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:53.257591 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:53.269472 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:53.234799 17390 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:53.234907 17235 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:53.265123 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:53.234977 17308 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:53.238032 17208 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:53.279110 15567 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:53.234187 17218 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:53.184350 17369 consensus_queue.cc:1059] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 13:18:53.240716 17232 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:18:53.258932 17402 log.cc:927] Time spent T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:18:53.268496 17227 consensus_queue.cc:1059] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:18:53.282321 15600 raft_consensus.cc:2393] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 5: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:53.269038 17376 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:53.270031 17330 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:53.285091 15642 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:53.269232 17295 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.250217 17367 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:53.278326 15598 raft_consensus.cc:1240] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:53.028899 17263 rpcz_store.cc:276] 0709 13:18:48.342123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.342926 (+   803us) service_pool.cc:224] Handling call
0709 13:18:48.544295 (+201369us) ts_tablet_manager.cc:934] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
0709 13:18:52.046112 (+3501817us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:18:52.618694 (+572582us) tablet_metadata.cc:665] Metadata flushed
0709 13:18:52.622767 (+  4073us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:18:52.941641 (+318874us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:18:52.958924 (+ 17283us) tablet_replica.cc:206] Creating consensus instance
0709 13:18:53.005422 (+ 46498us) ts_tablet_manager.cc:989] Replica registered.
0709 13:18:53.011185 (+  5763us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":15425,"tablet-copy.queue_time_us":117250,"thread_start_us":257328,"threads_started":2}
W20260709 13:18:53.384181 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:53.512317 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:53.399281 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 45863.9us
W20260709 13:18:53.512727 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.364086 17368 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:53.392127 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.296527 17308 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:18:53.480998 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.293942 17212 consensus_queue.cc:1048] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.216s
W20260709 13:18:53.421411 17406 log.cc:927] Time spent T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:18:53.514940 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:53.286692 17409 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:53.397686 17375 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.511056 17354 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:53.519431 16238 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.018s	sys 0.048s
I20260709 13:18:53.292657 17408 consensus_queue.cc:1059] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.222s
W20260709 13:18:53.521412 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 236377us
W20260709 13:18:53.523077 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.470561 17390 consensus_queue.cc:1059] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:18:53.531330 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:53.532524 15675 raft_consensus.cc:1240] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:53.521008 17232 consensus_queue.cc:1059] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:18:53.522058 17263 tablet_copy_client.cc:806] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:53.509831 17295 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:53.534338 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.534866 15674 raft_consensus.cc:1240] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:53.535794 17308 raft_consensus.cc:697] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:53.528801 17330 raft_consensus.cc:697] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:53.536725 15672 raft_consensus.cc:1240] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:53.554677 17320 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:53.555429 16238 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.018s	sys 0.048s
W20260709 13:18:53.534842 17404 log.cc:927] Time spent T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:18:53.553224 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:53.537137 17323 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.024s	sys 0.052s
I20260709 13:18:53.558533 15642 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:53.536798 17365 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:53.559849 17375 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:53.521236 17227 consensus_queue.cc:1048] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 13:18:53.528052 17368 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:53.636972 17375 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:53.637194 15599 raft_consensus.cc:606] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Received request to transfer leadership to TS dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:53.642673 16235 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:18:53.561209 16240 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 338) took 1834 ms (client timeout 2026 ms). Trace:
I20260709 13:18:53.562333 15596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 1478) took 1136 ms. Trace:
I20260709 13:18:53.571197 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:53.573109 15595 raft_consensus.cc:1240] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:53.537041 17409 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:18:53.648398 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 90000.1us
I20260709 13:18:53.648770 15603 raft_consensus.cc:2393] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 3: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
W20260709 13:18:53.517482 15641 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.817s	user 0.031s	sys 0.092s
I20260709 13:18:53.563643 17212 consensus_queue.cc:1059] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:18:53.585045 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:53.588999 17390 raft_consensus.cc:3055] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:53.634649 17327 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:53.635002 17365 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.574970 17354 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:53.647449 15596 rpcz_store.cc:276] 0709 13:18:52.425635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.427668 (+  2033us) service_pool.cc:224] Handling call
0709 13:18:52.435580 (+  7912us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:52.696840 (+261260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.562085 (+865245us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b5400169c54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:18:53.562276 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":593293184}
I20260709 13:18:53.599366 17263 tablet_copy_client.cc:670] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:53.650142 15602 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 2: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:53.595368 17232 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.594144 15598 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 3: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:53.671944 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:53.673305 15526 raft_consensus.cc:1275] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:18:53.628218 17323 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.025s	sys 0.052s
I20260709 13:18:53.626464 17345 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.673760 17357 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:53.646629 16240 rpcz_store.cc:269] 0709 13:18:51.726740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.727939 (+  1199us) service_pool.cc:224] Handling call
0709 13:18:53.561006 (+1833067us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400491bd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:53.561151 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2308011392}
I20260709 13:18:53.668656 17369 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:53.677711 17365 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:53.649194 15641 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.949s	user 0.031s	sys 0.092s
W20260709 13:18:53.683516 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 145084us
I20260709 13:18:53.597036 17367 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:53.649202 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:53.584259 17343 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:53.668303 17390 raft_consensus.cc:740] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:53.706746 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1483) took 1559 ms. Trace:
W20260709 13:18:53.711637 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.714046 15519 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:53.647838 17320 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:53.738981 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 1.04732e+06us
W20260709 13:18:53.738708 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 89600.8us
I20260709 13:18:53.701267 16231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 752) took 1734 ms. Trace:
W20260709 13:18:53.742568 16236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 337) took 2016 ms (client timeout 2004 ms). Trace:
I20260709 13:18:53.751842 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:53.767002 15642 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:53.650905 17409 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:53.711196 17310 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:53.697741 17316 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:53.693423 17220 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.764954 15641 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:53.722088 17326 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:53.768301 16236 rpcz_store.cc:269] 0709 13:18:51.726464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.726710 (+   246us) service_pool.cc:224] Handling call
0709 13:18:53.742356 (+2015646us) spinlock_profiling.cc:243] Waited 995 ms on lock 0x7b5400724a54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:53.742498 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2289122560}
I20260709 13:18:53.588583 17308 consensus_queue.cc:237] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:53.721400 17375 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.767053 15603 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:18:53.739743 17290 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:53.773090 15641 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:53.773247 15602 raft_consensus.cc:1275] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:53.745839 17309 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:53.745553 17212 raft_consensus.cc:3055] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:53.774075 15642 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:53.721655 17232 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:53.766479 17398 log.cc:927] Time spent T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:18:53.768900 17367 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:18:53.779901 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 86247.6us
I20260709 13:18:53.781141 17348 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.749655 17369 raft_consensus.cc:697] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:53.775970 17408 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:18:53.794265 15493 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.787026 15641 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:18:53.787755 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 84814.6us
I20260709 13:18:53.746400 17345 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:53.773973 17368 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:53.750627 17343 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:53.821352 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:53.785753 15599 raft_consensus.cc:3055] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:53.826104 16204 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:53.826809 15602 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:53.811671 17409 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:53.789412 17212 raft_consensus.cc:740] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:53.790351 17360 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:53.797888 17323 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.831290 16238 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 765) took 1050 ms. Trace:
W20260709 13:18:53.845547 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 58586.2us
I20260709 13:18:53.800338 17391 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:53.850858 15641 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:53.855545 15641 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:53.853606 15493 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:53.776001 17220 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:53.856763 15493 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:53.748886 15521 rpcz_store.cc:276] 0709 13:18:52.147620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.423030 (+275410us) service_pool.cc:224] Handling call
0709 13:18:52.423448 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:52.814864 (+391416us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.706457 (+891593us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b54002de1d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:18:53.706677 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":753232640}
I20260709 13:18:53.858279 15641 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:53.795974 17375 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:53.817381 17408 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:53.862030 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:53.828127 17367 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:53.783509 17326 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:53.864784 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:53.798533 17290 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:53.828739 15599 raft_consensus.cc:740] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:53.766489 16231 rpcz_store.cc:276] 0709 13:18:51.967767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.969225 (+  1458us) service_pool.cc:224] Handling call
0709 13:18:51.992800 (+ 23575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:52.766754 (+773954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.700982 (+934228us) spinlock_profiling.cc:243] Waited 773 ms on lock 0x7b540072ea54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:18:53.701195 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1778068736}
I20260709 13:18:53.772627 17330 consensus_queue.cc:237] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:53.866232 16236 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:53.870980 16232 raft_consensus.cc:1240] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:53.872524 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:53.833765 16238 rpcz_store.cc:276] 0709 13:18:52.780979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.781126 (+   147us) service_pool.cc:224] Handling call
0709 13:18:52.781474 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:52.814142 (+ 32668us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:52.814173 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:52.867618 (+ 53445us) log.cc:844] Serialized 26 byte log entry
0709 13:18:53.598893 (+731275us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:53.634309 (+ 35416us) log.cc:844] Serialized 14 byte log entry
0709 13:18:53.634567 (+   258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.634645 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:53.831038 (+196393us) raft_consensus.cc:1685] finished
0709 13:18:53.831060 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:53.831211 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29899,"prepare.queue_time_us":36273,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":87797,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":12313,"thread_start_us":730861,"threads_started":1,"wal-append.queue_time_us":806353}
I20260709 13:18:53.878480 16235 raft_consensus.cc:1240] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:53.883960 16240 raft_consensus.cc:2393] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 2: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:18:53.822544 17348 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:53.885533 15523 raft_consensus.cc:1240] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:53.869068 17408 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:53.857151 17323 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:53.891913 17295 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:53.898715 17416 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:18:53.903909 17428 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:53.904369 17428 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.904712 17428 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:53.882865 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:53.912693 17354 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:53.912976 17345 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:53.913718 17345 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.914213 17345 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:53.901607 17430 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:53.890645 17300 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:53.890543 17367 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:53.879159 17227 consensus_queue.cc:1048] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:18:53.906576 17391 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:53.910532 17233 consensus_meta.cc:287] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.642s	user 0.010s	sys 0.000s
I20260709 13:18:53.923614 17235 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:53.951098 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.925709 17320 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:53.923018 15521 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:53.946530 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:53.953094 17431 raft_consensus.cc:2764] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:18:53.948691 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:18:53.956765 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:53.961494 17419 log.cc:927] Time spent T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:18:53.961647 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:53.961774 17408 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:53.897182 17212 consensus_queue.cc:260] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:53.947111 17233 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:53.942296 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:53.933872 17430 raft_consensus.cc:697] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:53.972085 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:53.962404 17220 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:53.882961 17369 consensus_queue.cc:237] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:53.929060 15599 consensus_queue.cc:260] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:53.990166 16204 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:53.988037 17326 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:53.968166 17232 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:53.991786 17263 tablet_copy_client.cc:538] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:53.991485 17385 raft_consensus.cc:3055] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:53.994421 17220 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:53.956104 17235 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:54.014606 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:54.038287 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.051657 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:54.063035 17220 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.992574 17409 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:54.100124 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:54.004593 17345 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:54.115752 17427 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:54.115116 15677 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 1.
I20260709 13:18:54.072419 17212 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:54.073740 17385 raft_consensus.cc:740] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:54.125120 16235 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:54.079698 17220 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:54.098266 17290 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:54.080497 17375 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:54.161733 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:54.116421 17427 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:54.163668 16203 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:54.103698 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:54.098417 17343 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:18:54.156121 17417 log.cc:927] Time spent T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 13:18:54.164455 17262 tablet_copy_client.cc:538] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:54.123607 15599 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:54.167598 15677 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:18:54.122797 17425 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.166834 17275 tablet_copy_client.cc:538] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:54.164741 17380 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.119063 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:54.127960 17348 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:54.175836 17287 raft_consensus.cc:3055] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:54.173774 17375 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:54.116606 17336 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:54.164829 17323 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:54.099881 15687 tablet_copy_service.cc:140] P 3ef186ec12554a1d87814af0a0ace9a4: Received BeginTabletCopySession request for tablet bd36c2cfd99c45cea9e48096d54bed1a from peer dd1a14c3173d49dc9b55bb116e3783c9 ({username='slave'} at 127.0.0.1:57710)
W20260709 13:18:54.217648 15528 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1468) took 2128 ms (client timeout 2064 ms). Trace:
W20260709 13:18:54.230152 15528 rpcz_store.cc:269] 0709 13:18:52.088839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.150752 (+ 61913us) service_pool.cc:224] Handling call
0709 13:18:54.217389 (+2066637us) spinlock_profiling.cc:243] Waited 579 ms on lock 0x7b540029ccd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:54.217565 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1331613056}
I20260709 13:18:54.175748 17235 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:54.233132 15674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49094 (request call id 1579) took 1372 ms. Trace:
I20260709 13:18:54.233585 15674 rpcz_store.cc:276] 0709 13:18:52.860589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.860864 (+   275us) service_pool.cc:224] Handling call
0709 13:18:53.534657 (+673793us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:53.578184 (+ 43527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:54.232902 (+654718us) spinlock_profiling.cc:243] Waited 673 ms on lock 0x7b540046ee50. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:54.233061 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1548864384}
I20260709 13:18:54.234939 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:54.236085 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:54.236726 15674 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:18:54.239135 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:54.239827 15527 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:54.223132 17427 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:54.237653 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:54.226987 17355 raft_consensus.cc:3055] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:54.226840 15673 raft_consensus.cc:2410] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:18:54.169375 17385 consensus_queue.cc:260] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:54.256345 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:54.224941 17287 raft_consensus.cc:740] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:54.226239 17375 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:54.273007 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:54.268988 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:54.274497 15674 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:18:54.279958 15674 raft_consensus.cc:1240] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
W20260709 13:18:54.261857 17299 consensus_meta.cc:287] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 1.103s	user 0.008s	sys 0.005s
W20260709 13:18:54.269085 17376 consensus_meta.cc:287] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.797s	user 0.010s	sys 0.008s
I20260709 13:18:54.260495 17306 raft_consensus.cc:3055] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:18:54.288945 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.258740 17408 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:54.290035 15524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 364) took 1306 ms. Trace:
I20260709 13:18:54.268603 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:54.228787 17336 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:54.293112 15524 rpcz_store.cc:276] 0709 13:18:52.983442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.184426 (+200984us) service_pool.cc:224] Handling call
0709 13:18:53.188184 (+  3758us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:53.188552 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:53.188581 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:53.241152 (+ 52571us) log.cc:844] Serialized 26 byte log entry
0709 13:18:53.709005 (+467853us) spinlock_profiling.cc:243] Waited 4.91 ms on lock 0x7b18003e00e0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab0d1 00007f5ce49c6a0f
0709 13:18:53.829261 (+120256us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:53.829299 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.829362 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:54.285025 (+455663us) raft_consensus.cc:1685] finished
0709 13:18:54.285051 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:54.289965 (+  4914us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1808,"raft.queue_time_us":45658,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":9608,"spinlock_wait_cycles":11300224,"thread_start_us":118655,"threads_started":1,"wal-append.queue_time_us":963911}
W20260709 13:18:54.317291 17365 consensus_meta.cc:287] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.567s	user 0.008s	sys 0.005s
W20260709 13:18:54.343768 16205 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.074s	sys 0.207s
W20260709 13:18:54.344254 16205 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.074s	sys 0.207s
I20260709 13:18:54.275744 17375 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:54.256168 15687 tablet_copy_service.cc:161] P 3ef186ec12554a1d87814af0a0ace9a4: Beginning new tablet copy session on tablet bd36c2cfd99c45cea9e48096d54bed1a from peer dd1a14c3173d49dc9b55bb116e3783c9 at {username='slave'} at 127.0.0.1:57710: session id = dd1a14c3173d49dc9b55bb116e3783c9-bd36c2cfd99c45cea9e48096d54bed1a
I20260709 13:18:54.269503 17355 raft_consensus.cc:740] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:18:54.332480 15602 consensus_meta.cc:287] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.504s	user 0.007s	sys 0.007s
I20260709 13:18:54.319115 15528 raft_consensus.cc:1275] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:54.347023 15602 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:18:54.411743 15677 raft_consensus.cc:1240] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:54.229544 17380 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:54.318658 17306 raft_consensus.cc:740] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:54.327594 17316 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.333760 17354 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:54.430022 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:54.440649 15601 raft_consensus.cc:2393] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 3: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
W20260709 13:18:54.432967 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.434341 15595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 1543) took 1622 ms. Trace:
I20260709 13:18:54.441622 15595 rpcz_store.cc:276] 0709 13:18:52.811757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.831333 (+ 19576us) service_pool.cc:224] Handling call
0709 13:18:53.572894 (+741561us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:53.647189 (+ 74295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:54.434107 (+786918us) spinlock_profiling.cc:243] Waited 692 ms on lock 0x7b540030dc50. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:54.434269 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1591845760}
W20260709 13:18:54.446806 15520 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1476) took 2332 ms (client timeout 1948 ms). Trace:
I20260709 13:18:54.447985 15601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53324 (request call id 1550) took 1198 ms. Trace:
W20260709 13:18:54.447254 15520 rpcz_store.cc:269] 0709 13:18:52.094866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.149513 (+ 54647us) service_pool.cc:224] Handling call
0709 13:18:54.427503 (+2277990us) spinlock_profiling.cc:243] Waited 712 ms on lock 0x7b540029bdd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:54.427657 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1636564736}
I20260709 13:18:54.291096 17376 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:54.448355 15601 rpcz_store.cc:276] 0709 13:18:53.249957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.250572 (+   615us) service_pool.cc:224] Handling call
0709 13:18:54.429817 (+1179245us) spinlock_profiling.cc:243] Waited 3.01 ms on lock 0x7b5800113498. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99d10a0 00007f5ce99d0984 00007f5ce2f4b6d4 00007f5ce2f4b662
0709 13:18:54.447925 (+ 18108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6917504}
I20260709 13:18:54.423032 17391 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:54.500968 15599 raft_consensus.cc:1275] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:54.478484 15673 raft_consensus.cc:1275] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:54.420249 17219 consensus_queue.cc:1059] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:18:54.408360 17365 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:54.442942 15594 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53324 (request call id 1544) took 1617 ms (client timeout 1691 ms). Trace:
I20260709 13:18:54.514581 15669 raft_consensus.cc:1240] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:54.505721 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:54.516218 15675 raft_consensus.cc:1240] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:54.518602 15677 raft_consensus.cc:2393] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 3: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:18:54.224608 17430 consensus_queue.cc:237] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:54.231727 17425 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:54.513901 17324 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:18:54.475559 17316 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:54.426501 17287 consensus_queue.cc:260] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:54.475956 17354 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:54.549867 17423 raft_consensus.cc:3055] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:54.550406 17423 raft_consensus.cc:740] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:18:54.557312 16206 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.000s	sys 0.021s
W20260709 13:18:54.557730 16206 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.000s	sys 0.021s
W20260709 13:18:54.566587 15642 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.043s	sys 0.051s
W20260709 13:18:54.567027 15642 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.043s	sys 0.051s
W20260709 13:18:54.567303 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 777462us
I20260709 13:18:54.569316 17391 consensus_queue.cc:1059] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:54.569602 17311 consensus_queue.cc:1059] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:54.570622 17431 raft_consensus.cc:3055] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:54.570895 17409 raft_consensus.cc:3055] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:54.570991 17431 raft_consensus.cc:740] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:54.571266 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:54.572175 17431 consensus_queue.cc:260] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:54.572839 17311 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:54.571434 17409 raft_consensus.cc:740] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:54.573792 17431 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:54.574555 17409 consensus_queue.cc:260] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:54.574596 15642 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:54.575999 17409 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:54.576116 17311 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:18:54.582077 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 237145us
W20260709 13:18:54.589586 15642 connection.cc:256] RPC call timeout handler was delayed by -1.4415650809999079s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20040000000000002s for a response
W20260709 13:18:54.590094 15642 connection.cc:256] RPC call timeout handler was delayed by -1.4217650809998759s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2026s for a response
W20260709 13:18:54.585268 15523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.025s	sys 0.020s
W20260709 13:18:54.590095 15519 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.009s	sys 0.004s
W20260709 13:18:54.590847 15523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.026s	sys 0.020s
W20260709 13:18:54.591192 15519 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.009s	sys 0.004s
I20260709 13:18:54.532335 17284 consensus_queue.cc:1048] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:18:54.608572 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:54.608615 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:54.610522 15642 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:54.618498 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:54.619563 15674 raft_consensus.cc:2410] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:18:54.626765 16238 consensus_meta.cc:287] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.649s	user 0.004s	sys 0.004s
I20260709 13:18:54.627209 16238 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:54.551200 17423 consensus_queue.cc:260] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:54.652393 17423 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:54.663080 16240 consensus_meta.cc:287] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.696s	user 0.001s	sys 0.007s
I20260709 13:18:54.663359 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:54.664199 15674 raft_consensus.cc:2410] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:54.664933 16240 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:54.517663 17306 consensus_queue.cc:260] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:54.669698 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:54.668720 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 55613.2us
I20260709 13:18:54.668464 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:54.671881 16239 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:18:54.668464 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:54.673062 15674 raft_consensus.cc:2393] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 3: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:18:54.617391 17431 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:18:54.517025 15594 rpcz_store.cc:269] 0709 13:18:52.825893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.861614 (+ 35721us) service_pool.cc:224] Handling call
0709 13:18:54.252605 (+1390991us) spinlock_profiling.cc:243] Waited 7.72 ms on lock 0x7b5800113498. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99d10a0 00007f5ce99d0984 00007f5ce2f4b6d4 00007f5ce2f4b662
0709 13:18:54.429494 (+176889us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17748864}
I20260709 13:18:54.697156 17306 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:54.698385 15598 raft_consensus.cc:1240] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:54.677690 15602 raft_consensus.cc:1240] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:54.699080 15677 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:18:54.711588 15677 raft_consensus.cc:606] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Received request to transfer leadership to TS dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:54.702190 15599 raft_consensus.cc:1240] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:54.713085 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:54.705885 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:54.714073 15601 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:54.714147 15677 raft_consensus.cc:614] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:54.674186 17224 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:18:54.706869 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:54.715153 15595 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:54.716163 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:54.710001 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:54.711282 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:54.678390 15603 raft_consensus.cc:1240] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:54.668519 17287 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:54.696749 17423 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.721316 17218 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:54.728999 17218 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:54.730294 17310 consensus_meta.cc:287] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.958s	user 0.009s	sys 0.002s
I20260709 13:18:54.708796 17336 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:54.786474 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:18:54.794216 17376 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:54.783541 17218 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:54.786115 17365 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:54.785588 17316 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:18:54.787689 17327 consensus_meta.cc:287] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 1.109s	user 0.006s	sys 0.005s
W20260709 13:18:54.800482 17360 consensus_meta.cc:287] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.950s	user 0.007s	sys 0.000s
I20260709 13:18:54.800928 17360 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:18:54.663880 16232 consensus_meta.cc:287] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.678s	user 0.001s	sys 0.008s
I20260709 13:18:54.809331 15522 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
W20260709 13:18:54.762108 17357 consensus_meta.cc:287] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 1.010s	user 0.008s	sys 0.003s
I20260709 13:18:54.721364 17219 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.812039 17380 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:54.814052 15669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49094 (request call id 1584) took 1921 ms. Trace:
W20260709 13:18:54.814811 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.816743 17384 raft_consensus.cc:3055] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:54.731014 17354 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:18:54.784372 17212 consensus_meta.cc:287] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.624s	user 0.004s	sys 0.004s
I20260709 13:18:54.811022 15670 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:54.831252 17285 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:54.860037 15641 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.000s	user 0.005s	sys 0.013s
I20260709 13:18:54.755679 17423 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:54.816663 17262 tablet_bootstrap.cc:492] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:54.822382 17263 tablet_bootstrap.cc:492] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
W20260709 13:18:54.754801 17428 consensus_meta.cc:287] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.850s	user 0.007s	sys 0.010s
I20260709 13:18:54.851684 15676 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:54.680557 17220 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:54.860356 15596 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.032s	user 0.000s	sys 0.052s
W20260709 13:18:54.892937 15596 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.065s	user 0.000s	sys 0.054s
I20260709 13:18:54.877231 15669 rpcz_store.cc:276] 0709 13:18:52.892634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.281118 (+388484us) service_pool.cc:224] Handling call
0709 13:18:53.356890 (+ 75772us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:54.526900 (+1170010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:54.813799 (+286899us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54003886d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:18:54.813992 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2662045440}
I20260709 13:18:54.852264 15674 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:54.878906 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:54.895037 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:54.895666 16238 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:54.746726 17375 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:54.896467 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:54.896965 16238 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:54.880004 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:54.897619 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:54.898022 16234 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:54.898974 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:54.899973 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:54.900909 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:54.902971 16234 raft_consensus.cc:1240] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 1. Current term is 2. Ops: []
W20260709 13:18:54.876479 15641 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.017s	user 0.005s	sys 0.014s
W20260709 13:18:54.904515 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 1.04622e+06us
I20260709 13:18:54.905429 15641 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:54.877463 17285 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:54.908841 17357 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:54.851038 16232 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:54.908994 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:18:54.912998 15640 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.052s	user 0.013s	sys 0.059s
W20260709 13:18:54.913399 15640 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.052s	user 0.013s	sys 0.059s
W20260709 13:18:54.914505 16237 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.166s	user 0.019s	sys 0.010s
W20260709 13:18:54.914898 16237 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.167s	user 0.019s	sys 0.010s
I20260709 13:18:54.915493 16239 raft_consensus.cc:1240] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:54.881429 17384 raft_consensus.cc:740] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:54.909358 17357 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:54.919183 15491 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.015s	sys 0.017s
W20260709 13:18:54.919597 15491 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.015s	sys 0.017s
W20260709 13:18:54.919911 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 708028us
I20260709 13:18:54.920461 15641 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:54.916175 16237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 764) took 2139 ms. Trace:
I20260709 13:18:54.876549 17219 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:54.914644 17221 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.981s	user 0.008s	sys 0.024s
I20260709 13:18:55.038547 15528 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:18:54.997005 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:55.021724 15493 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.110s	user 0.012s	sys 0.010s
W20260709 13:18:55.041944 15493 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.182s	user 0.013s	sys 0.010s
I20260709 13:18:55.043819 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:55.044464 15669 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:18:54.931263 16237 rpcz_store.cc:276] 0709 13:18:52.776473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.777215 (+   742us) service_pool.cc:224] Handling call
0709 13:18:52.778237 (+  1022us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:52.778572 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:52.778596 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:53.596966 (+818370us) spinlock_profiling.cc:243] Waited 2.69 ms on lock 0x7b1800092380. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce18ec2a6 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce3304a11 00007f5ce3314372 00007f5ce4a46a01 00007f5ce4a0b43a
0709 13:18:53.597597 (+   631us) spinlock_profiling.cc:243] Waited 4.27 ms on lock 0x7b2c00047828. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce3304a11 00007f5ce3314372 00007f5ce4a46a01 00007f5ce4a0b43a 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2
0709 13:18:53.630438 (+ 32841us) log.cc:844] Serialized 26 byte log entry
0709 13:18:53.720829 (+ 90391us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:53.748011 (+ 27182us) log.cc:844] Serialized 14 byte log entry
0709 13:18:54.915255 (+1167244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:54.915909 (+   654us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:54.915962 (+    53us) raft_consensus.cc:1685] finished
0709 13:18:54.915974 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:54.916115 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40920,"prepare.queue_time_us":26996,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":38969,"raft.run_cpu_time_us":375,"raft.run_wall_time_us":28139,"spinlock_wait_cycles":16011904,"thread_start_us":1256739,"threads_started":2,"wal-append.queue_time_us":165610,"wal-append.run_cpu_time_us":2155,"wal-append.run_wall_time_us":1059386}
I20260709 13:18:55.046469 15528 raft_consensus.cc:1240] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
W20260709 13:18:55.043154 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 1.18637e+06us
I20260709 13:18:55.049578 15493 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:54.933540 17361 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:54.980317 17295 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.017s	user 0.007s	sys 0.026s
I20260709 13:18:54.927068 17357 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:54.950260 17307 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:55.035038 15520 raft_consensus.cc:1275] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:18:55.021978 17368 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.092s	user 0.003s	sys 0.019s
W20260709 13:18:55.030184 17330 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.112s	user 0.005s	sys 0.024s
I20260709 13:18:54.929783 15641 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:54.943428 16238 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:54.909618 17428 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:55.022329 15567 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.192s	user 0.005s	sys 0.022s
I20260709 13:18:55.054010 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:18:55.055835 15492 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.167s	user 0.005s	sys 0.024s
I20260709 13:18:55.028062 17329 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:55.032058 17430 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.005s	sys 0.016s
I20260709 13:18:55.064070 15566 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:18:55.032795 17369 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.821s	user 0.031s	sys 0.026s
I20260709 13:18:55.070436 15528 raft_consensus.cc:3055] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:18:55.043607 17221 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.116s	user 0.012s	sys 0.028s
W20260709 13:18:55.066948 17295 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.109s	user 0.007s	sys 0.027s
I20260709 13:18:55.096975 16204 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:18:55.112177 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:55.079066 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 149360us
W20260709 13:18:55.074856 15521 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.009s	sys 0.016s
W20260709 13:18:55.077833 17330 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.160s	user 0.006s	sys 0.024s
W20260709 13:18:55.077991 17368 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.177s	user 0.003s	sys 0.020s
I20260709 13:18:55.101132 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:55.101648 17435 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:55.105268 15567 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.274s	user 0.009s	sys 0.024s
W20260709 13:18:55.094399 17343 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.006s	sys 0.015s
W20260709 13:18:55.107836 15492 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.217s	user 0.005s	sys 0.024s
W20260709 13:18:55.124667 17369 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.913s	user 0.031s	sys 0.027s
W20260709 13:18:55.153541 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.146791 15526 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:55.068212 17442 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:55.110683 16236 raft_consensus.cc:2368] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4.
W20260709 13:18:55.111750 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 57864.8us
W20260709 13:18:55.112424 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 48450.1us
I20260709 13:18:55.155666 15493 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:55.117123 17355 consensus_queue.cc:260] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:55.117830 15596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 257) took 1367 ms. Trace:
I20260709 13:18:55.118228 17327 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:55.124195 15528 raft_consensus.cc:740] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:55.134065 15423 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49144 (request call id 348) took 4251 ms. Trace:
W20260709 13:18:55.134440 15521 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.009s	sys 0.016s
I20260709 13:18:55.125344 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:18:55.110111 17430 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.005s	sys 0.017s
I20260709 13:18:55.113585 17300 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:55.115767 17316 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:18:55.151311 15527 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.816s	user 0.008s	sys 0.024s
W20260709 13:18:55.151795 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 1.32113e+06us
I20260709 13:18:55.100422 17365 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:18:55.156653 16205 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.008s	sys 0.016s
I20260709 13:18:55.156428 15596 rpcz_store.cc:276] 0709 13:18:53.750133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.750550 (+   417us) service_pool.cc:224] Handling call
0709 13:18:53.774472 (+ 23922us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:53.784617 (+ 10145us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:53.784653 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:53.800485 (+ 15832us) log.cc:844] Serialized 26 byte log entry
0709 13:18:54.893457 (+1092972us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:54.893501 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:54.893572 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:55.117621 (+224049us) raft_consensus.cc:1685] finished
0709 13:18:55.117640 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:55.117769 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20278,"raft.queue_time_us":86127,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":12223,"thread_start_us":1065189,"threads_started":1,"wal-append.queue_time_us":1261236}
I20260709 13:18:55.112179 17329 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:55.135304 17301 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:18:55.249594 16205 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.008s	sys 0.017s
W20260709 13:18:55.273147 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 790819us
W20260709 13:18:55.245316 15527 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.009s	sys 0.024s
W20260709 13:18:55.245088 15567 leader_election.cc:336] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.844s (SENT)
W20260709 13:18:55.275779 15567 leader_election.cc:336] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.948s (SENT)
W20260709 13:18:55.244644 15519 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.041s	sys 0.025s
W20260709 13:18:55.246096 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.243938 15528 consensus_queue.cc:260] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:55.272881 16235 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.023s	sys 0.056s
I20260709 13:18:55.276180 15567 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:18:55.278031 15519 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.042s	sys 0.025s
I20260709 13:18:55.157428 17220 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:55.181260 17423 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:18:55.276705 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.157505 17443 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.245818 17316 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.277456 17416 consensus_queue.cc:1059] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:18:55.286168 17390 consensus_queue.cc:260] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:55.280200 17430 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:55.155440 17380 consensus_queue.cc:1048] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.122s
I20260709 13:18:55.292184 17369 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:55.273073 17301 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:55.294504 17438 log.cc:927] Time spent T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 13:18:55.244311 17445 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.115s
I20260709 13:18:55.295920 17376 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:55.297030 17385 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:55.282970 17442 consensus_queue.cc:1059] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:18:55.304037 16205 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:18:55.277225 16235 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.023s	sys 0.056s
W20260709 13:18:55.305680 16203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.032s	sys 0.041s
I20260709 13:18:55.302095 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:55.270968 17375 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:55.318382 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:55.333695 17329 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:55.320958 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:55.321485 15596 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:55.321599 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:55.241710 17326 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
W20260709 13:18:55.326789 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:55.380926 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 225353us
I20260709 13:18:55.282315 17391 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:55.383033 17391 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:55.294377 15673 raft_consensus.cc:1240] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:55.248564 17300 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:55.302632 17219 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:55.307157 17327 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.311257 17308 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:55.394954 15493 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:55.297465 17275 tablet_bootstrap.cc:492] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
W20260709 13:18:55.353180 16203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.872s	user 0.032s	sys 0.042s
I20260709 13:18:55.404240 16203 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:55.158277 15423 rpcz_store.cc:276] 0709 13:18:50.871651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.871829 (+   178us) service_pool.cc:224] Handling call
0709 13:18:55.123117 (+4251288us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:51.922454 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:52.038008 (+115554us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:52.038053 (+    45us) write_op.cc:435] Acquired schema lock
0709 13:18:52.039512 (+  1459us) tablet.cc:662] Decoding operations
0709 13:18:52.091443 (+ 51931us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:52.091507 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:52.091535 (+    28us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:18:52.113765 (+ 22230us) tablet.cc:701] Row locks acquired
0709 13:18:52.114638 (+   873us) write_op.cc:260] PREPARE: finished
0709 13:18:52.137910 (+ 23272us) write_op.cc:270] Start()
0709 13:18:52.145336 (+  7426us) write_op.cc:276] Timestamp: P: 1783603132135846 usec, L: 0
0709 13:18:52.148939 (+  3603us) op_driver.cc:348] REPLICATION: starting
0709 13:18:52.224638 (+ 75699us) log.cc:844] Serialized 1298 byte log entry
0709 13:18:53.170203 (+945565us) op_driver.cc:464] REPLICATION: finished
0709 13:18:53.589503 (+419300us) write_op.cc:301] APPLY: starting
0709 13:18:53.589602 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:53.589744 (+   142us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:53.589883 (+   139us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:53.590450 (+   567us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:53.590638 (+   188us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:18:53.590687 (+    49us) write_op.cc:312] APPLY: finished
0709 13:18:53.590916 (+   229us) log.cc:844] Serialized 35 byte log entry
0709 13:18:53.591103 (+   187us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:53.591202 (+    99us) write_op.cc:454] Released schema lock
0709 13:18:53.591253 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:53.591599 (+   346us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:53.591630 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":411777,"apply.run_cpu_time_us":2092,"apply.run_wall_time_us":2092,"num_ops":3,"prepare.queue_time_us":566412,"prepare.run_cpu_time_us":36920,"prepare.run_wall_time_us":792959,"raft.queue_time_us":328670,"raft.run_cpu_time_us":28422,"raft.run_wall_time_us":305109,"replication_time_us":1021104,"thread_start_us":1460614,"threads_started":4,"wal-append.queue_time_us":545415,"wal-append.run_cpu_time_us":13095,"wal-append.run_wall_time_us":1822866}]]}
W20260709 13:18:55.320462 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.333387 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:55.344030 17301 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:55.360968 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 57012.3us
I20260709 13:18:55.413335 16205 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:55.248670 17435 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:55.332566 17290 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:55.349730 15598 raft_consensus.cc:1275] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:55.365654 15595 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:18:55.370100 15601 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:55.354511 17390 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:55.312111 17443 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:55.374267 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:55.376603 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1490) took 1523 ms. Trace:
I20260709 13:18:55.369208 17329 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:55.364405 17459 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:55.378397 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:55.344202 17369 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:55.375711 17428 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:55.354815 17376 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:55.392203 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 87957.5us
I20260709 13:18:55.318921 17316 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:55.397791 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 269356us
W20260709 13:18:55.408509 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.410782 16203 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:55.391814 17391 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:55.390275 17295 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:55.417577 15493 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:55.432190 15519 rpcz_store.cc:276] 0709 13:18:53.853050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.874415 (+ 21365us) service_pool.cc:224] Handling call
0709 13:18:53.886831 (+ 12416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:53.887849 (+  1018us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:53.887883 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:53.893786 (+  5903us) log.cc:844] Serialized 26 byte log entry
0709 13:18:54.591608 (+697822us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:55.286182 (+694574us) log.cc:844] Serialized 14 byte log entry
0709 13:18:55.286426 (+   244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.286497 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:55.376390 (+ 89893us) raft_consensus.cc:1685] finished
0709 13:18:55.376412 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:55.376542 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2201,"prepare.queue_time_us":719930,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":27653,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":187,"thread_start_us":1265058,"threads_started":2,"wal-append.queue_time_us":1109752}
I20260709 13:18:55.420936 17220 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:55.449265 15596 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:55.416322 17208 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.455152 16205 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:55.425482 17423 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:55.457264 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:55.472502 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:55.419736 17448 log.cc:927] Time spent T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
W20260709 13:18:55.460031 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 49330.7us
I20260709 13:18:55.460511 17428 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:55.399428 17327 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:55.481469 17472 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:18:55.482795 17455 log.cc:927] Time spent T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:18:55.483179 17327 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:55.466905 17443 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:55.479513 17295 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:55.484304 17463 log.cc:927] Time spent T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:18:55.484138 17443 consensus_queue.cc:1059] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:18:55.485663 17460 log.cc:927] Time spent T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:18:55.486140 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 375) took 1576 ms. Trace:
I20260709 13:18:55.486970 17472 raft_consensus.cc:697] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:55.487340 17471 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:55.488113 17361 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:55.487907 17472 consensus_queue.cc:237] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:18:55.448963 17459 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.489512 16203 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:55.470222 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:55.490761 16203 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:55.490964 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:55.419761 17336 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:18:55.488051 17471 raft_consensus.cc:3055] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:55.463639 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:55.486521 15527 rpcz_store.cc:276] 0709 13:18:53.909383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.254122 (+344739us) service_pool.cc:224] Handling call
0709 13:18:54.254448 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:54.254797 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:54.254827 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:54.277185 (+ 22358us) log.cc:844] Serialized 26 byte log entry
0709 13:18:55.274940 (+997755us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:55.274977 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.275041 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:55.485953 (+210912us) raft_consensus.cc:1685] finished
0709 13:18:55.485970 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:55.486087 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":19003,"thread_start_us":939676,"threads_started":1,"wal-append.queue_time_us":1079164}
I20260709 13:18:55.448350 17464 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.419377 17375 raft_consensus.cc:2804] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:55.493796 15595 raft_consensus.cc:2410] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:18:55.457438 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:55.519791 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.426000 17445 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:18:55.426215 17461 consensus_queue.cc:1048] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
W20260709 13:18:55.551235 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 55192us
I20260709 13:18:55.431577 17430 raft_consensus.cc:697] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:55.425259 17290 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:55.489691 17429 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:55.479127 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:55.491672 17307 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:55.556907 15675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 372) took 1489 ms. Trace:
W20260709 13:18:55.557454 15522 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1474) took 3467 ms (client timeout 1844 ms). Trace:
I20260709 13:18:55.457020 17308 consensus_queue.cc:1059] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.093s
I20260709 13:18:55.492532 17220 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:55.486763 17428 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.602231 15642 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:55.489964 17208 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:55.457885 17329 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:55.550508 17326 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:18:55.606127 15522 rpcz_store.cc:269] 0709 13:18:52.094194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.134090 (+ 39896us) service_pool.cc:224] Handling call
0709 13:18:55.556133 (+3422043us) spinlock_profiling.cc:243] Waited 2.66 s on lock 0x7b5400261454. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:55.557281 (+  1148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6121571712}
I20260709 13:18:55.491930 17459 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:55.606273 15675 rpcz_store.cc:276] 0709 13:18:54.067108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.067716 (+   608us) service_pool.cc:224] Handling call
0709 13:18:54.068015 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:54.518159 (+450144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.556658 (+1038499us) spinlock_profiling.cc:243] Waited 448 ms on lock 0x7b54001a7f54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:18:55.556845 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1030355840}
W20260709 13:18:55.607836 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 148373us
W20260709 13:18:55.558475 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.494905 17471 raft_consensus.cc:740] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:55.485574 17295 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:55.611095 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:55.551682 17375 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.495203 15599 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:55.614424 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:55.551008 17361 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:55.605942 17212 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.600574 17385 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.615048 15675 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:55.605357 17429 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:55.618189 15672 raft_consensus.cc:2410] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:18:55.606822 17426 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.615829 15491 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:18:55.617069 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 199568us
I20260709 13:18:55.620021 16235 raft_consensus.cc:1275] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:55.611799 17365 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:55.608974 15525 raft_consensus.cc:1275] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:18:55.611919 17428 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:55.552214 17354 consensus_queue.cc:1059] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
W20260709 13:18:55.624989 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 74665.9us
I20260709 13:18:55.609323 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:55.610723 17441 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.611522 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 376) took 1688 ms. Trace:
I20260709 13:18:55.491863 17423 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:55.556427 17296 consensus_queue.cc:1059] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:18:55.619518 15673 raft_consensus.cc:2393] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 2: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:18:55.658166 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:55.556069 17464 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:55.617527 17361 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.624943 17375 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:55.647361 15493 connection.cc:256] RPC call timeout handler was delayed by -1.7599675470000875s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20299999999999999s for a response
I20260709 13:18:55.629597 17429 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:55.651898 15675 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:55.633548 15671 raft_consensus.cc:1240] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:55.657291 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:55.636466 15567 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:55.674013 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:18:55.669629 15493 connection.cc:256] RPC call timeout handler was delayed by -1.7446675470000628s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20470000000000002s for a response
I20260709 13:18:55.620690 17295 consensus_queue.cc:1048] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:18:55.675127 16203 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:18:55.675402 15493 connection.cc:256] RPC call timeout handler was delayed by -1.7386005619998741s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1779s for a response
W20260709 13:18:55.677510 15493 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.68:39627 timed out after 1.518s (SENT) [suppressed 42 similar messages]
I20260709 13:18:55.628042 17212 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:55.634907 17385 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:55.647830 17365 raft_consensus.cc:697] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:55.798015 17479 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:55.798877 17308 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:55.824306 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 149235us
W20260709 13:18:55.679562 15493 connection.cc:256] RPC call timeout handler was delayed by -1.1665088370000376s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21520000000000003s for a response
I20260709 13:18:55.676187 16204 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:18:55.816193 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 179823us
W20260709 13:18:55.648217 15672 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57710 (request call id 370) took 1581 ms (client timeout 1689 ms). Trace:
I20260709 13:18:55.641672 17426 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:55.814102 17468 log.cc:927] Time spent T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:18:55.801240 17208 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:55.814535 17409 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:18:55.837304 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 161186us
I20260709 13:18:55.816814 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:55.817698 15596 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:18:55.642299 17233 consensus_queue.cc:1048] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:18:55.820005 17476 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:55.668051 17300 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:18:55.674461 17287 consensus_meta.cc:287] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.951s	user 0.010s	sys 0.001s
I20260709 13:18:55.667656 17361 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:55.667513 17310 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:55.678534 17369 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:55.827240 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:55.829587 15520 raft_consensus.cc:1240] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 2. Current term is 3. Ops: [2.3-2.3]
I20260709 13:18:55.860643 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:55.861390 15520 raft_consensus.cc:2393] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 3: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:18:55.816555 15668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57710 (request call id 374) took 1605 ms. Trace:
I20260709 13:18:55.680101 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:55.840061 15675 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:18:55.866303 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:55.678653 17474 consensus_queue.cc:1059] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.186s
I20260709 13:18:55.668716 17471 consensus_queue.cc:260] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:55.840934 17220 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:18:55.841079 17470 log.cc:927] Time spent T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 13:18:55.842181 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:55.834883 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:55.843595 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:55.845682 15519 raft_consensus.cc:1240] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:55.847514 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:55.877044 15526 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:55.850384 17477 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:55.862790 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:55.880313 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:55.828073 17367 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.838593 17376 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:55.866878 15669 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:55.867420 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:55.830462 17308 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:55.875661 15493 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:55.671329 17441 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:55.895251 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:55.896025 17441 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:55.850721 17479 consensus_queue.cc:1059] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:18:55.897311 17479 raft_consensus.cc:3055] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:55.897683 17479 raft_consensus.cc:740] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:18:55.830096 15672 rpcz_store.cc:269] 0709 13:18:54.066544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.070536 (+  3992us) service_pool.cc:224] Handling call
0709 13:18:55.458907 (+1388371us) spinlock_profiling.cc:243] Waited 3.05 ms on lock 0x7b5800012398. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99d10a0 00007f5ce99d0984 00007f5ce2f4b6d4 00007f5ce2f4b662
0709 13:18:55.645100 (+186193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7018368}
I20260709 13:18:55.886056 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:55.901819 15675 raft_consensus.cc:2393] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:18:55.662914 17430 consensus_queue.cc:237] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:55.887552 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:55.848641 17409 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:55.863960 17310 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:55.909446 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:55.903092 15493 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:55.904450 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:55.872781 16232 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:55.902541 15520 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:55.919399 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:55.920024 15520 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:18:55.877317 17220 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:55.916265 15675 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:55.875761 17365 consensus_queue.cc:237] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:55.882094 17329 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:55.925980 15526 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:55.874303 17465 consensus_queue.cc:1059] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:18:55.926883 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:55.927908 15491 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:55.896608 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:55.929852 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:55.932055 15524 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:18:55.911810 17475 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:55.865384 15668 rpcz_store.cc:276] 0709 13:18:54.074432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.077099 (+  2667us) service_pool.cc:224] Handling call
0709 13:18:55.600064 (+1522965us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b5800012398. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99d10a0 00007f5ce99d0984 00007f5ce2f4b6d4 00007f5ce2f4b662
0709 13:18:55.680196 (+ 80132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27946240}
I20260709 13:18:55.916723 15493 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:55.937168 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:55.937175 15526 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:55.908145 17477 raft_consensus.cc:697] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:18:55.941111 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.946916 17426 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:55.928638 17220 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:55.928490 17474 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:55.948405 15491 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:55.956849 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:55.957464 15491 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:55.899186 17367 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:55.953987 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:55.964150 15519 raft_consensus.cc:1240] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:55.947799 15673 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 1.
I20260709 13:18:55.965512 15519 raft_consensus.cc:1240] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:55.898592 17479 consensus_queue.cc:260] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:55.925412 17301 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:55.952474 15524 raft_consensus.cc:1240] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:55.960117 17430 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:18:55.973384 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:55.972820 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 44245.8us
I20260709 13:18:55.976020 15520 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:18:55.984577 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:55.984999 15526 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:18:55.967648 17212 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:55.992991 15522 raft_consensus.cc:1240] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: []
W20260709 13:18:55.992578 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.015923 16203 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:56.013317 15526 raft_consensus.cc:1240] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:55.646821 15521 rpcz_store.cc:276] 0709 13:18:53.922228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.278786 (+356558us) service_pool.cc:224] Handling call
0709 13:18:54.422474 (+143688us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:54.483922 (+ 61448us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:55.269710 (+785788us) log.cc:844] Serialized 14 byte log entry
0709 13:18:55.554569 (+284859us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:55.554632 (+    63us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:55.554681 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.596183 (+ 41502us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:55.610165 (+ 13982us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5398,"prepare.queue_time_us":761880,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":4450,"raft.queue_time_us":62928,"spinlock_wait_cycles":13977984,"thread_start_us":892216,"threads_started":3,"wal-append.queue_time_us":206064}
I20260709 13:18:55.988010 15519 raft_consensus.cc:1240] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 1. Current term is 2. Ops: []
W20260709 13:18:55.992271 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:55.978497 17467 log.cc:927] Time spent T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.289s	user 0.001s	sys 0.000s
I20260709 13:18:55.979408 17430 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.977303 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:56.026156 15526 raft_consensus.cc:1240] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:56.028542 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:56.029155 15675 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:55.985178 17376 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:56.030296 17376 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.027459 17479 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:55.982836 17310 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:55.967275 17375 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.040486 17287 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:56.041214 17357 consensus_meta.cc:287] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.956s	user 0.010s	sys 0.001s
I20260709 13:18:55.986513 17329 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.043931 15675 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:18:56.012964 17482 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:55.986796 17385 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:56.016928 17464 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:56.052650 16203 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:56.019624 17472 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:55.981884 17301 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:56.026825 17480 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:56.032124 17430 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:56.059319 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 142680us
I20260709 13:18:56.056898 17482 raft_consensus.cc:697] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:56.055590 17329 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.065800 16235 raft_consensus.cc:1240] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:56.052035 17369 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.066433 17482 consensus_queue.cc:237] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:56.071928 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.026511 17428 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:56.086879 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:56.100709 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 48156.7us
I20260709 13:18:56.071406 17475 raft_consensus.cc:697] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:56.104326 17475 consensus_queue.cc:237] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.053400 17287 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.061242 17464 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:56.114818 15675 raft_consensus.cc:1240] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:56.120536 15675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 1623) took 1184 ms. Trace:
I20260709 13:18:56.120937 15675 rpcz_store.cc:276] 0709 13:18:54.936414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.056473 (+1120059us) service_pool.cc:224] Handling call
0709 13:18:56.114644 (+ 58171us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.120264 (+  5620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.120476 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:56.053718 17357 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.061503 17480 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:56.092410 17368 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:56.086439 17474 raft_consensus.cc:697] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:56.086174 17409 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:56.137015 17480 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:56.144052 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 186654us
I20260709 13:18:56.143879 17474 consensus_queue.cc:237] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.145154 15491 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:56.113243 17376 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:56.111210 17428 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:56.109656 17367 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:18:56.134421 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 49031.1us
I20260709 13:18:56.129793 17308 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:56.086838 17369 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.150827 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:56.116391 17220 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.151288 15672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 1621) took 1215 ms. Trace:
I20260709 13:18:56.152571 15491 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:18:56.157622 17390 consensus_meta.cc:287] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.722s	user 0.006s	sys 0.003s
I20260709 13:18:56.057264 17477 consensus_queue.cc:237] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.162398 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:56.163442 17357 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:56.164088 15668 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:56.164963 16203 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:56.148625 17368 raft_consensus.cc:697] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:56.176627 15491 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:56.158390 17361 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.179123 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:56.180420 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:56.173691 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:56.184367 16204 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:56.188166 16204 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:56.192106 17295 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:18:56.192274 17428 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:56.196707 17428 raft_consensus.cc:697] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:56.197732 17428 consensus_queue.cc:237] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.198419 17287 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:56.179672 15673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 1622) took 1242 ms. Trace:
I20260709 13:18:56.203684 17490 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:56.202183 15675 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:18:56.203873 17316 consensus_meta.cc:287] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.732s	user 0.006s	sys 0.003s
I20260709 13:18:56.158164 15672 rpcz_store.cc:276] 0709 13:18:54.935998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.964374 (+1028376us) service_pool.cc:224] Handling call
0709 13:18:55.964654 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:55.964853 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:18:56.054232 (+ 89379us) log.cc:844] Serialized 14 byte log entry
0709 13:18:56.130803 (+ 76571us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:56.133614 (+  2811us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:56.134813 (+  1199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.146295 (+ 11482us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:56.149036 (+  2741us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":63719,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":10050,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":2837,"thread_start_us":101996,"threads_started":2,"wal-append.queue_time_us":96916}
I20260709 13:18:56.205650 17476 raft_consensus.cc:3055] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:56.196776 17492 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:56.188794 17464 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:56.207306 17384 consensus_queue.cc:260] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:56.208189 17486 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:56.192373 17220 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:56.200985 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:56.209805 17429 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.237660 17369 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:56.238641 17464 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.208648 17375 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:56.243228 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:18:56.263175 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:56.265425 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.239673 17384 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:56.264045 16238 raft_consensus.cc:2410] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:18:56.266074 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.192760 17497 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:56.264662 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:56.261296 16232 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:18:56.234870 17476 raft_consensus.cc:740] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:56.235288 17316 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.274622 17476 consensus_queue.cc:260] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.278141 17316 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:56.279685 15668 raft_consensus.cc:1240] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:56.279843 17486 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.281378 15603 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 4: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:56.270282 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:56.294812 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:56.296116 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:56.297125 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:56.297639 16238 raft_consensus.cc:2376] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3. Re-sending same reply.
I20260709 13:18:56.292346 15641 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:56.299041 17491 raft_consensus.cc:2804] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:56.300690 15641 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:56.303917 17357 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:56.261603 17368 consensus_queue.cc:237] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.302670 15641 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:56.273309 17497 raft_consensus.cc:697] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:56.307067 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:56.307929 15641 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:56.309307 17357 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:56.305467 17287 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:56.313162 15641 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:56.299576 17491 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:56.315742 15641 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:56.315897 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:56.299216 17489 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:56.317400 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:56.318178 17367 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:56.320041 17307 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:56.305716 17365 consensus_queue.cc:1059] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 13:18:56.323709 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:18:56.328182 15493 consensus_peers.cc:597] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 -> Peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Couldn't send request to peer dd1a14c3173d49dc9b55bb116e3783c9. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:56.329133 15493 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:56.328085 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:56.315810 17409 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:56.330830 15493 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:56.301821 17369 consensus_queue.cc:1059] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:18:56.314640 17357 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:56.320214 17376 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:56.321645 17307 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.332154 17474 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:18:56.330960 17459 consensus_meta.cc:287] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.712s	user 0.007s	sys 0.003s
I20260709 13:18:56.318326 17491 raft_consensus.cc:3060] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:56.339013 15493 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:18:56.357179 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.280670 17486 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:56.346100 17459 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.382546 17474 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:18:56.402663 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.334539 17361 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:56.335158 17490 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:18:56.426366 17503 log.cc:927] Time spent T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:18:56.340974 17357 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:56.343739 17369 raft_consensus.cc:3055] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:18:56.427929 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 88567us
I20260709 13:18:56.431473 17307 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:56.442139 16240 raft_consensus.cc:2393] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:18:56.489456 17408 consensus_queue.cc:1059] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:56.490769 17408 raft_consensus.cc:3055] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:18:56.493330 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.497984 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:56.434054 17316 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.330808 17489 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:56.434150 17376 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:56.423054 17365 consensus_queue.cc:1059] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.112s
I20260709 13:18:56.438815 15672 raft_consensus.cc:1240] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:56.511828 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:56.494971 17308 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.494261 17385 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.444953 17361 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.503842 17220 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.505235 17464 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:56.498946 17375 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:56.565302 17489 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:56.500145 17377 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.497071 17367 raft_consensus.cc:697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:56.210793 15673 rpcz_store.cc:276] 0709 13:18:54.936205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.982102 (+1045897us) service_pool.cc:224] Handling call
0709 13:18:55.982448 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:55.982662 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:56.106711 (+124049us) log.cc:844] Serialized 14 byte log entry
0709 13:18:56.148854 (+ 42143us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:56.153054 (+  4200us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:56.154276 (+  1222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.162163 (+  7887us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:56.178242 (+ 16079us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":46175,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":77,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":2691,"thread_start_us":159900,"threads_started":2,"wal-append.queue_time_us":82933}
I20260709 13:18:56.488524 17369 raft_consensus.cc:740] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:56.491766 17408 raft_consensus.cc:740] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:56.487381 17429 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:56.570288 17365 raft_consensus.cc:3055] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:56.582995 17489 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:18:56.586023 15670 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 1581) took 3710 ms (client timeout 2116 ms). Trace:
W20260709 13:18:56.586001 17502 log.cc:927] Time spent T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:18:56.588810 17377 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.559640 17490 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:18:56.637009 15676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 1588) took 3721 ms (client timeout 2152 ms). Trace:
W20260709 13:18:56.656153 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.589833 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:56.593880 15519 raft_consensus.cc:1240] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 2. Current term is 3. Ops: []
W20260709 13:18:56.663448 15670 rpcz_store.cc:269] 0709 13:18:52.875903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.876613 (+   710us) service_pool.cc:224] Handling call
0709 13:18:53.158980 (+282367us) spinlock_profiling.cc:243] Waited 8.87 ms on lock 0x7b5800012398. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99d10a0 00007f5ce99d0984 00007f5ce2f4b6d4 00007f5ce2f4b662
0709 13:18:56.585794 (+3426814us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b540009c9d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:56.585933 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3635501184}
I20260709 13:18:56.438534 17430 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.583242 17464 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:56.669469 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:18:56.634357 17504 log.cc:927] Time spent T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:18:56.674947 15670 raft_consensus.cc:2393] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 3: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:18:56.660509 17263 log.cc:927] Time spent T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:18:56.678486 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:56.560156 17316 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:56.590219 17425 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:56.684193 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:18:56.667742 15676 rpcz_store.cc:269] 0709 13:18:52.915538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.248148 (+332610us) service_pool.cc:224] Handling call
0709 13:18:56.636803 (+3388655us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b540021dc54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:56.636934 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3672859008}
W20260709 13:18:56.667996 17456 log.cc:927] Time spent T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 1.373s	user 0.004s	sys 0.003s
I20260709 13:18:56.634268 17435 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:56.635717 17423 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:56.578702 17308 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.572158 17376 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:56.658990 17365 raft_consensus.cc:740] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:56.578552 17385 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.692433 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:56.692826 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:56.695827 15519 raft_consensus.cc:2410] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:18:56.580390 17220 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.580545 17361 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.583653 17375 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.701743 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:56.572962 17497 consensus_queue.cc:237] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.681437 17429 consensus_queue.cc:1059] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:18:56.715564 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:56.712721 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:56.675064 17489 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.681394 17464 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.716279 15519 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:56.589807 17480 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.590804 17369 consensus_queue.cc:260] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:56.722754 16238 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:18:56.704391 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:56.724757 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.724556 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:56.682929 17490 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.728199 15519 raft_consensus.cc:2410] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:18:56.735721 17375 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:56.740387 17511 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.774524 16239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 776) took 2049 ms. Trace:
I20260709 13:18:56.748555 17263 tablet_bootstrap.cc:492] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:56.745664 17486 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:56.777364 16239 rpcz_store.cc:276] 0709 13:18:54.725173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.725307 (+   134us) service_pool.cc:224] Handling call
0709 13:18:54.796263 (+ 70956us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:54.917638 (+121375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.774231 (+1856593us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54005370d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:18:56.774456 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":273805568}
W20260709 13:18:56.788014 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.725910 17510 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.701299 17408 consensus_queue.cc:260] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:56.793715 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.814777 17507 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.815495 17507 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.820683 15567 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:56.705349 17367 consensus_queue.cc:237] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.820015 17377 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:56.824501 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:56.790020 17369 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:56.782301 17365 consensus_queue.cc:260] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:56.827922 16239 raft_consensus.cc:1240] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:56.838479 15519 raft_consensus.cc:2393] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 3: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:18:56.842342 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1491) took 2979 ms. Trace:
I20260709 13:18:56.817870 17507 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:56.830461 17263 tablet_bootstrap.cc:492] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:56.850708 15595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 276) took 1432 ms. Trace:
I20260709 13:18:56.842829 15523 rpcz_store.cc:276] 0709 13:18:53.853297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.890239 (+ 36942us) service_pool.cc:224] Handling call
0709 13:18:53.911548 (+ 21309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:54.602693 (+691145us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:54.602726 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:54.603218 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 13:18:55.077708 (+474490us) spinlock_profiling.cc:243] Waited 5.72 ms on lock 0x7b4800002958. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce9934aa1 00007f5ce992bb37 00007f5ce992b9ca 00007f5ce9a2326f 00007f5ce9a22f37 00007f5ce9a3755c 00007f5ce9a37508 00007f5ce9a37488 00007f5ce9a37443 00007f5ce9a366ec 00007f5ce8960a96
0709 13:18:56.429693 (+1351985us) spinlock_profiling.cc:243] Waited 4.4 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:18:56.832279 (+402586us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b1800328180. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce45b34bc 00007f5ce190148b 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab0d1
0709 13:18:56.832463 (+   184us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:56.832501 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.832593 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:56.832640 (+    47us) raft_consensus.cc:1685] finished
0709 13:18:56.832653 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:56.832796 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":46107,"raft.run_cpu_time_us":6805,"raft.run_wall_time_us":1137274,"spinlock_wait_cycles":31012864,"thread_start_us":1127719,"threads_started":2,"wal-append.queue_time_us":636953}
I20260709 13:18:56.858049 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:56.858786 15519 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:56.854509 15595 rpcz_store.cc:276] 0709 13:18:55.418340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.614223 (+195883us) service_pool.cc:224] Handling call
0709 13:18:55.614490 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:55.614700 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:55.637022 (+ 22322us) log.cc:844] Serialized 14 byte log entry
0709 13:18:56.751056 (+1114034us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b1800304c60. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab23a 00007f5ce4a15a2a
0709 13:18:56.850299 (+ 99243us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:56.850366 (+    67us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:56.850415 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.850487 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:56.850631 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":21816,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":2864,"spinlock_wait_cycles":5266816,"thread_start_us":98957,"threads_started":1}
I20260709 13:18:56.863204 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:56.824708 17512 raft_consensus.cc:2804] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:56.864159 15595 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:56.865530 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:56.866335 15595 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:56.821798 17510 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.867354 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:56.848701 17507 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.845295 17486 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:56.871377 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:56.871515 15596 raft_consensus.cc:1240] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:56.781571 17511 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:56.874284 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:56.875523 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:56.875710 17511 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:56.876083 15596 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:56.877242 15596 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:18:56.877521 15596 pending_rounds.cc:85] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting all ops after (but not including) 5
I20260709 13:18:56.877723 15596 pending_rounds.cc:107] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting uncommitted NO_OP operation due to leader change: 3.6
W20260709 13:18:56.856542 17409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.002s	sys 0.027s
I20260709 13:18:56.854657 17365 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:56.852417 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:56.883771 15596 raft_consensus.cc:2889] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:56.889192 17384 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.890700 15568 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:18:56.891613 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:56.897174 15674 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 1593) took 2288 ms (client timeout 2210 ms). Trace:
I20260709 13:18:56.893581 15603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34102 (request call id 262) took 1593 ms. Trace:
I20260709 13:18:56.906594 15603 rpcz_store.cc:276] 0709 13:18:55.299744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.301786 (+  2042us) service_pool.cc:224] Handling call
0709 13:18:56.893329 (+1591543us) spinlock_profiling.cc:243] Waited 928 ms on lock 0x7b5400232654. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:56.893505 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2133860608}
I20260709 13:18:56.902623 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:56.908172 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:56.909049 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:18:56.906041 15674 rpcz_store.cc:269] 0709 13:18:54.608894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.709716 (+100822us) service_pool.cc:224] Handling call
0709 13:18:56.896968 (+2187252us) spinlock_profiling.cc:243] Waited 88.9 ms on lock 0x7b540042a254. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:56.897090 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":204401536}
I20260709 13:18:56.905292 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:56.900698 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:56.864003 17430 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:56.871476 17361 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:56.870604 17512 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:56.915323 15595 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:56.904140 15641 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:56.916800 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:56.916805 15568 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:56.918776 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:56.921470 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:56.923238 17523 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.923914 17523 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.926709 17523 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:18:56.899320 17409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.003s	sys 0.027s
I20260709 13:18:56.927815 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:56.914592 17263 ts_tablet_manager.cc:1403] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 2.092s	user 0.026s	sys 0.015s
I20260709 13:18:56.894557 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:56.932309 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:56.934038 17515 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:56.899712 17316 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:56.901213 17513 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:56.907608 17308 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:56.894913 17507 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:56.901415 17518 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:56.927359 17409 raft_consensus.cc:697] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:56.899845 17486 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:56.940501 15594 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:56.943396 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:56.951892 16239 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:18:56.937570 15596 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:18:56.927940 17376 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:56.928618 17511 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:56.931491 17212 consensus_queue.cc:1059] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:18:56.944007 15673 raft_consensus.cc:2393] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 2: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:18:56.929417 17512 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:56.935477 17361 raft_consensus.cc:2804] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:56.950398 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:56.965965 15519 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:18:56.915951 17384 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:56.936075 17385 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:56.937966 17220 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:56.927364 17523 consensus_queue.cc:1059] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:18:56.961175 17263 raft_consensus.cc:359] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:56.967129 17361 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:56.968266 17516 log.cc:927] Time spent T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:18:56.965588 17464 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:56.970036 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:56.969971 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:56.954370 17480 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:56.989617 16239 raft_consensus.cc:2393] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 3: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:18:56.996559 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:56.997450 15519 raft_consensus.cc:2410] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:18:56.997430 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 392) took 1081 ms. Trace:
W20260709 13:18:56.998107 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.997481 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:56.984864 17263 raft_consensus.cc:740] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:56.984710 17361 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:57.012933 16236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57258 (request call id 538) took 2956 ms (client timeout 2152 ms). Trace:
W20260709 13:18:57.013278 16236 rpcz_store.cc:269] 0709 13:18:54.050978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.051365 (+   387us) service_pool.cc:224] Handling call
0709 13:18:57.007002 (+2955637us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54007b02d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:57.007183 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2390852352}
I20260709 13:18:57.020150 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:56.951129 17513 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:57.025943 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.026829 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:57.029685 15519 raft_consensus.cc:2410] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:18:57.021550 17514 log.cc:927] Time spent T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.084s	user 0.001s	sys 0.001s
W20260709 13:18:57.044096 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.034861 16239 raft_consensus.cc:1240] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:18:56.952942 17518 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:57.068012 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:57.074733 15596 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:57.002159 17409 consensus_queue.cc:237] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:57.025326 15668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49094 (request call id 1606) took 1097 ms. Trace:
W20260709 13:18:57.079982 17522 log.cc:927] Time spent T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:18:57.081328 17384 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:57.082866 17511 raft_consensus.cc:3055] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:57.083292 17511 raft_consensus.cc:740] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:57.084117 17511 consensus_queue.cc:260] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:57.085414 17511 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:57.074913 17262 log.cc:927] Time spent T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:18:57.093748 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:57.087420 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:57.089154 17524 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.089789 17486 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:57.090384 17285 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:57.092443 15524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1487) took 1262 ms. Trace:
I20260709 13:18:57.095646 15519 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:18:57.098271 15524 rpcz_store.cc:276] 0709 13:18:55.829563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.938786 (+109223us) service_pool.cc:224] Handling call
0709 13:18:55.939051 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:55.977176 (+ 38125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.092123 (+1114947us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b54002ddcd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:18:57.092369 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30399872}
I20260709 13:18:57.099269 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:57.099972 15519 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:57.100898 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:57.101495 15519 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:18:57.106832 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:57.106971 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:57.107365 15596 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:57.109098 15519 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:57.079041 15668 rpcz_store.cc:276] 0709 13:18:55.927797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.309789 (+381992us) service_pool.cc:224] Handling call
0709 13:18:56.310129 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:56.310353 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:56.310378 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:56.310899 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 13:18:56.688285 (+377386us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:56.688683 (+   398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.690745 (+  2062us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:57.021990 (+331245us) raft_consensus.cc:1685] finished
0709 13:18:57.023142 (+  1152us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.025189 (+  2047us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":376960,"threads_started":1,"wal-append.queue_time_us":475591}
I20260709 13:18:57.110265 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:57.114364 15519 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:57.116357 17290 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:57.116991 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:57.028928 17263 consensus_queue.cc:260] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:57.118273 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:57.119777 15524 raft_consensus.cc:2393] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 3: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:18:57.118309 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:57.103742 17507 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:57.107987 17524 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:57.130760 17471 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:57.131158 17262 tablet_bootstrap.cc:492] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:57.135880 17524 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:57.141537 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:57.142504 16239 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:18:57.133967 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:57.142972 15491 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:57.127405 17513 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:57.137778 17287 raft_consensus.cc:3055] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:57.149139 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:57.146443 17486 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:57.151376 15674 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:57.152267 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:57.152545 15674 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:57.152922 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:57.153689 15523 raft_consensus.cc:2393] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:18:57.154600 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:18:57.132624 17491 consensus_meta.cc:287] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.705s	user 0.007s	sys 0.004s
I20260709 13:18:57.155544 16239 raft_consensus.cc:2393] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 3: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:18:57.155755 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:57.156292 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:57.148516 17521 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:18:57.160473 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:57.160792 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:57.161415 15668 raft_consensus.cc:2410] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:18:57.162066 17262 tablet_bootstrap.cc:492] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:57.162539 15668 raft_consensus.cc:3055] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:57.162937 15668 raft_consensus.cc:740] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:57.165155 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:57.161515 16237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57258 (request call id 540) took 1506 ms. Trace:
I20260709 13:18:57.154312 17287 raft_consensus.cc:740] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:57.166273 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:57.148797 17507 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.170142 17287 consensus_queue.cc:260] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:57.170852 17507 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:57.162874 17262 ts_tablet_manager.cc:1403] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 2.346s	user 0.038s	sys 0.040s
I20260709 13:18:57.169386 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:57.169219 17361 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:57.170207 16239 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
W20260709 13:18:57.161538 17357 consensus_meta.cc:287] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.674s	user 0.007s	sys 0.004s
I20260709 13:18:57.171289 17287 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:57.165723 16237 rpcz_store.cc:276] 0709 13:18:55.654957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.656996 (+  2039us) service_pool.cc:224] Handling call
0709 13:18:57.160653 (+1503657us) spinlock_profiling.cc:243] Waited 3.86 ms on lock 0x7b580009ff98. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99d10a0 00007f5ce99d0984 00007f5ce2f4b6d4 00007f5ce2f4b662
0709 13:18:57.161457 (+   804us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8869248}
I20260709 13:18:57.163872 15594 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:57.259898 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:57.260762 15524 raft_consensus.cc:2368] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5.
W20260709 13:18:57.262887 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:57.272820 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.176381 17507 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:57.331970 15674 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:57.333160 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:57.175587 15673 raft_consensus.cc:2393] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 3: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:18:57.175701 16231 raft_consensus.cc:1240] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:57.175884 17445 raft_consensus.cc:3055] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:57.148933 17365 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.200323 16239 raft_consensus.cc:1275] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:18:57.298477 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.313938 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:57.174750 15596 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:18:57.259471 17357 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:57.326516 17361 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:57.326593 16237 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:57.327309 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:57.157708 17491 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:57.344486 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:57.345464 17491 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:57.163805 15668 consensus_queue.cc:260] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:57.346359 15668 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:57.346361 16231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57258 (request call id 531) took 3453 ms. Trace:
I20260709 13:18:57.346894 16231 rpcz_store.cc:276] 0709 13:18:53.892786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.896888 (+  4102us) service_pool.cc:224] Handling call
0709 13:18:55.287506 (+1390618us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b580009ff98. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99e4fc0 00007f5ce99cf929 00007f5ce99cf3d2 00007f5ce2f46bd4
0709 13:18:55.287736 (+   230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.175529 (+1887793us) spinlock_profiling.cc:243] Waited 60.5 ms on lock 0x7b54006b06a0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4982dc0 00007f5ce4a0d1c0 00007f5ce4a0e231 00007f5ce4a0ad8f 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c
0709 13:18:57.341208 (+165679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.346297 (+  5089us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165731456}
W20260709 13:18:57.164410 15599 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34102 (request call id 266) took 1863 ms (client timeout 1623 ms). Trace:
I20260709 13:18:57.283622 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:57.345737 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:18:57.350220 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 66709.3us
I20260709 13:18:57.350448 15524 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:57.342767 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 178153us
W20260709 13:18:57.357949 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:57.359041 15493 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.333911 15674 raft_consensus.cc:2393] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 3: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:18:57.335315 17538 log.cc:927] Time spent T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
W20260709 13:18:57.336725 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 104086us
I20260709 13:18:57.337574 17445 raft_consensus.cc:740] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:57.341462 17355 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:57.402189 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:57.446607 16235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57258 (request call id 539) took 1792 ms. Trace:
I20260709 13:18:57.373594 17521 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:57.447707 16204 leader_election.cc:400] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5.
I20260709 13:18:57.451438 16204 leader_election.cc:403] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:57.453807 15524 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:18:57.454924 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:57.455670 15524 raft_consensus.cc:2393] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 3: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:18:57.456043 15524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1499) took 1311 ms. Trace:
I20260709 13:18:57.448918 15642 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:57.447113 16235 rpcz_store.cc:276] 0709 13:18:55.654647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.656532 (+  1885us) service_pool.cc:224] Handling call
0709 13:18:55.656831 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.066311 (+409480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.446348 (+1380037us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b54003c87d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:18:57.446540 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":930895104}
W20260709 13:18:57.456565 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.457530 15642 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:57.458732 15642 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:57.456359 15524 rpcz_store.cc:276] 0709 13:18:56.145074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.454644 (+1309570us) service_pool.cc:224] Handling call
0709 13:18:57.455992 (+  1348us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:57.352543 17512 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:57.403501 17491 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.460398 15642 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:57.342949 17365 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:57.465194 15642 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:57.446621 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:57.470206 17527 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:57.472955 17521 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:57.402485 17507 consensus_queue.cc:1059] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.080s
W20260709 13:18:57.373522 15599 rpcz_store.cc:269] 0709 13:18:55.300788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.302795 (+  2007us) service_pool.cc:224] Handling call
0709 13:18:57.164217 (+1861422us) spinlock_profiling.cc:243] Waited 77.1 ms on lock 0x7b5400389ad4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:57.164343 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177227008}
W20260709 13:18:57.464686 17369 consensus_meta.cc:287] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.623s	user 0.005s	sys 0.005s
I20260709 13:18:57.471501 17263 ts_tablet_manager.cc:1434] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.522s	user 0.007s	sys 0.003s
I20260709 13:18:57.463167 17523 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:57.461174 17361 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:57.463460 17525 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:57.480166 17521 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:57.484454 15673 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:57.476938 17491 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:57.487888 17523 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:18:57.484223 17263 ts_tablet_manager.cc:1467] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9: Tablet startup took 549ms
I20260709 13:18:57.491796 17369 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:57.490705 17525 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:57.494762 17361 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.494305 17521 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:57.484006 17445 consensus_queue.cc:260] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:57.509290 17523 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:57.511787 17525 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:57.518091 17512 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:57.522255 17531 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.522944 17491 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:57.522862 17531 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:57.466861 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:57.554071 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:57.555476 15524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1509) took 1269 ms. Trace:
I20260709 13:18:57.555917 15524 rpcz_store.cc:276] 0709 13:18:56.286436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.553167 (+1266731us) service_pool.cc:224] Handling call
0709 13:18:57.555403 (+  2236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:57.557456 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:57.559253 15524 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:18:57.559567 15527 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:57.561460 15594 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:57.562772 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:18:57.562988 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 97697.2us
I20260709 13:18:57.566458 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:57.578732 17212 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.590860 17445 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:57.584789 17326 raft_consensus.cc:3055] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:57.585963 17327 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:57.600010 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.524813 17525 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:57.599153 17329 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:57.602563 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:57.603015 17525 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:57.604169 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:57.590418 17357 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:57.591020 17365 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:18:57.508507 17263 ts_tablet_manager.cc:1469] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9: Trace:
0709 13:18:48.342123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.342926 (+   803us) service_pool.cc:224] Handling call
0709 13:18:48.544295 (+201369us) ts_tablet_manager.cc:934] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
0709 13:18:52.046112 (+3501817us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:18:52.618694 (+572582us) tablet_metadata.cc:665] Metadata flushed
0709 13:18:52.622767 (+  4073us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:18:52.941641 (+318874us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:18:52.958924 (+ 17283us) tablet_replica.cc:206] Creating consensus instance
0709 13:18:53.005422 (+ 46498us) ts_tablet_manager.cc:989] Replica registered.
0709 13:18:53.011185 (+  5763us) inbound_call.cc:177] Queueing success response
0709 13:18:54.822000 (+1810815us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:18:56.944962 (+2122962us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:18:56.956994 (+ 12032us) tablet_replica.cc:261] Starting instrumentation
0709 13:18:56.959075 (+  2081us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":15425,"tablet-copy.queue_time_us":117250,"tablet-download-pool.queue_time_us":5191,"tablet-download-pool.run_cpu_time_us":1268,"tablet-download-pool.run_wall_time_us":293737,"thread_start_us":257328,"threads_started":2}
I20260709 13:18:57.597728 17212 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:18:57.514153 17361 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:57.610517 17327 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:57.630465 15675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57710 (request call id 397) took 1508 ms (client timeout 1527 ms). Trace:
W20260709 13:18:57.630918 15675 rpcz_store.cc:269] 0709 13:18:56.121628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.135887 (+ 14259us) service_pool.cc:224] Handling call
0709 13:18:57.630035 (+1494148us) spinlock_profiling.cc:243] Waited 46.9 ms on lock 0x7b54003fb454. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:57.630246 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":107810048}
I20260709 13:18:57.632261 17327 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:57.611526 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1481) took 1773 ms. Trace:
I20260709 13:18:57.599488 17326 raft_consensus.cc:740] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:57.618261 15524 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:57.594556 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:57.623322 17471 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:18:57.627238 17329 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:57.629174 17212 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:18:57.596004 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:18:57.634385 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.632980 15522 rpcz_store.cc:276] 0709 13:18:55.826126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.826247 (+   121us) service_pool.cc:224] Handling call
0709 13:18:55.826534 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.013103 (+186569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.598641 (+1585538us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b540029bdd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:18:57.599078 (+   437us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":382406016}
I20260709 13:18:57.634446 15676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 398) took 1434 ms. Trace:
I20260709 13:18:57.636080 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:57.638514 15676 rpcz_store.cc:276] 0709 13:18:56.200003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.685427 (+485424us) service_pool.cc:224] Handling call
0709 13:18:56.685803 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:56.715038 (+ 29235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:56.715068 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:56.719705 (+  4637us) log.cc:844] Serialized 26 byte log entry
0709 13:18:56.839127 (+119422us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:56.839167 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.632963 (+793796us) spinlock_profiling.cc:243] Waited 28.8 ms on lock 0x7b540046d554. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:18:57.633033 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:57.633108 (+    75us) raft_consensus.cc:1685] finished
0709 13:18:57.633120 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.634200 (+  1080us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99507,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":126,"spinlock_wait_cycles":66327296,"thread_start_us":104851,"threads_started":1,"wal-append.queue_time_us":175361}
I20260709 13:18:57.641078 17361 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:57.642336 17326 consensus_queue.cc:260] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:57.644790 17326 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:57.646664 15677 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:57.648619 15687 tablet_copy_source_session.cc:215] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:57.649538 16240 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 367) took 1483 ms. Trace:
I20260709 13:18:57.650044 16240 rpcz_store.cc:276] 0709 13:18:56.165824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.165992 (+   168us) service_pool.cc:224] Handling call
0709 13:18:57.649082 (+1483090us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b5400640ad4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:57.649350 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":580547456}
I20260709 13:18:57.650467 15687 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:57710 (request call id 371) took 3583 ms. Trace:
W20260709 13:18:57.655563 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.645048 16232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 366) took 1493 ms. Trace:
I20260709 13:18:57.659087 17357 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:57.657510 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 248581us
I20260709 13:18:57.599170 17521 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:57.659289 15594 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:57.670560 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:57.672166 17221 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:18:57.663383 15599 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:57.672308 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:57.678870 17262 raft_consensus.cc:359] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:57.668069 17227 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:57.691527 17262 raft_consensus.cc:740] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:57.661187 16232 rpcz_store.cc:276] 0709 13:18:56.151036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.152049 (+  1013us) service_pool.cc:224] Handling call
0709 13:18:57.644402 (+1492353us) spinlock_profiling.cc:243] Waited 52.9 ms on lock 0x7b54006ea0d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:57.644699 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":121704704}
I20260709 13:18:57.660611 15687 rpcz_store.cc:276] 0709 13:18:54.066833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.077141 (+ 10308us) service_pool.cc:224] Handling call
0709 13:18:56.588029 (+2510888us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b540046d054. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a153f7 00007f5ce99ab036 00007f5ce99b4a3e 00007f5ce127827b 00007f5ce99b49ba 00007f5ce99afacc 00007f5ce99aac7b 00007f5ce999f8d1 00007f5ce8a534b4 00007f5ce8a53442
0709 13:18:57.641642 (+1053613us) spinlock_profiling.cc:243] Waited 899 ms on lock 0x7b540046d054. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4975090 00007f5ce4a13f3a 00007f5ce99ab1bb 00007f5ce99b4a3e 00007f5ce127827b 00007f5ce99b49ba 00007f5ce99afacc 00007f5ce99aac7b 00007f5ce999f8d1 00007f5ce8a534b4 00007f5ce8a53442
0709 13:18:57.650326 (+  8684us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2985669248}
I20260709 13:18:57.682219 17329 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:57.686093 17357 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:57.693379 17262 consensus_queue.cc:260] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:57.694190 17429 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:57.690353 16204 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:57.701740 15594 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:57.702689 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:57.704119 17459 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:57.708552 15524 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:18:57.712425 17518 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:57.712805 16234 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
W20260709 13:18:57.348410 17307 consensus_queue.cc:1175] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" responder_term: 2 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:18:57.720640 17490 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:57.723214 17221 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5.
W20260709 13:18:57.723937 17462 log.cc:927] Time spent T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 2.349s	user 0.001s	sys 0.005s
I20260709 13:18:57.723824 17523 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:57.742955 17227 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:57.751255 17375 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.751842 17375 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:57.753047 17429 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:57.757505 17357 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:18:57.758024 15493 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.725452 17521 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:57.746058 17262 ts_tablet_manager.cc:1434] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.574s	user 0.009s	sys 0.002s
I20260709 13:18:57.743060 15602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 268) took 2441 ms. Trace:
W20260709 13:18:57.744032 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 55545us
W20260709 13:18:57.760699 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 57238.7us
W20260709 13:18:57.737900 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.750135 15677 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:18:57.761613 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 394) took 1842 ms. Trace:
I20260709 13:18:57.761579 15602 rpcz_store.cc:276] 0709 13:18:55.301319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.301425 (+   106us) service_pool.cc:224] Handling call
0709 13:18:55.323535 (+ 22110us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:55.323895 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:55.323923 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:57.611470 (+2287547us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b2c000fb608. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce3304a11 00007f5ce3314372 00007f5ce4a46a01 00007f5ce4a0b43a 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2
0709 13:18:57.612364 (+   894us) log.cc:844] Serialized 26 byte log entry
0709 13:18:57.636975 (+ 24611us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:57.638242 (+  1267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.660045 (+ 21803us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:57.730934 (+ 70889us) raft_consensus.cc:1685] finished
0709 13:18:57.730955 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.742832 (+ 11877us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3447,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":3514,"spinlock_wait_cycles":4567296,"thread_start_us":23879,"threads_started":1,"wal-append.queue_time_us":76796}
I20260709 13:18:57.766412 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:57.764833 15523 rpcz_store.cc:276] 0709 13:18:55.916872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.660369 (+1743497us) service_pool.cc:224] Handling call
0709 13:18:57.670020 (+  9651us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.670535 (+   515us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:57.670993 (+   458us) log.cc:844] Serialized 14 byte log entry
0709 13:18:57.747364 (+ 76371us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:57.747422 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:57.758542 (+ 11120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.758620 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.758776 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":273,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":7886,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":8918,"thread_start_us":69033,"threads_started":1}
I20260709 13:18:57.767764 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:57.768147 15523 raft_consensus.cc:1240] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:18:57.768960 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 396) took 1851 ms. Trace:
I20260709 13:18:57.769519 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:57.770728 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:57.748296 17526 raft_consensus.cc:3055] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:57.774142 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:57.774255 15594 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:57.765276 15599 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 4: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:57.762743 17445 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:18:57.764581 15677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57710 (request call id 390) took 2354 ms (client timeout 1746 ms). Trace:
I20260709 13:18:57.771373 17357 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:57.762130 17262 ts_tablet_manager.cc:1467] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: Tablet startup took 589ms
I20260709 13:18:57.777062 16204 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:18:57.777828 15677 rpcz_store.cc:269] 0709 13:18:55.407235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.607913 (+200678us) service_pool.cc:224] Handling call
0709 13:18:57.634877 (+2026964us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b5c000c8d0c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce8375161 00007f5ce8516a79 00007f5ce99d09b6 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:57.761510 (+126633us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2681088}
I20260709 13:18:57.780285 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:57.769320 15523 rpcz_store.cc:276] 0709 13:18:55.917848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.767271 (+1849423us) service_pool.cc:224] Handling call
0709 13:18:57.767785 (+   514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:57.768650 (+   865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.768883 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:57.775173 17526 raft_consensus.cc:740] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:57.784118 15642 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:57.784828 15671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 1617) took 2849 ms. Trace:
I20260709 13:18:57.786747 15669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 1618) took 2851 ms. Trace:
I20260709 13:18:57.787113 15669 rpcz_store.cc:276] 0709 13:18:54.935356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.045109 (+109753us) service_pool.cc:224] Handling call
0709 13:18:55.093886 (+ 48777us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:55.884973 (+791087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.786222 (+1901249us) spinlock_profiling.cc:243] Waited 773 ms on lock 0x7b54001a8954. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:18:57.786564 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1776805760}
I20260709 13:18:57.788791 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:57.791407 17489 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:57.794090 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:18:57.779062 17262 ts_tablet_manager.cc:1469] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: Trace:
0709 13:18:48.313483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.315768 (+  2285us) service_pool.cc:224] Handling call
0709 13:18:48.362961 (+ 47193us) ts_tablet_manager.cc:934] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
0709 13:18:51.497568 (+3134607us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:18:52.212955 (+715387us) tablet_metadata.cc:665] Metadata flushed
0709 13:18:52.217278 (+  4323us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:18:52.934565 (+717287us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:18:52.934869 (+   304us) tablet_replica.cc:206] Creating consensus instance
0709 13:18:52.935191 (+   322us) ts_tablet_manager.cc:989] Replica registered.
0709 13:18:52.935248 (+    57us) inbound_call.cc:177] Queueing success response
0709 13:18:54.816272 (+1881024us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:18:57.171769 (+2355497us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:18:57.474975 (+303206us) tablet_replica.cc:261] Starting instrumentation
0709 13:18:57.475522 (+   547us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":31131,"mutex_wait_us":197512,"raft.queue_time_us":4459,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":1100,"tablet-copy.queue_time_us":41461,"tablet-download-pool.queue_time_us":419,"tablet-download-pool.run_cpu_time_us":6079,"tablet-download-pool.run_wall_time_us":1136374,"thread_start_us":110999,"threads_started":2}
I20260709 13:18:57.796877 17526 consensus_queue.cc:260] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:57.799100 17526 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:57.802899 17479 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.803843 17479 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:57.804852 16205 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:57.806165 17326 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:57.806759 17219 raft_consensus.cc:3055] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:57.807232 17219 raft_consensus.cc:740] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:57.807157 17426 consensus_queue.cc:1059] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:57.808137 17326 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:57.808657 17426 raft_consensus.cc:3055] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:57.808542 17219 consensus_queue.cc:260] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:57.806779 17479 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:57.812911 15670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 393) took 1793 ms. Trace:
I20260709 13:18:57.814456 16233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 760) took 5048 ms. Trace:
I20260709 13:18:57.813331 15670 rpcz_store.cc:276] 0709 13:18:56.019055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.628085 (+1609030us) service_pool.cc:224] Handling call
0709 13:18:57.628863 (+   778us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:57.629603 (+   740us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:57.629641 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:57.646976 (+ 17335us) log.cc:844] Serialized 26 byte log entry
0709 13:18:57.664937 (+ 17961us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:57.664974 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.665043 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:57.812601 (+147558us) raft_consensus.cc:1685] finished
0709 13:18:57.812618 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.812734 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5802,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":12175,"thread_start_us":17496,"threads_started":1,"wal-append.queue_time_us":80970}
I20260709 13:18:57.809142 17426 raft_consensus.cc:740] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:18:57.819939 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 55988.9us
I20260709 13:18:57.821990 17506 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:57.822499 17506 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:57.822816 17506 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:57.825314 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:57.826144 15602 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:18:57.827414 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:57.828409 15602 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:57.829466 17287 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:57.837219 17287 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:57.785337 15671 rpcz_store.cc:276] 0709 13:18:54.934831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.951027 (+ 16196us) service_pool.cc:224] Handling call
0709 13:18:55.043893 (+ 92866us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:55.679386 (+635493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.784279 (+2104893us) spinlock_profiling.cc:243] Waited 571 ms on lock 0x7b540026c854. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:18:57.784628 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1314413824}
I20260709 13:18:57.825680 15669 raft_consensus.cc:1240] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:57.838050 15568 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:57.830211 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:57.839499 15676 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:18:57.839514 17479 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:57.840142 17479 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:57.817395 17219 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:57.841225 17343 consensus_queue.cc:1059] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:57.842131 17287 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:57.842309 17343 raft_consensus.cc:3055] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:57.842733 17343 raft_consensus.cc:740] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:57.829493 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:57.840576 17479 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:57.846716 17221 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:57.847524 17221 raft_consensus.cc:697] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:57.848456 15601 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:18:57.848819 17221 consensus_queue.cc:237] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:57.843829 17343 consensus_queue.cc:260] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:57.817721 17521 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:18:57.851449 15525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1461) took 5767 ms. Trace:
I20260709 13:18:57.851626 17343 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:57.843911 15602 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:57.854130 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:57.858927 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:57.859824 15599 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:18:57.859930 17476 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:57.862643 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 395) took 1939 ms. Trace:
I20260709 13:18:57.869757 15522 rpcz_store.cc:276] 0709 13:18:55.917571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.801195 (+1883624us) service_pool.cc:224] Handling call
0709 13:18:57.801688 (+   493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.824786 (+ 23098us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:57.824819 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:57.856727 (+ 31908us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:57.856770 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.856832 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.856952 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:57.870311 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:57.872422 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:57.872608 15670 raft_consensus.cc:1240] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:57.876024 17227 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:57.878465 17375 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:57.878921 17472 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.879261 15602 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:57.879575 17472 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:57.880501 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:57.881071 15528 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:57.881217 15602 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:57.851976 15525 rpcz_store.cc:276] 0709 13:18:52.086232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.166829 (+ 80597us) service_pool.cc:224] Handling call
0709 13:18:55.604949 (+3438120us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:55.638685 (+ 33736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.850990 (+2212305us) spinlock_profiling.cc:243] Waited 3.43 s on lock 0x7b5400261450. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:57.851226 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7886764288}
I20260709 13:18:56.997946 15526 rpcz_store.cc:276] 0709 13:18:55.916384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.883132 (+966748us) service_pool.cc:224] Handling call
0709 13:18:56.883439 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.883631 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:56.944221 (+ 60590us) log.cc:844] Serialized 14 byte log entry
0709 13:18:56.996932 (+ 52711us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:56.997039 (+   107us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:56.997103 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.997194 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:56.997374 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5111,"prepare.queue_time_us":101343,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":1517,"thread_start_us":107284,"threads_started":2}
I20260709 13:18:57.883730 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:57.888121 15537 tablet_copy_service.cc:342] P d3c4f5bd76e0445795bad5f6dd93ff69: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-0af5649725244ae599dfc6ed7aeac5f1 received from {username='slave'} at 127.0.0.1:56840
I20260709 13:18:57.884244 15526 raft_consensus.cc:1240] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: []
W20260709 13:18:57.888947 15567 connection.cc:256] RPC call timeout handler was delayed by -1.0300985409999157s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1795s for a response
I20260709 13:18:57.889194 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1494) took 1745 ms. Trace:
I20260709 13:18:57.884465 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:57.885648 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:57.889608 15526 rpcz_store.cc:276] 0709 13:18:56.143535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.883593 (+1740058us) service_pool.cc:224] Handling call
0709 13:18:57.883929 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.888919 (+  4990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.889104 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:57.890761 15602 raft_consensus.cc:2410] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 4 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 13:18:57.890753 15567 connection.cc:256] RPC call timeout handler was delayed by -1.0075985410001067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18200000000000002s for a response
I20260709 13:18:57.858840 17489 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.894117 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:57.894992 15595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53310 (request call id 1517) took 1014 ms. Trace:
I20260709 13:18:57.894893 15602 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:57.895467 15595 rpcz_store.cc:276] 0709 13:18:56.880218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.885214 (+1004996us) service_pool.cc:224] Handling call
0709 13:18:57.894935 (+  9721us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:57.896202 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:57.896626 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:57.897768 15595 raft_consensus.cc:2393] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:18:57.896927 15602 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
W20260709 13:18:57.895433 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.898581 15672 raft_consensus.cc:1240] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:57.888873 15537 tablet_copy_service.cc:434] P d3c4f5bd76e0445795bad5f6dd93ff69: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-0af5649725244ae599dfc6ed7aeac5f1 on tablet 0af5649725244ae599dfc6ed7aeac5f1 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:57.881644 17479 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:57.908746 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:57.919302 15602 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:57.920104 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:57.891341 17472 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:57.920961 15519 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:18:57.920863 15602 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:18:57.922395 15602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 1571) took 1032 ms. Trace:
I20260709 13:18:57.922725 15602 rpcz_store.cc:276] 0709 13:18:56.889877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.920096 (+1030219us) service_pool.cc:224] Handling call
0709 13:18:57.920495 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.922127 (+  1632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.922325 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:57.923426 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:57.923687 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:57.924378 17510 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:18:57.926090 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.895746 17523 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:57.926909 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:57.927819 15676 raft_consensus.cc:2410] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 4 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:18:57.916110 15598 raft_consensus.cc:1275] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:57.928890 15598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 1570) took 1039 ms. Trace:
I20260709 13:18:57.929384 15598 rpcz_store.cc:276] 0709 13:18:56.889654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.915246 (+1025592us) service_pool.cc:224] Handling call
0709 13:18:57.915678 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.928565 (+ 12887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.928798 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:57.925827 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:57.929972 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:57.930928 15595 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:18:57.866837 17357 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:57.931354 15598 raft_consensus.cc:1275] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:18:57.911573 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.912068 15672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 395) took 1808 ms. Trace:
I20260709 13:18:57.913846 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:18:57.917177 17408 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:57.926688 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:57.930665 15669 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:57.931573 15595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 1565) took 1043 ms. Trace:
I20260709 13:18:57.933254 15641 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:57.896673 17489 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:57.934036 15668 raft_consensus.cc:1275] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:57.954180 17531 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:18:57.959873 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.960577 17357 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:57.961004 15641 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:57.961275 17357 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:18:57.951323 17547 log.cc:927] Time spent T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.135s	user 0.001s	sys 0.001s
I20260709 13:18:57.962198 15599 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:57.962775 15599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53324 (request call id 1558) took 1076 ms. Trace:
W20260709 13:18:57.960856 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.958163 15595 rpcz_store.cc:276] 0709 13:18:56.888454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.904412 (+1015958us) service_pool.cc:224] Handling call
0709 13:18:57.930554 (+ 26142us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.931342 (+   788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.931503 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:57.961737 17357 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:57.814898 16233 rpcz_store.cc:276] 0709 13:18:52.765982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.772037 (+  6055us) service_pool.cc:224] Handling call
0709 13:18:52.828777 (+ 56740us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:52.895317 (+ 66540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:52.895355 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:52.895905 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 13:18:54.207972 (+1312067us) spinlock_profiling.cc:243] Waited 2.33 ms on lock 0x7b48002508d8. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce9934aa1 00007f5ce992bb37 00007f5ce992b9ca 00007f5ce9a2326f 00007f5ce9a22f37 00007f5ce9a3755c 00007f5ce9a37508 00007f5ce9a37488 00007f5ce9a37443 00007f5ce9a366ec 00007f5ce8960a96
0709 13:18:54.719512 (+511540us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b18001ebc80. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab0d1 00007f5ce49c6a0f
0709 13:18:55.239439 (+519927us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:55.286895 (+ 47456us) log.cc:844] Serialized 14 byte log entry
0709 13:18:55.318563 (+ 31668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.333697 (+ 15134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:57.722840 (+2389143us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:18:57.814054 (+ 91214us) raft_consensus.cc:1685] finished
0709 13:18:57.814080 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.814276 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16623,"prepare.queue_time_us":42600,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":34325,"raft.run_cpu_time_us":19023,"raft.run_wall_time_us":1319064,"spinlock_wait_cycles":16690304,"thread_start_us":436532,"threads_started":1,"wal-append.queue_time_us":2446932}
W20260709 13:18:57.966130 17549 log.cc:927] Time spent T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:18:57.965183 17357 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:57.952411 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:57.976053 16203 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:57.939180 17526 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:57.963099 17489 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:57.933375 17521 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.986770 15641 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:57.987535 15598 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:18:57.988198 17287 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:57.988705 15519 raft_consensus.cc:1240] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:57.988907 17287 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:57.952402 17426 consensus_queue.cc:260] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:57.990269 17426 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:57.978374 17227 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:57.967228 15672 rpcz_store.cc:276] 0709 13:18:56.104123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.897809 (+1793686us) service_pool.cc:224] Handling call
0709 13:18:57.898219 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.903911 (+  5692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.911982 (+  8071us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:57.997632 17521 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:57.998689 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:57.999918 17521 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:58.002261 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:58.003129 15674 raft_consensus.cc:2393] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:18:58.003928 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:58.004551 15674 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:18:58.004773 15491 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:18:58.004899 15642 leader_election.cc:336] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Timed out: RequestConsensusVote RPC to 127.14.23.66:41593 timed out after 1.502s (SENT)
I20260709 13:18:58.005712 17464 consensus_queue.cc:1059] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:58.006959 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:58.007310 17531 consensus_queue.cc:1048] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:58.008340 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:58.009095 15673 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:18:58.009385 17531 raft_consensus.cc:3055] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:58.009862 17531 raft_consensus.cc:740] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:57.934082 17523 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:58.006426 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1495) took 1862 ms. Trace:
I20260709 13:18:58.005800 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:57.988129 15641 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:18:58.014739 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 46653.5us
I20260709 13:18:58.006289 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:58.023454 15528 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:58.030645 15519 rpcz_store.cc:276] 0709 13:18:56.143923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.934740 (+1790817us) service_pool.cc:224] Handling call
0709 13:18:57.988540 (+ 53800us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.006180 (+ 17640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.006372 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:58.039243 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 56325.4us
W20260709 13:18:58.044219 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.954440 17506 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:58.053226 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:58.012776 17445 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:18:58.052254 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.980258 17526 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:57.963107 15599 rpcz_store.cc:276] 0709 13:18:56.886720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.892116 (+1005396us) service_pool.cc:224] Handling call
0709 13:18:57.962717 (+ 70601us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:58.052117 15522 raft_consensus.cc:1240] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:58.074055 15677 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:18:58.055099 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 387) took 2979 ms. Trace:
I20260709 13:18:58.049364 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:57.994935 17429 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:58.053077 16238 raft_consensus.cc:2393] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:18:58.058104 15597 raft_consensus.cc:1240] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:58.055109 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:58.059012 15676 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:18:58.059331 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:58.079752 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:58.060143 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 72114.9us
I20260709 13:18:58.060655 17521 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:58.061095 15673 raft_consensus.cc:1240] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: []
W20260709 13:18:58.061257 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.062134 16203 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:58.063323 17472 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.063360 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 397) took 2145 ms. Trace:
I20260709 13:18:58.063494 17479 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:58.065014 17506 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:58.085458 15676 raft_consensus.cc:1240] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 1. Current term is 2. Ops: []
W20260709 13:18:58.085583 17275 log.cc:927] Time spent T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:18:58.086347 17275 tablet_bootstrap.cc:492] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:18:58.086500 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.086850 17519 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.087417 17519 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:58.088718 17556 log.cc:927] Time spent T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:18:58.089524 17353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd36c2cfd99c45cea9e48096d54bed1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:58.090251 15526 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:58.065426 17482 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.091028 15669 raft_consensus.cc:3055] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:58.067488 15525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 398) took 2142 ms. Trace:
I20260709 13:18:58.035907 17489 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:58.056640 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:57.994257 17227 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:58.005195 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:58.075505 15670 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:18:58.074550 15528 rpcz_store.cc:276] 0709 13:18:55.069857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.070015 (+   158us) service_pool.cc:224] Handling call
0709 13:18:55.070313 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.879977 (+2809664us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b54003023a0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4980a1d 00007f5ce49671a4 00007f5ce49e9447 00007f5ce4a01fe7 00007f5ce4a037de 00007f5ce4a0db49 00007f5ce4a0e416 00007f5ce4a0ad8f 00007f5ce4a0a83e 00007f5ce99cf419
0709 13:18:58.030831 (+150854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.047971 (+ 17140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":378576128}
I20260709 13:18:58.078872 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:58.094863 16240 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:58.080572 15641 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:58.099499 17408 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:58.099929 17408 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:58.100351 17408 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:58.101222 17361 consensus_queue.cc:1059] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:58.102187 17361 consensus_queue.cc:1243] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer d3c4f5bd76e0445795bad5f6dd93ff69 log is divergent from this leader: its last log entry 5.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:18:58.102439 17367 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.102980 17367 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:58.094859 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:58.103976 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:58.103478 15528 raft_consensus.cc:1240] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:58.106169 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1506) took 1241 ms. Trace:
W20260709 13:18:58.106540 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 49289.6us
I20260709 13:18:58.108264 16204 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:58.106499 15528 rpcz_store.cc:276] 0709 13:18:56.864691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.102913 (+1238222us) service_pool.cc:224] Handling call
0709 13:18:58.103283 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.105890 (+  2607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.106091 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:58.110517 17275 tablet_bootstrap.cc:492] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:18:58.111562 17275 ts_tablet_manager.cc:1403] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 2.814s	user 0.017s	sys 0.028s
I20260709 13:18:58.081103 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1496) took 1936 ms. Trace:
W20260709 13:18:58.073608 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 199753us
I20260709 13:18:58.083655 15597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 1509) took 1221 ms. Trace:
I20260709 13:18:58.083122 15523 rpcz_store.cc:276] 0709 13:18:55.918106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.782469 (+1864363us) service_pool.cc:224] Handling call
0709 13:18:57.783108 (+   639us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:57.784034 (+   926us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:57.784069 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:57.784741 (+   672us) log.cc:844] Serialized 26 byte log entry
0709 13:18:57.905834 (+121093us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:57.905872 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.905955 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:58.063086 (+157131us) raft_consensus.cc:1685] finished
0709 13:18:58.063111 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.063245 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":150,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":147,"thread_start_us":118144,"threads_started":1,"wal-append.queue_time_us":265555}
I20260709 13:18:58.113337 15522 rpcz_store.cc:276] 0709 13:18:56.144215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.942724 (+1798509us) service_pool.cc:224] Handling call
0709 13:18:58.014279 (+ 71555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.079193 (+ 64914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.081044 (+  1851us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:58.011354 17531 consensus_queue.cc:260] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:58.091394 15669 raft_consensus.cc:740] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:58.056942 17523 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:58.134976 15600 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
W20260709 13:18:58.122462 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.113477 15597 rpcz_store.cc:276] 0709 13:18:56.856469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.856897 (+   428us) service_pool.cc:224] Handling call
0709 13:18:58.045867 (+1188970us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b5800113498. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99e4fc0 00007f5ce99cf929 00007f5ce99cf3d2 00007f5ce2f46bd4
0709 13:18:58.057805 (+ 11938us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.077883 (+ 20078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.078091 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37153664}
I20260709 13:18:58.144511 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:58.147167 17526 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:58.150520 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:58.152098 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:18:58.152782 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.138321 15675 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:18:58.092502 17489 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.135044 17375 raft_consensus.cc:2804] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:58.165947 17489 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:58.171272 15669 consensus_queue.cc:260] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:18:58.173202 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 403) took 1025 ms. Trace:
I20260709 13:18:58.176205 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:58.182399 15522 rpcz_store.cc:276] 0709 13:18:57.147459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.114217 (+966758us) service_pool.cc:224] Handling call
0709 13:18:58.172512 (+ 58295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.172817 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:58.172843 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:58.172903 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:58.172933 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.173007 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.173129 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:58.153667 15642 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:18:58.187949 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:58.192392 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 130351us
I20260709 13:18:58.199774 16240 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
W20260709 13:18:58.202318 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 48718.6us
I20260709 13:18:58.179106 17361 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:18:58.203373 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 236854us
I20260709 13:18:58.180455 17375 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.179715 17526 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:58.209369 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:58.213529 17506 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:58.166481 17326 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:18:58.168746 16233 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:58.214879 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:58.112982 17482 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:58.163712 17479 consensus_queue.cc:1059] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:18:58.154084 17472 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:58.222184 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 407) took 1057 ms. Trace:
I20260709 13:18:58.224509 15523 rpcz_store.cc:276] 0709 13:18:57.165287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.138494 (+973207us) service_pool.cc:224] Handling call
0709 13:18:58.179023 (+ 40529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.215847 (+ 36824us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:58.215877 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:58.221862 (+  5985us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:58.221905 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.221990 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.222122 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:58.217098 15642 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:58.218497 15597 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:58.203020 17429 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:58.211028 17375 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:58.242720 15669 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:58.248716 15601 raft_consensus.cc:2410] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:18:58.243522 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 96788.2us
I20260709 13:18:58.252938 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:58.253676 15597 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:58.233450 17429 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:58.265579 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.229138 17526 raft_consensus.cc:697] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:58.262434 15523 raft_consensus.cc:1275] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:18:58.267688 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.271727 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:58.249706 17506 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:58.250257 17476 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.273468 16205 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:58.286558 15568 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:58.289589 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 408) took 1038 ms. Trace:
I20260709 13:18:58.297742 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:58.298693 17227 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:58.300894 17472 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:58.268862 17429 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:58.289717 15597 raft_consensus.cc:2376] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3. Re-sending same reply.
I20260709 13:18:58.308641 16240 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:58.290297 15520 rpcz_store.cc:276] 0709 13:18:57.172794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.177693 (+1004899us) service_pool.cc:224] Handling call
0709 13:18:58.193281 (+ 15588us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.210911 (+ 17630us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:58.210945 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:58.210996 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:58.211029 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.211094 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.211225 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:58.315131 17519 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:58.316848 15599 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:58.318503 15524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1505) took 2131 ms. Trace:
I20260709 13:18:58.318919 15524 rpcz_store.cc:276] 0709 13:18:56.187252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.051061 (+1863809us) service_pool.cc:224] Handling call
0709 13:18:58.083503 (+ 32442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:58.102448 (+ 18945us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:58.102489 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:58.102936 (+   447us) log.cc:844] Serialized 26 byte log entry
0709 13:18:58.196621 (+ 93685us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:58.216058 (+ 19437us) log.cc:844] Serialized 14 byte log entry
0709 13:18:58.216542 (+   484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.216638 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:58.318300 (+101662us) raft_consensus.cc:1685] finished
0709 13:18:58.318317 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.318446 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4646,"prepare.queue_time_us":18983,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":4280,"raft.queue_time_us":14139,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":38430,"thread_start_us":93174,"threads_started":1,"wal-append.queue_time_us":166556}
I20260709 13:18:58.321540 15669 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:18:58.282409 17506 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.325910 17307 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:18:58.326421 17553 log.cc:927] Time spent T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:18:58.092216 15525 rpcz_store.cc:276] 0709 13:18:55.924955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.882788 (+1957833us) service_pool.cc:224] Handling call
0709 13:18:57.883332 (+   544us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:57.883837 (+   505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:57.883868 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:57.884431 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 13:18:57.980869 (+ 96438us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:57.982135 (+  1266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.986002 (+  3867us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:58.067235 (+ 81233us) raft_consensus.cc:1685] finished
0709 13:18:58.067262 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.067410 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5760,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":113,"thread_start_us":85067,"threads_started":1,"wal-append.queue_time_us":170889}
I20260709 13:18:58.327981 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:58.350035 16240 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
W20260709 13:18:58.309355 17554 log.cc:927] Time spent T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:18:58.325810 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:58.323405 17476 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:58.358513 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 92739.2us
I20260709 13:18:58.359376 16205 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:18:58.362404 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 77077.5us
W20260709 13:18:58.345297 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 146162us
I20260709 13:18:58.355891 15597 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:58.326164 16233 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:58.365604 17461 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:58.332248 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 402) took 1187 ms. Trace:
I20260709 13:18:58.368036 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:58.334136 17307 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.366436 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:58.335417 17523 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:58.333369 17506 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:58.289580 17224 consensus_queue.cc:1059] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.163s
I20260709 13:18:58.372248 15528 rpcz_store.cc:276] 0709 13:18:57.144673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.110838 (+966165us) service_pool.cc:224] Handling call
0709 13:18:58.111231 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.111557 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:58.111872 (+   315us) log.cc:844] Serialized 14 byte log entry
0709 13:18:58.331460 (+219588us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:58.331591 (+   131us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:58.331693 (+   102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.331855 (+   162us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.332121 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":304,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":24875,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":9855,"thread_start_us":194251,"threads_started":1}
W20260709 13:18:58.373610 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 126296us
I20260709 13:18:58.372778 15599 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:18:58.382246 15599 raft_consensus.cc:1275] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:58.383015 17570 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:58.384338 15602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 277) took 1468 ms. Trace:
I20260709 13:18:58.384582 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:58.385109 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:58.385828 16233 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:58.385939 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:58.387701 15602 rpcz_store.cc:276] 0709 13:18:56.915861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.923603 (+1007742us) service_pool.cc:224] Handling call
0709 13:18:57.923974 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:57.924465 (+   491us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:57.924495 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:57.925050 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 13:18:58.034469 (+109419us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:58.034510 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.034577 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:58.384102 (+349525us) raft_consensus.cc:1685] finished
0709 13:18:58.384125 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.384267 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":613,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":112,"thread_start_us":88909,"threads_started":1,"wal-append.queue_time_us":241706}
W20260709 13:18:58.378464 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 114166us
W20260709 13:18:58.387140 16234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 369) took 2144 ms (client timeout 1574 ms). Trace:
W20260709 13:18:58.392725 16234 rpcz_store.cc:269] 0709 13:18:56.242824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.242975 (+   151us) service_pool.cc:224] Handling call
0709 13:18:58.386940 (+2143965us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54006937d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:58.387054 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3328860928}
I20260709 13:18:58.375388 17307 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:58.375748 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:58.401760 17361 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:58.402941 15642 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:58.403055 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:58.403124 16205 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:58.406603 15597 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:58.393759 17357 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:58.409355 15597 raft_consensus.cc:1275] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:58.434657 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:58.379055 17343 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:18:58.433858 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:58.409416 17361 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.409891 17479 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:58.456434 17361 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:58.438678 15567 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:18:58.457522 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 48411.5us
W20260709 13:18:58.457779 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 59005.8us
I20260709 13:18:58.458513 15567 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:58.459839 15567 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:58.460538 17476 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:58.461036 15567 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:58.410521 17526 consensus_queue.cc:237] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:58.462370 17219 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:58.456605 17479 raft_consensus.cc:697] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:58.464430 17479 consensus_queue.cc:237] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:58.465019 17521 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:58.465615 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:58.466370 15671 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:18:58.466652 17576 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:58.439272 17571 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:58.439788 17275 raft_consensus.cc:359] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:58.442359 17429 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:58.447273 17562 log.cc:927] Time spent T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:18:58.454341 15594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 278) took 1507 ms. Trace:
I20260709 13:18:58.469198 17275 raft_consensus.cc:740] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:18:58.454739 17523 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:58.471530 17275 consensus_queue.cc:260] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:58.455067 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:58.472669 17579 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:58.473143 17579 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:58.473508 17579 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:58.473868 15671 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:18:58.475556 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.477435 17367 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:58.478266 16238 raft_consensus.cc:2393] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:18:58.465538 17521 raft_consensus.cc:697] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:58.471088 15594 rpcz_store.cc:276] 0709 13:18:56.946675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.927803 (+981128us) service_pool.cc:224] Handling call
0709 13:18:57.928206 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:57.928652 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:57.928684 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:57.929255 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 13:18:57.974152 (+ 44897us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:58.208909 (+234757us) log.cc:844] Serialized 14 byte log entry
0709 13:18:58.372089 (+163180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.375404 (+  3315us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:58.454065 (+ 78661us) raft_consensus.cc:1685] finished
0709 13:18:58.454083 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.454250 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":323607,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":67,"raft.queue_time_us":98523,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":22918,"thread_start_us":292295,"threads_started":3,"wal-append.queue_time_us":83871}
I20260709 13:18:58.481489 15567 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:58.482682 17568 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:58.483115 17568 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.483474 17568 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:58.470244 17570 consensus_queue.cc:1048] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:18:58.466948 17445 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:18:58.468803 16203 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:58.463302 17357 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:58.487746 16203 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:58.488967 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:58.490742 16239 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:18:58.484661 15567 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:58.471419 15687 tablet_copy_service.cc:342] P 3ef186ec12554a1d87814af0a0ace9a4: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-b99eaeae24b44aa78cfa8c90caca288e received from {username='slave'} at 127.0.0.1:57710
I20260709 13:18:58.514302 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:58.515079 15669 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:58.523108 15566 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:18:58.527047 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.527932 15687 tablet_copy_service.cc:434] P 3ef186ec12554a1d87814af0a0ace9a4: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-b99eaeae24b44aa78cfa8c90caca288e on tablet b99eaeae24b44aa78cfa8c90caca288e with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:58.538228 15567 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:58.528631 17429 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:58.533391 17408 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:58.559192 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 156392us
I20260709 13:18:58.558828 15566 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:58.548780 17576 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:58.563165 17326 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:58.563771 17583 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.556293 17521 consensus_queue.cc:237] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:58.564251 17357 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:58.563822 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.565148 17307 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:58.576056 15669 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:58.566139 17343 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:18:58.577889 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:58.577536 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 51960.2us
I20260709 13:18:58.557497 17429 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:58.547889 17375 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:58.582047 15566 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:18:58.582261 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 122740us
I20260709 13:18:58.567651 17576 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.580683 17357 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:58.571424 17571 raft_consensus.cc:697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:58.584601 17583 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:58.571457 17219 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:58.568264 17489 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:58.592944 17512 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:58.619758 15567 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:58.607275 17343 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:58.622048 15566 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:58.593371 17576 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:58.586434 17578 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:58.623077 17275 ts_tablet_manager.cc:1434] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.511s	user 0.008s	sys 0.014s
I20260709 13:18:58.624788 17408 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:58.622609 17357 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:58.627442 17343 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:58.630296 17275 ts_tablet_manager.cc:1467] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9: Tablet startup took 517ms
I20260709 13:18:58.636978 15566 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:58.633553 17343 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:58.639298 15595 raft_consensus.cc:1275] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:58.635437 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:58.627812 17361 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:58.633575 15669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:18:58.670158 15669 raft_consensus.cc:606] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Received request to transfer leadership to TS dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:58.685798 15669 raft_consensus.cc:614] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:18:58.687703 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 200048us
I20260709 13:18:58.688455 16203 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:58.691534 15567 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:58.672386 17512 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:58.697109 15566 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:18:58.695236 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 44133.5us
I20260709 13:18:58.672290 17307 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:58.653215 17408 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:18:58.701501 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1501) took 2553 ms. Trace:
I20260709 13:18:58.682183 15594 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:18:58.700654 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:58.738848 17593 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:58.739534 17593 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:58.740060 17593 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:58.699239 17445 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:18:58.739624 17589 log.cc:927] Time spent T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:18:58.755156 17569 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:58.774183 17569 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.774619 17569 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:58.756538 17583 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:18:58.757943 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 57345.8us
I20260709 13:18:58.781147 17353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:53976 (request call id 342) took 7052 ms. Trace:
W20260709 13:18:58.758553 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 69432.9us
I20260709 13:18:58.759879 17572 consensus_queue.cc:1059] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 13:18:58.784870 17572 raft_consensus.cc:3055] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:58.785271 17572 raft_consensus.cc:740] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:58.695359 17571 consensus_queue.cc:237] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:58.786178 17572 consensus_queue.cc:260] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:58.787372 17572 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:58.760710 17574 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:58.761173 17590 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.090s
I20260709 13:18:58.789672 17361 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:58.790230 17590 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:58.761528 17326 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:58.792702 17326 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:58.761780 15603 raft_consensus.cc:1240] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:58.762429 17489 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:58.762617 17461 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:58.742019 17580 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:58.822676 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:58.763649 17445 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:58.875926 15603 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:58.764484 17526 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:18:58.765252 17584 log.cc:927] Time spent T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:18:58.766561 17579 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:58.768702 17506 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:58.769209 17553 log.cc:927] Time spent T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:18:58.710994 17375 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:18:58.740000 17591 log.cc:927] Time spent T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.082s	user 0.000s	sys 0.002s
I20260709 13:18:58.771162 17594 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:58.701858 15519 rpcz_store.cc:276] 0709 13:18:56.145609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.048630 (+1903021us) service_pool.cc:224] Handling call
0709 13:18:58.058355 (+  9725us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:58.058760 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:58.058802 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:58.059255 (+   453us) log.cc:844] Serialized 26 byte log entry
0709 13:18:58.226685 (+167430us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:58.403365 (+176680us) log.cc:844] Serialized 14 byte log entry
0709 13:18:58.403599 (+   234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.403676 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:58.624029 (+220353us) spinlock_profiling.cc:243] Waited 4.9 ms on lock 0x7b180030ff40. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce191e3f3 00007f5ce19207c1 00007f5ce1920779 00007f5ce1920709 00007f5ce19206d1 00007f5ce191f9cd 000000000038f354 000000000038f199 00007f5ce190183a
0709 13:18:58.693968 (+ 69939us) raft_consensus.cc:1685] finished
0709 13:18:58.696760 (+  2792us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.698699 (+  1939us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25650,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":12322,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":11664,"spinlock_wait_cycles":11261824,"thread_start_us":318757,"threads_started":2,"wal-append.queue_time_us":258925}
I20260709 13:18:58.896279 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:58.806035 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:58.820813 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:58.897320 16236 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:58.897320 16239 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:58.821801 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:58.881237 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:58.898622 16240 raft_consensus.cc:2393] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 2: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:18:58.875406 17580 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.892130 16234 raft_consensus.cc:1240] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: []
W20260709 13:18:58.875949 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 185799us
I20260709 13:18:58.877223 17361 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.903261 15600 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:58.882329 17408 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.904464 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:58.879143 17526 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:58.909578 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:58.907891 17526 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:58.911823 17594 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:58.922585 16240 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:18:58.914300 15566 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:18:58.900873 17307 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.934684 15603 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:58.934191 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:58.935606 15669 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:58.903429 17580 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:58.922372 17326 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:58.909421 17586 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.978596 17307 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:58.978947 17586 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:58.979740 17598 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:58.910216 17408 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:58.983072 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:58.913960 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:58.922820 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:58.988780 15642 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:18:58.991655 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 77440.2us
I20260709 13:18:58.992869 15566 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:58.998029 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:58.979717 17568 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:59.003521 15603 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:58.933991 17361 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:58.781669 17353 rpcz_store.cc:276] 0709 13:18:51.729066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.737558 (+  8492us) service_pool.cc:224] Handling call
0709 13:18:52.189869 (+452311us) ts_tablet_manager.cc:934] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
0709 13:18:58.082075 (+5892206us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:18:58.449382 (+367307us) tablet_metadata.cc:665] Metadata flushed
0709 13:18:58.449521 (+   139us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:18:58.780159 (+330638us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:18:58.780501 (+   342us) tablet_replica.cc:206] Creating consensus instance
0709 13:18:58.780882 (+   381us) ts_tablet_manager.cc:989] Replica registered.
0709 13:18:58.780998 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":277,"tablet-copy.queue_time_us":321421,"thread_start_us":665231,"threads_started":2}
I20260709 13:18:59.005280 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:59.006125 15527 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1482) took 3174 ms (client timeout 1697 ms). Trace:
I20260709 13:18:59.008023 15493 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:59.010398 15669 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:18:59.008853 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:59.010510 15527 rpcz_store.cc:269] 0709 13:18:55.826928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.827040 (+   112us) service_pool.cc:224] Handling call
0709 13:18:58.987873 (+3160833us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b5400070dd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:59.001170 (+ 13297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3880480128}
W20260709 13:18:59.021055 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.992595 17479 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:59.011216 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:59.052352 15603 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:58.996220 16233 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:18:58.998520 17598 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:18:59.054512 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 346481us
W20260709 13:18:59.062528 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 354416us
I20260709 13:18:59.059301 17598 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.009577 17408 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:18:59.063232 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 67432.1us
I20260709 13:18:59.054570 17489 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:59.088358 16203 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:59.096540 15493 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:59.099704 17568 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:59.066399 17594 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:18:59.130353 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.067016 17353 tablet_copy_client.cc:806] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:59.081293 17598 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:59.094014 17408 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:59.111773 17506 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:18:59.147790 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 80852.9us
W20260709 13:18:59.148993 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 160288us
W20260709 13:18:58.640527 17275 ts_tablet_manager.cc:1469] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9: Trace:
0709 13:18:48.544613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.565506 (+ 20893us) service_pool.cc:224] Handling call
0709 13:18:48.863064 (+297558us) ts_tablet_manager.cc:934] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9: Initiating tablet copy from peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
0709 13:18:51.486165 (+2623101us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:18:51.692007 (+205842us) tablet_metadata.cc:665] Metadata flushed
0709 13:18:51.707054 (+ 15047us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:18:52.015243 (+308189us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:18:52.015611 (+   368us) tablet_replica.cc:206] Creating consensus instance
0709 13:18:52.015954 (+   343us) ts_tablet_manager.cc:989] Replica registered.
0709 13:18:52.016020 (+    66us) inbound_call.cc:177] Queueing success response
0709 13:18:55.297074 (+3281054us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:18:58.111905 (+2814831us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:18:58.361273 (+249368us) tablet_replica.cc:261] Starting instrumentation
0709 13:18:58.386993 (+ 25720us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":261,"mutex_wait_us":4887,"raft.queue_time_us":51562,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":6423,"tablet-copy.queue_time_us":277622,"tablet-download-pool.queue_time_us":41284,"tablet-download-pool.run_cpu_time_us":2541,"tablet-download-pool.run_wall_time_us":2023622,"thread_start_us":1035856,"threads_started":3}
W20260709 13:18:59.144049 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 186500us
I20260709 13:18:59.145562 17429 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:59.151875 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:59.152626 15597 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:59.136724 17329 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:59.138867 17579 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:59.160229 15493 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:18:59.160378 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.142660 17521 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.161239 15603 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:18:59.111173 17343 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:59.141986 17408 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:18:59.167201 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.144972 17353 tablet_copy_client.cc:670] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:59.156865 17429 raft_consensus.cc:697] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:59.170522 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:59.171231 15603 raft_consensus.cc:2393] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 6: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:59.171882 15521 raft_consensus.cc:1275] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:59.175663 15687 tablet_copy_service.cc:342] P 3ef186ec12554a1d87814af0a0ace9a4: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-582343936346420b87ecf2eebd00dc25 received from {username='slave'} at 127.0.0.1:57710
I20260709 13:18:59.178963 15602 raft_consensus.cc:2393] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:59.174081 17429 consensus_queue.cc:237] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:59.170065 17567 log.cc:927] Time spent T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.718s	user 0.000s	sys 0.017s
I20260709 13:18:59.169046 17408 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:59.181465 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:59.168628 17489 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:59.180655 17612 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:59.183647 17612 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:59.188326 15601 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34102 (request call id 281) took 1509 ms (client timeout 1994 ms). Trace:
W20260709 13:18:59.188777 15601 rpcz_store.cc:269] 0709 13:18:57.678933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.855093 (+176160us) service_pool.cc:224] Handling call
0709 13:18:59.188072 (+1332979us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b540016c454. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:18:59.188225 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":865934592}
I20260709 13:18:59.181079 15687 tablet_copy_service.cc:434] P 3ef186ec12554a1d87814af0a0ace9a4: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-582343936346420b87ecf2eebd00dc25 on tablet 582343936346420b87ecf2eebd00dc25 with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:59.167979 17521 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:59.192747 17521 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:59.194913 17361 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:59.195916 15596 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:18:59.196352 15596 pending_rounds.cc:85] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting all ops after (but not including) 4
I20260709 13:18:59.196573 15596 pending_rounds.cc:107] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:18:59.196826 15596 raft_consensus.cc:2889] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:59.197755 16236 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:18:59.202476 16205 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:18:59.206274 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:59.206354 17307 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:59.210516 17343 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:59.211067 17353 tablet_copy_client.cc:538] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:59.211184 15677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49094 (request call id 1626) took 1306 ms. Trace:
I20260709 13:18:59.211606 15677 rpcz_store.cc:276] 0709 13:18:57.905853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.914682 (+  8829us) service_pool.cc:224] Handling call
0709 13:18:57.963615 (+ 48933us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:58.074481 (+110866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.210897 (+1136416us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b540026b6d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:18:59.211088 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":246174464}
I20260709 13:18:59.218573 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:59.219174 15677 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:59.224748 15642 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:18:59.224551 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 74729.6us
I20260709 13:18:59.234236 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:18:59.236023 17576 consensus_meta.cc:287] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.602s	user 0.009s	sys 0.000s
I20260709 13:18:59.236284 17489 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:59.236802 17489 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:59.237181 17489 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:59.228821 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:59.239069 15601 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:18:59.238893 17593 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:59.228972 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:59.229578 15595 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:18:59.230284 17427 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:59.230602 15597 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:59.230906 17570 consensus_queue.cc:1059] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:59.232174 17604 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:59.232558 17224 consensus_queue.cc:1059] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:59.240732 15601 raft_consensus.cc:1275] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:59.266040 15595 raft_consensus.cc:606] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
W20260709 13:18:59.249378 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.266454 15595 raft_consensus.cc:614] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:59.246430 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:18:59.258749 16231 raft_consensus.cc:1240] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:59.185817 17612 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:59.271085 17329 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.270810 17569 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:59.271652 17329 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:59.259534 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:59.265133 17479 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.272794 15673 raft_consensus.cc:2393] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 2: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:18:59.272940 17479 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:59.273981 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:59.274197 17224 raft_consensus.cc:3055] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:59.260403 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:59.274560 17224 raft_consensus.cc:740] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:59.268826 15493 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:59.188957 17526 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:59.258646 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:18:59.272222 17573 log.cc:927] Time spent T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.708s	user 0.007s	sys 0.011s
I20260709 13:18:59.244999 15677 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:18:59.279944 15675 raft_consensus.cc:2393] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:18:59.280499 16234 raft_consensus.cc:1240] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:59.254842 17427 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:18:59.219939 17583 consensus_queue.cc:1059] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:18:59.273507 17570 raft_consensus.cc:3055] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:59.273877 17569 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:18:59.281845 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 57182.4us
I20260709 13:18:59.236528 17576 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:59.275482 17224 consensus_queue.cc:260] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:59.285215 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:59.369895 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:59.370702 15677 raft_consensus.cc:2393] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:18:59.371510 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:59.372165 15677 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
W20260709 13:18:59.373024 17287 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:18:59.374789 16203 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:59.375104 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:59.375934 15669 raft_consensus.cc:2393] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:18:59.373044 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:59.376874 15677 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:59.384636 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 46505.3us
I20260709 13:18:59.310973 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:59.377471 15642 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:59.373349 17598 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:59.393169 15603 raft_consensus.cc:2393] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 3: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:59.394266 17598 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:59.290361 15566 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:18:59.398819 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 108586us
W20260709 13:18:59.400563 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 131840us
W20260709 13:18:59.411525 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.281733 17521 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:59.284157 17427 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:59.348001 17612 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:59.334372 17329 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:59.419871 15493 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:59.421727 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:59.383652 17570 raft_consensus.cc:740] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:59.400468 17429 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:18:59.419849 17589 log.cc:927] Time spent T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:18:59.437000 15675 raft_consensus.cc:2393] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 3: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:18:59.438815 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:59.437417 15642 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:59.420521 17506 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:59.419498 17598 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:59.422431 17601 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.444352 15642 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:59.447402 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:59.448869 15675 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:59.420208 17521 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.402205 17489 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:59.456151 15677 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:18:59.457502 15566 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:59.442906 17329 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:59.460122 17426 raft_consensus.cc:3055] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:18:59.462663 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 59331.8us
I20260709 13:18:59.454707 17615 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:59.445606 17506 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.440739 17429 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:18:59.460582 17616 log.cc:927] Time spent T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:18:59.424058 17612 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:59.448297 17593 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:59.452950 17521 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:59.467164 15642 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:18:59.454893 17598 raft_consensus.cc:697] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:59.495481 15493 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:59.495863 15677 raft_consensus.cc:1240] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:59.496775 15642 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:59.462754 17579 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:59.467656 17526 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:59.463268 17426 raft_consensus.cc:740] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:59.498459 15642 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:59.470232 17429 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:59.514427 15493 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:18:59.466661 17506 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:59.462950 17611 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:59.465469 17610 consensus_queue.cc:1048] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 13:18:59.546789 17610 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:59.468019 17307 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.521102 17625 raft_consensus.cc:2804] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:59.546352 17598 consensus_queue.cc:237] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:59.503695 17583 raft_consensus.cc:2764] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:18:59.547256 17569 consensus_queue.cc:1059] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:18:59.545387 15675 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:18:59.556504 17623 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.557093 15594 raft_consensus.cc:1240] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:59.560237 15676 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:59.562088 17611 raft_consensus.cc:697] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:59.564314 17506 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:59.567039 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:59.567219 17586 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:59.521560 17329 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:18:59.575206 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.512326 17361 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:59.582284 15674 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:18:59.578301 15669 raft_consensus.cc:1240] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:59.604005 15675 raft_consensus.cc:1240] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:59.547214 17610 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.521534 17579 raft_consensus.cc:697] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:59.603904 15674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49100 (request call id 1646) took 1588 ms. Trace:
I20260709 13:18:59.466929 17601 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:59.470382 17570 consensus_queue.cc:260] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:59.512636 17609 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.129s
W20260709 13:18:59.730892 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.728055 15676 raft_consensus.cc:1240] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:59.735684 16203 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:59.555954 17625 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:59.652565 17219 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.739058 17625 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:59.629029 17329 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:59.749837 15674 rpcz_store.cc:276] 0709 13:18:58.015414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.037600 (+ 22186us) service_pool.cc:224] Handling call
0709 13:18:59.566895 (+1529295us) spinlock_profiling.cc:243] Waited 1.58 ms on lock 0x7b5800012398. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6e9c 00007f5ce18e6a5b 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99d10a0 00007f5ce99d0984 00007f5ce2f4b6d4 00007f5ce2f4b662
0709 13:18:59.603841 (+ 36946us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5309568}
I20260709 13:18:59.740887 17459 raft_consensus.cc:3055] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:18:59.750316 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 251903us
W20260709 13:18:59.655603 17580 consensus_meta.cc:287] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.664s	user 0.010s	sys 0.000s
I20260709 13:18:59.732309 17506 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:18:59.629124 17343 raft_consensus.cc:3055] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:59.609126 17610 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:59.654414 17576 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:59.595855 17307 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:18:59.770088 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.671217 17489 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:59.635638 17569 consensus_queue.cc:1059] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.135s
I20260709 13:18:59.603504 17361 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.628201 17615 raft_consensus.cc:697] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:59.756600 17459 raft_consensus.cc:740] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:59.772678 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:18:59.751611 17408 consensus_meta.cc:287] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.533s	user 0.008s	sys 0.002s
I20260709 13:18:59.795526 17408 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:59.773629 15642 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:59.755867 17482 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:59.782817 17355 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:59.784226 15676 raft_consensus.cc:1240] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:59.786026 17307 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:59.656888 17611 consensus_queue.cc:237] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:59.746538 17594 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:18:59.790583 17329 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:18:59.774684 15491 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:59.760764 17583 raft_consensus.cc:3055] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:59.654927 17427 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:59.771952 17612 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:59.755695 17219 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:59.794096 17601 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:59.605823 17623 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:59.776818 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.772958 17343 raft_consensus.cc:740] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:18:59.802407 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:59.789067 17361 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:59.770830 17580 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:18:59.827018 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 52431.9us
I20260709 13:18:59.654085 17426 consensus_queue.cc:260] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:59.797425 17489 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:18:59.842830 15642 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:18:59.851253 17521 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:59.852406 17307 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:59.852881 17307 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:18:59.853250 17307 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:59.839123 17579 consensus_queue.cc:237] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:18:59.809669 17355 raft_consensus.cc:697] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:18:59.858850 17355 consensus_queue.cc:237] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:59.830494 15673 raft_consensus.cc:2393] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:18:59.844359 17612 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:59.863301 17612 raft_consensus.cc:697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:59.864357 17612 consensus_queue.cc:237] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:59.844587 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:59.866452 15676 raft_consensus.cc:2410] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:18:59.867432 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:59.868736 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:59.828819 17482 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:59.869693 17482 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.870203 17482 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:59.838234 17329 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:59.873651 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:18:59.875015 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:59.875701 17329 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:59.845988 17343 consensus_queue.cc:260] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:59.826653 17583 raft_consensus.cc:740] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:18:59.847592 15491 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:59.877440 17583 consensus_queue.cc:260] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:59.878357 17531 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:59.879247 17343 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:59.880671 17623 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:59.881222 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:59.869338 15676 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:18:59.882206 17489 consensus_queue.cc:1059] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:59.883337 17353 tablet_bootstrap.cc:492] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap starting.
I20260709 13:18:59.885236 17489 raft_consensus.cc:3055] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:59.885634 17489 raft_consensus.cc:740] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:18:59.887825 17459 consensus_queue.cc:260] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:59.887931 17489 consensus_queue.cc:260] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:59.889024 17459 raft_consensus.cc:3060] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:59.889092 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:18:59.889366 17489 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:59.881857 15524 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:59.890918 15528 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:59.886526 17219 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:18:59.886711 15527 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:59.892767 17408 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:18:59.876284 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:18:59.895334 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.895639 15675 raft_consensus.cc:2393] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:18:59.897043 15491 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:18:59.878566 17583 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:59.904150 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:59.906288 15670 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 3: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:18:59.906430 15675 raft_consensus.cc:2393] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 6: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:18:59.905385 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:59.909214 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:18:59.829666 17429 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:18:59.881270 17623 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:18:59.943604 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:59.950465 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 53511.3us
I20260709 13:18:59.902678 17506 consensus_queue.cc:1048] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:18:59.902870 17609 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:18:59.940691 17623 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.968739 15524 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:18:59.953207 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:59.975139 17623 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:59.985852 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.978356 17631 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:59.974892 17639 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:59.993089 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:59.993695 15600 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:59.907917 17615 consensus_queue.cc:237] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:18:59.996408 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:18:59.997329 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:18:59.998103 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:59.996476 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:00.006372 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:00.006932 16235 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:19:00.007032 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:00.004344 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:18:59.983702 16231 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:00.010279 15596 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:18:59.991524 17631 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.019142 15524 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:00.022866 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1499) took 4185 ms. Trace:
I20260709 13:19:00.023408 15521 rpcz_store.cc:276] 0709 13:18:55.837805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.035551 (+197746us) service_pool.cc:224] Handling call
0709 13:18:59.029765 (+2994214us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.172326 (+142561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.022608 (+850282us) spinlock_profiling.cc:243] Waited 2.9 s on lock 0x7b5400070dd0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:00.022802 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6680085248}
I20260709 13:19:00.024057 16239 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:00.025068 15599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 290) took 2343 ms. Trace:
I20260709 13:18:59.991052 17639 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.025617 15599 rpcz_store.cc:276] 0709 13:18:57.681905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.383117 (+701212us) service_pool.cc:224] Handling call
0709 13:18:58.383452 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:58.384807 (+  1355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:58.384836 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:58.385424 (+   588us) log.cc:844] Serialized 26 byte log entry
0709 13:18:58.520221 (+134797us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:58.548459 (+ 28238us) log.cc:844] Serialized 14 byte log entry
0709 13:18:58.561882 (+ 13423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.569402 (+  7520us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:59.998512 (+1429110us) spinlock_profiling.cc:243] Waited 6.79 ms on lock 0x7b18002030e0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce191e3f3 00007f5ce19207c1 00007f5ce1920779 00007f5ce1920709 00007f5ce19206d1 00007f5ce191f9cd 000000000038f354 000000000038f199 00007f5ce190183a
0709 13:19:00.024668 (+ 26156us) raft_consensus.cc:1685] finished
0709 13:19:00.024802 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:00.024995 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12649,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":5812,"raft.queue_time_us":60172,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":8716,"spinlock_wait_cycles":15615232,"thread_start_us":129201,"threads_started":1,"wal-append.queue_time_us":173772}
W20260709 13:19:00.031340 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.050323 15642 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:00.051524 15642 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:18:59.992970 17598 consensus_queue.cc:1059] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:18:59.989950 17625 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:00.030850 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:00.197659 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 146202us
I20260709 13:19:00.017330 16233 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 1.
W20260709 13:19:00.345512 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:00.237725 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.081959 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:00.353139 15677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 1650) took 1356 ms. Trace:
I20260709 13:19:00.249119 15600 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:00.354336 15677 rpcz_store.cc:276] 0709 13:18:58.996782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.518936 (+522154us) service_pool.cc:224] Handling call
0709 13:18:59.519293 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:59.644066 (+124773us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:59.644101 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:59.775255 (+131154us) log.cc:844] Serialized 26 byte log entry
0709 13:18:59.956913 (+181658us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:00.351017 (+394104us) log.cc:844] Serialized 14 byte log entry
0709 13:19:00.352819 (+  1802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.352901 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:00.352942 (+    41us) raft_consensus.cc:1685] finished
0709 13:19:00.352953 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:00.353081 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":723,"raft.queue_time_us":35096,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":35289,"thread_start_us":350624,"threads_started":3,"wal-append.queue_time_us":189536}
I20260709 13:19:00.356139 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:00.355773 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:00.357007 15677 raft_consensus.cc:2368] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3.
I20260709 13:19:00.081884 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:00.357064 15600 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 3: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:00.032692 17631 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:00.027484 17580 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:00.281565 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:00.336925 17642 log.cc:927] Time spent T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.346s	user 0.001s	sys 0.000s
I20260709 13:19:00.357702 16235 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
W20260709 13:19:00.338908 17652 log.cc:927] Time spent T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:19:00.052577 17586 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:00.106309 17581 log.cc:927] Time spent T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 1.540s	user 0.000s	sys 0.014s
I20260709 13:19:00.051132 17639 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:00.337546 15668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49094 (request call id 1627) took 2370 ms. Trace:
I20260709 13:19:00.210574 15521 raft_consensus.cc:1240] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:19:00.382010 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:00.063541 17612 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:00.383208 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:00.104099 17429 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:00.062628 17427 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:00.322541 16231 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:19:00.213608 17578 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:00.234925 17644 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:00.386320 17586 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:00.388528 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.054647 17629 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:19:00.392234 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.159758 17361 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:00.232191 17479 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:00.062618 17521 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:00.398638 17578 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:19:00.409711 17531 consensus_meta.cc:287] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.002s
I20260709 13:19:00.398356 17612 raft_consensus.cc:697] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
W20260709 13:19:00.410990 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.412698 15568 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
W20260709 13:19:00.427278 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 72734.2us
I20260709 13:19:00.362175 15594 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:19:00.342420 17307 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:00.434901 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.405588 17629 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.437953 16203 leader_election.cc:400] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3.
I20260709 13:19:00.448261 16203 leader_election.cc:403] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:19:00.449728 16235 raft_consensus.cc:1240] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:19:00.411682 17621 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.342666 17482 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:00.452721 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.436419 17479 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:00.519460 15677 raft_consensus.cc:1240] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:00.428517 17427 consensus_queue.cc:1059] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
W20260709 13:19:00.525045 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 179209us
I20260709 13:19:00.450639 17629 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:00.532907 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:00.538175 17459 consensus_meta.cc:287] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.649s	user 0.005s	sys 0.019s
I20260709 13:19:00.539927 17361 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:00.541023 16236 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:19:00.419670 17644 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:00.532755 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:00.532102 17479 raft_consensus.cc:697] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:00.547269 17219 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.573163 17219 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:00.575816 17219 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:00.550182 17610 consensus_meta.cc:287] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.760s	user 0.009s	sys 0.011s
W20260709 13:19:00.577250 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 148423us
I20260709 13:19:00.510699 17612 consensus_queue.cc:237] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:00.576781 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:00.498961 17621 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:00.384774 15668 rpcz_store.cc:276] 0709 13:18:57.968185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.979196 (+ 11011us) service_pool.cc:224] Handling call
0709 13:18:58.055818 (+ 76622us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:58.144135 (+ 88317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:58.144191 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:58.144904 (+   713us) log.cc:844] Serialized 26 byte log entry
0709 13:18:59.569443 (+1424539us) spinlock_profiling.cc:243] Waited 9.81 ms on lock 0x7b180046fb40. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab0d1 00007f5ce49c6a0f
0709 13:18:59.844828 (+275385us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:59.844875 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.844988 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:00.158248 (+313260us) raft_consensus.cc:1685] finished
0709 13:19:00.179957 (+ 21709us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:00.337454 (+157497us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3550,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":134,"spinlock_wait_cycles":22564096,"thread_start_us":270819,"threads_started":1,"wal-append.queue_time_us":1792652}
I20260709 13:19:00.592559 16233 raft_consensus.cc:2393] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
W20260709 13:19:00.596781 17659 log.cc:927] Time spent T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:19:00.597482 17307 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:00.600942 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:00.602116 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:00.624899 15600 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:19:00.606433 17651 log.cc:927] Time spent T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
W20260709 13:19:00.607182 17656 log.cc:927] Time spent T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:19:00.610212 17479 consensus_queue.cc:237] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:00.593964 17625 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:00.611872 17586 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:00.612208 15594 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:00.595278 17623 consensus_meta.cc:287] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.603s	user 0.005s	sys 0.005s
I20260709 13:19:00.619174 17644 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:00.592785 17662 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3.
W20260709 13:19:00.620805 15493 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.089s	user 0.014s	sys 0.013s
W20260709 13:19:00.647631 15493 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.126s	user 0.015s	sys 0.013s
I20260709 13:19:00.621260 15674 raft_consensus.cc:1240] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 3. Current term is 4. Ops: []
W20260709 13:19:00.619035 15673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57710 (request call id 417) took 1449 ms (client timeout 1533 ms). Trace:
I20260709 13:19:00.580608 17610 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:00.637246 15597 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:19:00.648931 15600 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:19:00.629419 17580 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:00.656908 15641 connection.cc:256] RPC call timeout handler was delayed by -1.397806545000094s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1505s for a response
W20260709 13:19:00.658473 15641 connection.cc:256] RPC call timeout handler was delayed by -1.225006545000042s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16970000000000002s for a response
W20260709 13:19:00.647814 17583 consensus_meta.cc:287] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.738s	user 0.008s	sys 0.002s
W20260709 13:19:00.658861 15641 connection.cc:256] RPC call timeout handler was delayed by -1.1764065449999634s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
I20260709 13:19:00.660481 15597 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:19:00.666059 15641 connection.cc:256] RPC call timeout handler was delayed by -1.1314065450001181s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1801s for a response
W20260709 13:19:00.666496 15641 connection.cc:256] RPC call timeout handler was delayed by -1.0603065450000031s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.188s for a response
W20260709 13:19:00.666827 15641 connection.cc:256] RPC call timeout handler was delayed by -1.0459065449999798s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1896s for a response
W20260709 13:19:00.667130 15641 connection.cc:256] RPC call timeout handler was delayed by -1.0351065449999624s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1908s for a response
W20260709 13:19:00.671622 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 1.15725e+06us
I20260709 13:19:00.672600 15493 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:00.659454 17586 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.659195 17307 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
W20260709 13:19:00.649050 15673 rpcz_store.cc:269] 0709 13:18:59.169429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.273666 (+104237us) service_pool.cc:224] Handling call
0709 13:19:00.618811 (+1345145us) spinlock_profiling.cc:243] Waited 523 ms on lock 0x7b54001a7a54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:00.618970 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1202406144}
I20260709 13:19:00.668032 17580 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:00.669761 17521 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.663455 17482 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:00.685626 17654 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.692870 17586 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:00.699913 15672 raft_consensus.cc:1240] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:00.698581 15493 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:00.693424 17307 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:00.697351 16203 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:00.707248 15493 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:00.708194 17633 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:00.646756 17623 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:00.704751 17644 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:00.708702 17429 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:00.708570 17307 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:00.696980 17343 raft_consensus.cc:3055] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:00.715230 17343 raft_consensus.cc:740] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:00.716089 17343 consensus_queue.cc:260] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:00.699074 17445 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:00.708923 17531 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:00.718422 17531 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:00.718781 17531 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:00.708572 17633 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:00.721521 17633 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:00.709885 17644 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:00.726430 17521 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:00.728408 17644 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:00.728665 17521 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:00.699913 17625 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:00.710557 17623 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:00.729033 17343 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:00.730011 15596 raft_consensus.cc:1275] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:00.729950 15493 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:00.732266 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:00.732913 15596 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:00.733908 17623 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:19:00.693712 17661 log.cc:927] Time spent T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:19:00.733544 17654 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:00.735447 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:00.733071 15493 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:00.744082 15675 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:19:00.746584 15493 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:00.749356 17489 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:00.749814 17489 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.750146 17489 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:00.750459 17586 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:00.752228 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:00.756310 15493 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:00.695561 17580 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:00.758541 17287 raft_consensus.cc:3055] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:00.759008 17287 raft_consensus.cc:740] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:00.759879 17287 consensus_queue.cc:260] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:00.760871 17287 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:00.761493 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.762526 17576 raft_consensus.cc:3055] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:00.762934 17576 raft_consensus.cc:740] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:00.763702 17576 consensus_queue.cc:260] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:00.764798 17576 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:00.766006 17489 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:00.766696 16238 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 381) took 2602 ms (client timeout 1908 ms). Trace:
I20260709 13:19:00.734395 17623 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:00.767843 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.768158 17580 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:00.778321 17623 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:00.776897 17445 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
W20260709 13:19:00.780514 17649 log.cc:927] Time spent T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.522s	user 0.003s	sys 0.000s
I20260709 13:19:00.782049 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:00.782223 17580 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:00.783160 17531 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:00.784417 17580 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:00.774925 17574 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:00.785224 17531 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:00.785537 15597 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:00.743888 17482 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:00.744510 17639 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:00.774209 16238 rpcz_store.cc:269] 0709 13:18:58.164370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.168309 (+  3939us) service_pool.cc:224] Handling call
0709 13:19:00.766449 (+2598140us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54007201d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:00.766618 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":591156608}
W20260709 13:19:00.790654 15641 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.505s (SENT) [suppressed 24 similar messages]
I20260709 13:19:00.769767 17429 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:00.748914 17633 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:00.797675 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:00.775805 15594 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:00.788704 17482 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:00.806663 15674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 1659) took 1307 ms. Trace:
I20260709 13:19:00.807179 17654 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:00.807794 17610 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:00.796144 17631 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:00.798204 17574 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:00.812220 17629 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:00.814018 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:00.815393 15641 leader_election.cc:336] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.697s (SENT)
I20260709 13:19:00.820569 15597 raft_consensus.cc:2410] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:19:00.822224 15674 rpcz_store.cc:276] 0709 13:18:59.499276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.769361 (+270085us) service_pool.cc:224] Handling call
0709 13:19:00.620006 (+850645us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:00.648254 (+ 28248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.806396 (+158142us) spinlock_profiling.cc:243] Waited 850 ms on lock 0x7b54001a7a50. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:00.806565 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1955513088}
I20260709 13:19:00.795363 17580 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:00.795028 17678 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:00.831663 15493 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:00.835929 17574 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:00.836755 17633 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:00.831629 17678 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.837790 17678 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:00.823472 17482 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:00.827783 17531 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:00.850072 15676 raft_consensus.cc:2393] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:00.845250 15595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57344 (request call id 18) took 1626 ms. Trace:
I20260709 13:19:00.850365 17476 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:00.852788 17476 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.853147 17476 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:00.853513 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.837774 17625 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:19:00.857506 15567 connection.cc:256] RPC call timeout handler was delayed by -1.2516232440000294s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30760000000000004s for a response
W20260709 13:19:00.863582 15567 connection.cc:256] RPC call timeout handler was delayed by -1.211123243999964s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31210000000000004s for a response
I20260709 13:19:00.825042 17639 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:00.859606 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.852131 15595 rpcz_store.cc:276] 0709 13:18:59.219326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.229397 (+ 10071us) service_pool.cc:224] Handling call
0709 13:19:00.839847 (+1610450us) spinlock_profiling.cc:243] Waited 24.8 ms on lock 0x7b540016b054. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a05853 00007f5ce99d360a 00007f5ce2f597d4 00007f5ce2f59762 00007f5ce2f59691 00007f5ce2f5960c 00007f5ce2f588c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:00.839959 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57061248}
I20260709 13:19:00.839375 17580 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:00.877665 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 49809.2us
I20260709 13:19:00.867970 17574 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:00.884765 17574 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:00.887080 17574 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:00.887650 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.890786 15670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49100 (request call id 1660) took 1152 ms. Trace:
I20260709 13:19:00.891218 15670 rpcz_store.cc:276] 0709 13:18:59.738515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.754381 (+ 15866us) service_pool.cc:224] Handling call
0709 13:19:00.890571 (+1136190us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b540042a254. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:00.890722 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":262487808}
I20260709 13:19:00.869082 17479 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:00.917876 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:00.919777 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.871867 17578 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:00.942421 17380 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.943063 17380 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:00.943984 17578 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:00.945403 17380 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:00.946624 17489 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:00.947639 17629 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:00.948695 17479 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:00.951066 17479 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:00.954305 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:00.954944 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:00.959347 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.956610 17574 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:00.990573 15595 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:19:00.992880 15595 raft_consensus.cc:606] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:00.993248 15595 raft_consensus.cc:614] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:00.991102 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:00.992441 17574 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.006829 17574 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:01.011823 17623 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.014468 17623 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:01.019976 15670 raft_consensus.cc:2410] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:19:01.020192 17631 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:00.998231 17489 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
W20260709 13:19:01.026543 17645 log.cc:927] Time spent T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.925s	user 0.006s	sys 0.009s
I20260709 13:19:01.030236 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:01.030901 15601 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 4
W20260709 13:19:01.034441 15603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53310 (request call id 1550) took 1857 ms (client timeout 2086 ms). Trace:
I20260709 13:19:01.035079 17570 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:19:01.034966 15603 rpcz_store.cc:269] 0709 13:18:59.177526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.177658 (+   132us) service_pool.cc:224] Handling call
0709 13:19:01.034208 (+1856550us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b5400389ad4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:01.034368 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":482867072}
I20260709 13:19:01.037273 17489 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.037814 17489 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.040512 17580 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:01.041201 17489 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:01.054131 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.056114 15601 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
W20260709 13:19:01.060731 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 50614.7us
I20260709 13:19:01.072023 17476 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:01.078214 17631 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.078816 17631 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:01.081116 17631 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:01.091233 17482 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.110508 17482 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:01.009929 17681 consensus_queue.cc:1048] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.133s
I20260709 13:19:01.089651 17678 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:01.143303 15493 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:01.144884 17681 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:01.145702 17681 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:01.146165 17681 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:19:01.146420 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.147298 17580 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.147792 17580 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.151283 15595 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:19:01.151919 15595 raft_consensus.cc:606] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Received request to transfer leadership to TS dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:01.153219 17576 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:01.153831 17576 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.156260 17631 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:01.156849 17631 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.157986 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:01.158936 15603 raft_consensus.cc:2410] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 3 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:19:01.165732 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:01.166666 15595 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:01.170009 15603 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:19:01.170782 15603 raft_consensus.cc:606] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:01.171875 17287 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.172446 17287 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:01.174685 17287 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:01.176671 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:01.141707 17476 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:01.187424 17631 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:01.189182 17678 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:01.200824 17678 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:01.201557 17678 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:01.210230 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:01.213977 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:01.214780 15601 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 4: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
W20260709 13:19:01.212865 17427 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.013s	sys 0.019s
I20260709 13:19:01.220484 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:01.221344 15601 raft_consensus.cc:2393] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:01.223045 15669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 1652) took 2116 ms. Trace:
I20260709 13:19:01.224123 15669 rpcz_store.cc:276] 0709 13:18:59.106740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.548590 (+441850us) service_pool.cc:224] Handling call
0709 13:18:59.578087 (+ 29497us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.603886 (+ 25799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:01.222850 (+1618964us) spinlock_profiling.cc:243] Waited 20.3 ms on lock 0x7b540009c4d0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:01.222986 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46796544}
I20260709 13:19:01.223073 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:01.225709 16237 raft_consensus.cc:2410] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:19:01.226192 16239 raft_consensus.cc:1240] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4. Ops: []
W20260709 13:19:01.218369 17427 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.013s	sys 0.019s
I20260709 13:19:01.251461 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:01.252193 16238 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 1.
I20260709 13:19:01.256407 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:01.257682 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:01.261428 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.262845 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:01.263618 15601 raft_consensus.cc:2393] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:01.266218 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:19:01.270310 15675 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:19:01.270736 15675 pending_rounds.cc:85] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting all ops after (but not including) 5
I20260709 13:19:01.270983 15675 pending_rounds.cc:107] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:19:01.271209 15675 raft_consensus.cc:2889] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:01.271462 17568 consensus_queue.cc:1059] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:01.271813 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:01.273595 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 265646us
W20260709 13:19:01.274914 15567 leader_election.cc:336] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.738s (SENT)
I20260709 13:19:01.274830 15674 raft_consensus.cc:1240] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:01.276355 15567 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:01.282927 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:01.286126 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:01.287711 15674 raft_consensus.cc:1275] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:19:01.288959 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:19:01.290160 15567 leader_election.cc:336] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.738s (SENT)
I20260709 13:19:01.290771 15567 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:01.291456 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:01.293543 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:01.294384 15603 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 3: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:01.298319 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:01.300279 15674 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:19:01.303599 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:01.275629 17343 consensus_queue.cc:1059] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:19:01.287513 17568 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:01.308005 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:01.308519 16235 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:19:01.310392 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:01.293901 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:01.314749 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:01.315632 15493 connection.cc:256] RPC call timeout handler was delayed by -2.8285099820002415s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
I20260709 13:19:01.296245 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:01.314843 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:01.318008 15603 raft_consensus.cc:2393] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 3: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:01.294516 17631 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:19:01.305663 17621 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:01.322005 15493 connection.cc:256] RPC call timeout handler was delayed by -2.6867285900000297s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31360000000000005s for a response
I20260709 13:19:01.317236 17476 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.325748 15674 raft_consensus.cc:1275] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:19:01.323603 15493 connection.cc:256] RPC call timeout handler was delayed by -2.3273014580001927s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2171s for a response
I20260709 13:19:01.312563 17572 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:19:01.327114 15493 connection.cc:256] RPC call timeout handler was delayed by -2.3161574129999281s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1928s for a response
W20260709 13:19:01.327500 15493 connection.cc:256] RPC call timeout handler was delayed by -2.2530834320002668s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1955s for a response
W20260709 13:19:01.327756 15493 connection.cc:256] RPC call timeout handler was delayed by -2.19055133400002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1773s for a response
I20260709 13:19:01.316303 17678 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:01.328058 15493 connection.cc:256] RPC call timeout handler was delayed by -1.995204054000169s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15760000000000002s for a response
I20260709 13:19:01.318310 15673 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:01.329090 15493 connection.cc:256] RPC call timeout handler was delayed by -1.7718998670002293s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.197s for a response
W20260709 13:19:01.329632 15493 connection.cc:256] RPC call timeout handler was delayed by -1.6766618500000732s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1874s for a response
W20260709 13:19:01.329962 15493 connection.cc:256] RPC call timeout handler was delayed by -1.5470040540001264s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2074s for a response
I20260709 13:19:01.329838 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:19:01.330511 15602 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53310 (request call id 1546) took 2433 ms (client timeout 1582 ms). Trace:
W20260709 13:19:01.330924 15602 rpcz_store.cc:269] 0709 13:18:58.893627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.909316 (+ 15689us) service_pool.cc:224] Handling call
0709 13:19:01.327120 (+2417804us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b540043b054. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:01.327244 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":617656064}
I20260709 13:19:01.331809 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:01.332239 15602 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:01.336588 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:01.333885 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:01.322832 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:01.322832 17482 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:01.339882 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:01.340018 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 49897.2us
I20260709 13:19:01.326924 17621 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:01.348102 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:01.348613 15603 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:01.350977 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.335176 17476 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:01.363535 17681 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.365824 17681 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:01.366158 16235 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
W20260709 13:19:01.374202 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.342079 17482 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.381532 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:01.343475 17621 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:01.410072 17681 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:01.380873 17687 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:19:01.415449 15602 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:19:01.411028 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:01.412191 17326 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.412890 17427 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:01.413594 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.374003 17593 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.418313 15603 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:19:01.419148 17681 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.429761 17681 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:01.434046 17593 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:01.435186 17673 log.cc:927] Time spent T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.736s	user 0.004s	sys 0.003s
I20260709 13:19:01.436345 17593 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:01.437566 17476 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:01.437803 17326 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.439087 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:01.420468 17678 consensus_queue.cc:1059] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:01.420627 17489 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:01.421591 15493 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:01.422104 17623 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:01.415741 17380 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:01.425122 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.425482 17427 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.459664 17427 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:01.461004 17380 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.463231 17380 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:01.427984 17571 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:01.463863 17571 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.439940 17326 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:01.420323 17482 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:01.448130 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:01.452332 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:01.453786 15673 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:01.454330 15671 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:19:01.456153 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.456457 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:01.456836 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:01.487872 15602 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 4: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:01.461359 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:01.489039 15601 raft_consensus.cc:2393] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:01.441478 17621 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:01.466650 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:01.475656 17224 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:01.476025 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:01.493489 15597 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:01.477789 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:01.480939 15672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 411) took 3015 ms. Trace:
I20260709 13:19:01.483584 17580 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:01.485314 17578 raft_consensus.cc:3055] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:19:01.486050 16236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 395) took 1493 ms (client timeout 1673 ms). Trace:
I20260709 13:19:01.467952 17623 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:01.515939 16236 rpcz_store.cc:269] 0709 13:18:59.992330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.992554 (+   224us) service_pool.cc:224] Handling call
0709 13:19:01.485858 (+1493304us) spinlock_profiling.cc:243] Waited 533 ms on lock 0x7b5400491bd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:01.485991 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1226346368}
I20260709 13:19:01.490774 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:01.536166 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:01.536746 15601 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:01.491333 15603 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:01.491444 16235 raft_consensus.cc:1240] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:01.492046 17476 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:19:01.497843 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.502305 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:01.503158 15599 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53310 (request call id 1559) took 1514 ms (client timeout 1757 ms). Trace:
W20260709 13:19:01.503803 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.504715 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:01.543260 15600 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:01.507563 17212 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:01.475507 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:01.508391 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:01.508701 17572 consensus_queue.cc:1059] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:01.511315 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:01.512148 15669 raft_consensus.cc:1240] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:01.509301 17687 consensus_queue.cc:1048] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:19:01.474205 17593 consensus_queue.cc:1059] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:19:01.513861 15597 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:01.508396 15566 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:19:01.488946 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 67459.7us
I20260709 13:19:01.495030 17353 tablet_bootstrap.cc:492] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:01.503075 17576 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:01.567538 17630 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:01.546759 17221 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.551908 15601 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:19:01.509333 16231 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:01.588279 16205 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:01.588848 15675 raft_consensus.cc:2393] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:01.539232 17571 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:01.596067 17630 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:01.596431 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:01.546597 17670 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:01.597142 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:01.597822 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:01.594916 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:01.599318 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:01.598202 15493 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:01.601341 15603 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:01.516567 17578 raft_consensus.cc:740] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:01.602102 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:01.603825 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 95513us
I20260709 13:19:01.541119 17476 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:01.607092 15602 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:01.609982 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:01.611229 15566 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:01.595515 15598 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:01.614776 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:01.615379 16240 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:19:01.604671 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:01.614251 17476 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:01.624269 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:01.625102 15669 raft_consensus.cc:2393] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:01.612162 15671 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:01.634270 15673 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 4: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:01.638266 17681 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:01.638753 17681 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:01.639108 17681 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:01.639513 17574 raft_consensus.cc:3055] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:01.640022 17574 raft_consensus.cc:740] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:01.640302 17476 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.651465 16240 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:19:01.613253 17572 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:01.509840 15672 rpcz_store.cc:276] 0709 13:18:58.465811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.467999 (+  2188us) service_pool.cc:224] Handling call
0709 13:18:59.782056 (+1314057us) spinlock_profiling.cc:243] Waited 3.89 ms on lock 0x7b5800012398. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99e4fc0 00007f5ce99cf929 00007f5ce99cf3d2 00007f5ce2f46bd4
0709 13:18:59.782327 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:00.706339 (+924012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:01.480656 (+774317us) spinlock_profiling.cc:243] Waited 904 ms on lock 0x7b540046df54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:01.480866 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2087002752}
I20260709 13:19:01.658815 15493 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:01.546113 15599 rpcz_store.cc:269] 0709 13:18:59.989217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.028014 (+ 38797us) service_pool.cc:224] Handling call
0709 13:19:01.449828 (+1421814us) spinlock_profiling.cc:243] Waited 4.89 ms on lock 0x7b5800113498. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99d10a0 00007f5ce99d0984 00007f5ce2f4b6d4 00007f5ce2f4b662
0709 13:19:01.503083 (+ 53255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11253120}
I20260709 13:19:01.616037 17594 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:01.618710 17580 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:01.612550 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:01.667863 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.670950 15600 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:19:01.602792 17221 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.673761 15675 raft_consensus.cc:1240] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:01.675254 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:01.649210 17476 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:01.676792 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:01.677621 15600 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:19:01.670290 17353 tablet_bootstrap.cc:492] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9: Bootstrap complete.
I20260709 13:19:01.616794 17576 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:19:01.681375 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:01.680510 15600 raft_consensus.cc:1240] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:19:01.680013 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 79961.7us
I20260709 13:19:01.681957 15597 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:01.664386 17572 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.621447 17687 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 13:19:01.666173 17621 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.685343 15600 raft_consensus.cc:1240] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:01.617630 17670 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.680866 17353 ts_tablet_manager.cc:1403] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9: Time spent bootstrapping tablet: real 1.798s	user 0.046s	sys 0.040s
I20260709 13:19:01.666306 15424 catalog_manager.cc:5697] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 3 to 4, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 4 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:01.689745 15600 raft_consensus.cc:1240] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:01.692648 16204 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:01.704927 16204 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:01.705309 15493 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:01.681181 17580 raft_consensus.cc:697] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:01.686564 17572 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:01.683928 17688 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:01.717640 17227 raft_consensus.cc:3055] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:01.718240 17227 raft_consensus.cc:740] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:01.719148 17227 consensus_queue.cc:260] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:01.725543 17227 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:01.709270 17224 raft_consensus.cc:2764] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:19:01.709110 17326 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:01.722587 17623 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:01.723264 15677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49094 (request call id 1643) took 1821 ms. Trace:
I20260709 13:19:01.730232 15677 rpcz_store.cc:276] 0709 13:18:59.901514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.390745 (+489231us) service_pool.cc:224] Handling call
0709 13:19:00.391044 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:00.520146 (+129102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:01.722981 (+1202835us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b5400186d54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:01.723190 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":249548032}
I20260709 13:19:01.723315 17482 raft_consensus.cc:3055] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:01.734817 17482 raft_consensus.cc:740] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:01.735834 17482 consensus_queue.cc:260] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.737087 17482 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:01.737574 17580 consensus_queue.cc:237] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:01.738838 15677 raft_consensus.cc:1240] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:01.739567 15675 raft_consensus.cc:1240] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:01.724790 15595 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:19:01.669541 17654 consensus_queue.cc:1059] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:19:01.740279 17630 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:01.741523 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:01.742312 15672 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:19:01.743997 17578 consensus_queue.cc:260] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:01.748844 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:01.750283 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:01.752596 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:01.689495 17582 consensus_queue.cc:1048] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:19:01.702368 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:01.780424 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:01.787652 16205 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:01.788692 15676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 1634) took 1935 ms. Trace:
I20260709 13:19:01.789299 15676 rpcz_store.cc:276] 0709 13:18:59.852984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.902831 (+ 49847us) service_pool.cc:224] Handling call
0709 13:19:01.788469 (+1885638us) spinlock_profiling.cc:243] Waited 931 ms on lock 0x7b540009b5d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:01.788619 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2141165952}
W20260709 13:19:01.789889 15567 leader_election.cc:336] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.182s (SENT)
I20260709 13:19:01.790426 15567 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:01.791229 17569 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:01.781749 17669 raft_consensus.cc:3055] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:01.791913 17669 raft_consensus.cc:740] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:01.792801 17669 consensus_queue.cc:260] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.794960 17593 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:01.773542 17343 consensus_queue.cc:1059] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:19:01.808877 17574 consensus_queue.cc:260] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.797540 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:01.707962 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:01.838100 17623 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:19:01.846339 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:01.862092 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.888972 17343 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:01.889810 17343 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.893546 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:01.713490 15424 catalog_manager.cc:5697] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 1 to 2, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 2 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
W20260709 13:19:01.914335 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 209067us
I20260709 13:19:01.915202 15493 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:01.916018 17476 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:01.916592 17476 raft_consensus.cc:697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:01.917562 17476 consensus_queue.cc:237] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.915438 15424 catalog_manager.cc:5697] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 3 to 4, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 4 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:01.919690 15424 catalog_manager.cc:5697] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 3 to 4, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 4 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:01.903473 17578 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:01.904561 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.928157 17353 raft_consensus.cc:359] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:01.929116 17353 raft_consensus.cc:740] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Initialized, Role: LEARNER
I20260709 13:19:01.929203 15603 raft_consensus.cc:1240] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:01.929837 17353 consensus_queue.cc:260] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:01.931160 15603 raft_consensus.cc:1240] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:01.932112 17353 ts_tablet_manager.cc:1434] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9: Time spent starting tablet: real 0.228s	user 0.003s	sys 0.006s
W20260709 13:19:01.906260 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 201396us
I20260709 13:19:01.907334 17670 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:01.933565 16204 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:01.933508 17571 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:01.935232 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:01.933801 17355 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:19:01.935470 17582 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:01.935267 17571 raft_consensus.cc:2804] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:01.933888 15424 catalog_manager.cc:5697] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 1 to 2, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 2 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:01.937160 16204 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:01.937754 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:01.937122 17571 raft_consensus.cc:697] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:01.938879 17571 consensus_queue.cc:237] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.942368 17670 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:01.944514 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:01.945400 15675 raft_consensus.cc:2368] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4.
I20260709 13:19:01.945978 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:01.944945 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:01.946748 15603 raft_consensus.cc:2410] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:01.947070 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:01.947775 15675 raft_consensus.cc:2393] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 3: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:01.948096 15603 raft_consensus.cc:1240] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:01.948704 15642 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:01.943542 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:01.950285 15676 raft_consensus.cc:2393] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 5: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:01.951618 15671 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
W20260709 13:19:01.952021 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 162179us
I20260709 13:19:01.952193 17670 consensus_queue.cc:1059] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:01.953095 17593 raft_consensus.cc:3055] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:01.952666 15424 catalog_manager.cc:5697] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 1 to 2, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 2 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:01.922290 17574 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:01.922993 17221 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:01.961392 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:01.963251 16235 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEARNER]: Advancing to term 4
I20260709 13:19:01.965679 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:01.966588 15520 raft_consensus.cc:2410] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:01.966652 17343 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:01.967151 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:01.967818 15521 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:19:01.969841 16235 raft_consensus.cc:1275] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEARNER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:19:01.970057 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:01.970685 15522 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:19:01.971426 17604 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.971796 15521 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:01.970574 15519 raft_consensus.cc:1240] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:19:01.971987 17604 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.974100 17604 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:01.976589 15597 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:19:01.979645 16238 raft_consensus.cc:1240] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:01.952616 17669 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:01.951758 17681 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.985795 17681 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:01.986155 17630 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:01.986707 17630 raft_consensus.cc:697] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:01.987886 17692 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.987605 17630 consensus_queue.cc:237] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:01.945447 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:01.988441 17692 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.953469 17593 raft_consensus.cc:740] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:01.990300 17593 consensus_queue.cc:260] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:01.921820 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:01.910670 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 123120us
W20260709 13:19:01.997596 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 47862.6us
W20260709 13:19:01.955475 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.902491 17621 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:01.957288 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:01.922895 16232 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:19:01.958283 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:01.960415 16231 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:01.905810 17426 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:01.990725 17692 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:01.955314 15567 leader_election.cc:336] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.292s (SENT)
W20260709 13:19:02.008648 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.029381 15595 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: []
W20260709 13:19:02.045394 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 90102.3us
W20260709 13:19:02.029448 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 46964.4us
W20260709 13:19:02.046092 15567 leader_election.cc:336] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.301s (SENT)
I20260709 13:19:02.046844 15568 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:02.047914 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:02.048574 15595 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
W20260709 13:19:01.958986 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 43893.4us
I20260709 13:19:02.033520 16205 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:02.041436 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:01.991544 17593 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:02.043663 15603 raft_consensus.cc:1240] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:02.052821 15597 raft_consensus.cc:1240] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:02.031894 17662 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.053977 17227 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:02.056730 15676 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:02.055011 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:02.054798 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:02.043277 15424 catalog_manager.cc:5697] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 3 to 4, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 4 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:02.074877 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:02.077090 15568 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:02.056313 17227 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:02.088595 15493 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:02.090404 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:02.091234 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:02.094413 15677 raft_consensus.cc:2393] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 5: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:02.091518 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:02.095664 16238 raft_consensus.cc:2393] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
W20260709 13:19:02.096985 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 98506.6us
I20260709 13:19:02.101884 15493 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:02.056557 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:02.099541 16205 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:02.043977 17660 consensus_queue.cc:1059] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:19:02.063097 17670 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:02.108500 15642 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:02.121429 15493 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:02.123282 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:02.114542 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:02.127562 16239 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:02.128096 16238 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:02.090183 17687 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:02.097318 17227 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:02.134675 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:02.073829 17684 consensus_queue.cc:1059] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:19:02.138823 16239 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:02.104050 17681 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:01.909824 17572 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:02.179109 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:02.109519 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:02.114637 17623 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:02.089964 17576 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:19:02.103267 17427 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:19:02.087785 17582 consensus_queue.cc:1059] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:19:02.183209 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:02.193949 16239 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:19:02.133577 17429 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:02.108553 17570 consensus_queue.cc:1059] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:19:02.118945 17670 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:02.186460 15524 raft_consensus.cc:2376] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2. Re-sending same reply.
I20260709 13:19:02.188067 16203 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:02.205354 15600 raft_consensus.cc:1240] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:02.098913 17578 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
I20260709 13:19:02.137598 17482 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:02.191058 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:02.191347 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:02.135766 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:02.193217 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:02.129442 17326 consensus_queue.cc:1059] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:19:02.198014 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:02.120146 17594 consensus_queue.cc:1059] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:19:02.137719 17224 consensus_queue.cc:1059] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:19:02.208954 15675 raft_consensus.cc:2393] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:02.108407 17688 consensus_queue.cc:1059] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.085s
I20260709 13:19:02.208971 15677 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:19:02.212399 15527 raft_consensus.cc:2393] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:02.131803 15594 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:19:02.213336 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:02.135442 17380 consensus_queue.cc:1059] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:19:02.193694 15687 tablet_copy_service.cc:342] P 3ef186ec12554a1d87814af0a0ace9a4: Request end of tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-bd36c2cfd99c45cea9e48096d54bed1a received from {username='slave'} at 127.0.0.1:57710
I20260709 13:19:02.224205 16239 raft_consensus.cc:2393] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 3: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:02.228298 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:02.201969 17429 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.062216 17662 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:02.209677 17670 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:02.388324 16236 raft_consensus.cc:2393] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 3: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:02.332612 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:02.394265 15524 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:19:02.336247 16240 raft_consensus.cc:2410] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:02.357105 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:02.381039 16204 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:02.387962 15528 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:02.301365 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:19:02.377054 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.396236 16235 raft_consensus.cc:2393] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 3: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:02.335951 16234 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 1 LEARNER]: Advancing to term 3
W20260709 13:19:02.321801 15641 leader_election.cc:336] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.505s (SENT)
W20260709 13:19:02.397302 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 75535.2us
I20260709 13:19:02.296517 15523 raft_consensus.cc:1240] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:02.286202 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:02.372716 16238 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:02.379298 15674 raft_consensus.cc:1275] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:19:02.392761 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:02.343221 17604 raft_consensus.cc:3055] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:02.227746 17687 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.485170 17687 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:02.486215 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:02.406520 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:02.486925 15519 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:19:02.406520 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:02.487574 16236 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:02.436092 15527 raft_consensus.cc:2368] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 3. Current term is 4.
I20260709 13:19:02.488449 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:02.488816 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:02.213863 17684 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.256s
I20260709 13:19:02.488974 17687 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:02.443099 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.489698 17687 consensus_queue.cc:1048] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:02.493541 17604 raft_consensus.cc:740] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:19:02.494546 17702 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:19:02.334481 15598 consensus_meta.cc:287] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.711s	user 0.010s	sys 0.005s
W20260709 13:19:02.444864 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.358321 17510 raft_consensus.cc:3055] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:02.323086 17621 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:02.392560 17429 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:02.227751 16233 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:19:02.354943 17688 raft_consensus.cc:3055] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:02.334662 17221 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:02.346499 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:02.346695 17482 raft_consensus.cc:697] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:02.430653 17580 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 13:19:02.224085 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:02.479993 15526 raft_consensus.cc:2368] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 3. Current term is 4.
W20260709 13:19:02.480218 17574 consensus_meta.cc:287] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.518s	user 0.009s	sys 0.000s
I20260709 13:19:02.481964 17716 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:02.482352 15596 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:19:02.558779 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:02.563699 15523 raft_consensus.cc:2368] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 3. Current term is 4.
I20260709 13:19:02.403662 15687 tablet_copy_service.cc:434] P 3ef186ec12554a1d87814af0a0ace9a4: ending tablet copy session dd1a14c3173d49dc9b55bb116e3783c9-bd36c2cfd99c45cea9e48096d54bed1a on tablet bd36c2cfd99c45cea9e48096d54bed1a with peer dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:02.487860 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:02.354640 16232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57244 (request call id 808) took 3145 ms (client timeout 2058 ms). Trace:
I20260709 13:19:02.497079 15527 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:02.489037 15524 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
W20260709 13:19:02.585220 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.585779 15519 raft_consensus.cc:2393] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:19:02.498159 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 85267.7us
I20260709 13:19:02.496924 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:02.504518 15528 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:19:02.586655 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:02.587357 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:02.587397 15523 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
W20260709 13:19:02.587729 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 206859us
W20260709 13:19:02.504881 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 396444us
I20260709 13:19:02.321588 17715 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.178s
I20260709 13:19:02.525136 17688 raft_consensus.cc:740] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
W20260709 13:19:02.423271 15641 leader_election.cc:336] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.530s (SENT)
W20260709 13:19:02.529279 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.532274 15568 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
W20260709 13:19:02.594687 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 194086us
W20260709 13:19:02.595638 15641 leader_election.cc:336] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.582s (SENT)
I20260709 13:19:02.595907 15641 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:02.533663 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 306852us
I20260709 13:19:02.541617 16236 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:02.506357 17510 raft_consensus.cc:740] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:02.596808 16203 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:02.515028 17426 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.606748 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:02.588436 16204 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:02.523963 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:02.607564 15676 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:02.591219 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:02.609298 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:02.609961 15523 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:02.610769 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:02.607571 15524 raft_consensus.cc:2393] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:02.613385 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:02.614095 15528 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:19:02.558548 17569 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.604980 16237 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:19:02.620774 16237 pending_rounds.cc:85] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9: Aborting all ops after (but not including) 5
I20260709 13:19:02.621037 16237 pending_rounds.cc:107] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:19:02.621254 16237 raft_consensus.cc:2889] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:02.605223 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:02.605415 15527 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:02.589684 15598 raft_consensus.cc:2410] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:19:02.617513 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:02.611279 15523 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:02.665189 15602 raft_consensus.cc:1240] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:02.665419 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:02.601052 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:02.624253 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:02.625190 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:02.592871 17572 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:02.615548 15525 raft_consensus.cc:2393] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:02.661499 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:02.664033 15526 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:19:02.676553 15519 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:19:02.584509 17718 consensus_queue.cc:1059] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.110s
I20260709 13:19:02.494594 17604 consensus_queue.cc:260] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:02.708343 15596 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53310 (request call id 1570) took 1956 ms (client timeout 1533 ms). Trace:
I20260709 13:19:02.666595 15523 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:02.709226 15596 rpcz_store.cc:269] 0709 13:19:00.751778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.751944 (+   166us) service_pool.cc:224] Handling call
0709 13:19:02.708144 (+1956200us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b540025ab54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:02.708262 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3977366400}
W20260709 13:19:02.682112 17714 log.cc:927] Time spent T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:19:02.712023 15520 raft_consensus.cc:2393] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 3: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:02.718669 15527 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
W20260709 13:19:02.584324 16232 rpcz_store.cc:269] 0709 13:18:59.208914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.222826 (+ 13912us) service_pool.cc:224] Handling call
0709 13:19:02.354400 (+3131574us) spinlock_profiling.cc:243] Waited 2.42 s on lock 0x7b5400514d54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:02.354550 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5573427584}
I20260709 13:19:02.717275 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:02.665964 15598 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53324 (request call id 1600) took 1208 ms (client timeout 1562 ms). Trace:
I20260709 13:19:02.620185 17426 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:02.729815 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:02.666846 17728 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:02.735025 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:02.730868 15520 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:02.735960 15519 raft_consensus.cc:2368] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 3. Current term is 4.
W20260709 13:19:02.737689 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 205450us
I20260709 13:19:02.739262 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:02.740005 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:19:02.741117 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 152305us
I20260709 13:19:02.740510 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:02.742863 15520 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:19:02.647398 17569 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:02.728914 17687 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:02.736663 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:02.739410 17727 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.772507 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:02.899304 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:02.740387 17604 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:02.913596 15519 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:02.695791 17482 consensus_queue.cc:237] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:02.703299 17688 consensus_queue.cc:260] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:02.715018 17510 consensus_queue.cc:260] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:02.731519 15598 rpcz_store.cc:269] 0709 13:19:01.457639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.475214 (+ 17575us) service_pool.cc:224] Handling call
0709 13:19:02.665874 (+1190660us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:03.136307 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.189518 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:03.212424 15528 raft_consensus.cc:2393] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:03.213363 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:03.213949 15528 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:02.936544 17739 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.215049 17739 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:03.192653 16204 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.006s	sys 0.010s
W20260709 13:19:03.216591 16204 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.007s	sys 0.010s
W20260709 13:19:03.216921 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 628532us
I20260709 13:19:03.217628 16204 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:03.184203 16205 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:03.164250 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 149752us
W20260709 13:19:03.189699 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 594048us
I20260709 13:19:02.938061 17725 raft_consensus.cc:2764] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:19:03.064989 17688 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:02.917742 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 145312us
I20260709 13:19:03.227416 15642 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:19:03.014595 17703 log.cc:927] Time spent T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.457s	user 0.001s	sys 0.000s
W20260709 13:19:03.063097 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:03.201508 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 495853us
W20260709 13:19:03.104436 17699 log.cc:927] Time spent T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.581s	user 0.000s	sys 0.001s
I20260709 13:19:03.004681 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:19:03.029101 16234 consensus_meta.cc:287] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.628s	user 0.008s	sys 0.002s
I20260709 13:19:03.238157 16234 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEARNER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:19:03.014235 17570 raft_consensus.cc:3055] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:02.771404 17687 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:19:03.218997 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:03.246177 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:03.220211 15641 leader_election.cc:336] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.660s (SENT)
W20260709 13:19:03.183359 17720 log.cc:927] Time spent T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.565s	user 0.000s	sys 0.001s
W20260709 13:19:03.182646 17712 log.cc:927] Time spent T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.577s	user 0.000s	sys 0.001s
W20260709 13:19:03.182192 17707 log.cc:927] Time spent T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.474s	user 0.001s	sys 0.000s
I20260709 13:19:02.774370 17717 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:19:02.922721 17593 consensus_meta.cc:287] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.860s	user 0.007s	sys 0.003s
I20260709 13:19:02.914742 17733 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:03.239780 15528 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:03.219242 16204 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:03.020555 17716 raft_consensus.cc:697] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:02.898439 17630 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.328s
W20260709 13:19:03.278931 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 59778us
I20260709 13:19:03.255089 16234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53976 (request call id 412) took 1119 ms. Trace:
I20260709 13:19:03.245512 17570 raft_consensus.cc:740] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:03.253764 17687 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:03.284052 16204 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:03.280925 16234 rpcz_store.cc:276] 0709 13:19:02.119831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.119974 (+   143us) service_pool.cc:224] Handling call
0709 13:19:02.303121 (+183147us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b580009ff98. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99e4fc0 00007f5ce99cf929 00007f5ce99cf3d2 00007f5ce2f46bd4
0709 13:19:02.335823 (+ 32702us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:03.238548 (+902725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:03.238778 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28303232}
I20260709 13:19:02.938319 17727 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:03.247905 15642 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:03.275239 17744 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:03.225011 17594 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:03.232362 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:03.235988 17734 raft_consensus.cc:2804] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:03.264431 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:03.292615 17728 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:02.936460 17426 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:03.316054 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:03.287951 17380 consensus_queue.cc:1059] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:19:03.322515 17745 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:19:03.325654 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 77963.1us
I20260709 13:19:03.286900 17593 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:19:03.330574 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 729534us
I20260709 13:19:03.377213 15670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 1683) took 1612 ms. Trace:
I20260709 13:19:03.328464 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:03.245707 17715 consensus_queue.cc:1048] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
W20260709 13:19:03.391569 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 143525us
I20260709 13:19:03.393060 15528 raft_consensus.cc:2393] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:03.479688 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:03.481768 15676 consensus_meta.cc:287] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.874s	user 0.008s	sys 0.002s
I20260709 13:19:03.482257 15676 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:19:03.483332 15566 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:03.320407 17744 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:03.390364 16231 raft_consensus.cc:2410] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:19:03.464772 15526 raft_consensus.cc:2376] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5. Re-sending same reply.
I20260709 13:19:03.288640 17726 consensus_queue.cc:1059] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.105s
I20260709 13:19:03.507804 17426 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:03.463635 15670 rpcz_store.cc:276] 0709 13:19:01.764818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.765010 (+   192us) service_pool.cc:224] Handling call
0709 13:19:01.771497 (+  6487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:01.771878 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:01.771907 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:01.772423 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 13:19:02.099930 (+327507us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:02.099970 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.100034 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:03.376978 (+1276944us) raft_consensus.cc:1685] finished
0709 13:19:03.376999 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:03.377140 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1151,"raft.run_cpu_time_us":5591,"raft.run_wall_time_us":166502,"thread_start_us":327059,"threads_started":1,"wal-append.queue_time_us":664909}
I20260709 13:19:03.387300 17717 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:19:03.391045 17712 log.cc:927] Time spent T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 13:19:03.486900 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.464879 15673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49094 (request call id 1658) took 1452 ms. Trace:
I20260709 13:19:03.379307 17734 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:03.393978 17227 consensus_meta.cc:287] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 1.217s	user 0.004s	sys 0.007s
I20260709 13:19:03.315526 17725 consensus_queue.cc:1059] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.165s
I20260709 13:19:03.386543 17742 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.534860 15669 raft_consensus.cc:1275] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:03.507826 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1526) took 1075 ms. Trace:
W20260709 13:19:03.530496 15641 leader_election.cc:336] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.686s (SENT)
I20260709 13:19:03.465207 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:03.385721 15521 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 2.
I20260709 13:19:03.843837 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:03.845348 15528 raft_consensus.cc:2376] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3. Re-sending same reply.
I20260709 13:19:03.846045 15641 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:03.847663 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 317190us
W20260709 13:19:03.848439 15641 leader_election.cc:336] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.718s (SENT)
I20260709 13:19:03.848915 15641 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:03.720541 15519 consensus_meta.cc:287] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.806s	user 0.009s	sys 0.000s
I20260709 13:19:03.850375 15519 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:03.849999 15526 rpcz_store.cc:276] 0709 13:19:02.433142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.706274 (+273132us) service_pool.cc:224] Handling call
0709 13:19:03.328286 (+622012us) spinlock_profiling.cc:243] Waited 5.82 ms on lock 0x7b5800003698. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99d10a0 00007f5ce99d0984 00007f5ce2f4b6d4 00007f5ce2f4b662
0709 13:19:03.507748 (+179462us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13393664}
I20260709 13:19:03.850898 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1553) took 1355 ms. Trace:
W20260709 13:19:03.593777 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:03.577730 15523 consensus_meta.cc:287] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.868s	user 0.009s	sys 0.000s
I20260709 13:19:03.539906 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:03.814824 16237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53976 (request call id 407) took 1756 ms. Trace:
I20260709 13:19:03.462990 15599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 1602) took 1936 ms. Trace:
I20260709 13:19:03.508872 17744 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:03.852664 15523 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:03.851696 15519 rpcz_store.cc:276] 0709 13:19:02.495974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.740682 (+244708us) service_pool.cc:224] Handling call
0709 13:19:03.850846 (+1110164us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:03.864506 15523 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1528) took 1377 ms (client timeout 1805 ms). Trace:
W20260709 13:19:03.865018 15523 rpcz_store.cc:269] 0709 13:19:02.487358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.665181 (+177823us) service_pool.cc:224] Handling call
0709 13:19:03.864319 (+1199138us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:03.859496 15492 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.012s	sys 0.015s
I20260709 13:19:03.463491 17593 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:19:03.866195 15492 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.013s	sys 0.015s
I20260709 13:19:03.853827 16237 rpcz_store.cc:276] 0709 13:19:02.058577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.058691 (+   114us) service_pool.cc:224] Handling call
0709 13:19:02.598553 (+539862us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:02.621680 (+ 23127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:03.814545 (+1192865us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b5400496950. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:03.814741 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1237690112}
I20260709 13:19:03.511827 17745 raft_consensus.cc:697] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:03.489705 17733 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:03.841689 17750 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.362s	user 0.000s	sys 0.001s
W20260709 13:19:03.842406 17738 log.cc:927] Time spent T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.462s	user 0.000s	sys 0.000s
W20260709 13:19:03.835008 17723 log.cc:927] Time spent T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.622s	user 0.000s	sys 0.001s
W20260709 13:19:03.868876 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.866551 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:03.869433 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.869237 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:03.877177 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:19:03.869248 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 328521us
W20260709 13:19:03.804522 17572 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.000s	sys 0.010s
I20260709 13:19:03.584326 17716 consensus_queue.cc:237] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:03.518134 17570 consensus_queue.cc:260] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:04.251472 17687 consensus_meta.cc:287] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.945s	user 0.002s	sys 0.006s
I20260709 13:19:04.274497 17765 consensus_queue.cc:1059] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:19:03.578661 17717 raft_consensus.cc:697] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:04.278191 17727 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:04.258579 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:03.641148 17732 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.303574 15519 raft_consensus.cc:2410] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:19:04.146147 15677 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.858s	user 0.008s	sys 0.021s
I20260709 13:19:03.717485 15671 raft_consensus.cc:1240] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:19:03.880188 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:03.536477 17728 raft_consensus.cc:3055] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:04.277081 17684 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:04.305090 15526 raft_consensus.cc:2376] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3. Re-sending same reply.
I20260709 13:19:03.879395 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 438) took 1906 ms. Trace:
W20260709 13:19:04.301214 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 452786us
I20260709 13:19:04.348471 17684 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.349262 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:04.348615 15522 rpcz_store.cc:276] 0709 13:19:01.973233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.973357 (+   124us) service_pool.cc:224] Handling call
0709 13:19:01.973671 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:01.974021 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:01.974051 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:01.974486 (+   435us) log.cc:844] Serialized 26 byte log entry
0709 13:19:02.186537 (+212051us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:02.216922 (+ 30385us) log.cc:844] Serialized 14 byte log entry
0709 13:19:02.337496 (+120574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.337569 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:03.682597 (+1345028us) raft_consensus.cc:1685] finished
0709 13:19:03.685279 (+  2682us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:03.866899 (+181620us) spinlock_profiling.cc:243] Waited 23.4 ms on lock 0x7b1800469d20. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce191e3f3 00007f5ce19207c1 00007f5ce1920779 00007f5ce1920709 00007f5ce19206d1 00007f5ce191f9cd 000000000038f354 000000000038f199 00007f5ce190183a
0709 13:19:03.879303 (+ 12404us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23236,"prepare.queue_time_us":119373,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":28855,"raft.queue_time_us":121891,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":8431,"spinlock_wait_cycles":53839232,"thread_start_us":211697,"threads_started":1,"wal-append.queue_time_us":523503}
W20260709 13:19:04.350970 15641 leader_election.cc:336] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.751s (SENT)
I20260709 13:19:04.351274 17684 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:04.351874 15641 leader_election.cc:336] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.801s (SENT)
I20260709 13:19:03.855053 17734 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:04.352406 15641 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:04.353181 17611 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:19:04.354303 15641 leader_election.cc:336] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.805s (SENT)
I20260709 13:19:04.354779 15641 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:04.303649 15642 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:03.641359 17722 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.534804 17755 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:19:04.309414 15491 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.009s	sys 0.014s
W20260709 13:19:04.356415 15491 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.010s	sys 0.014s
I20260709 13:19:04.356669 17611 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:04.313172 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1524) took 1967 ms. Trace:
I20260709 13:19:04.357077 17611 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.357443 17611 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:04.357332 15519 rpcz_store.cc:276] 0709 13:19:02.346040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.868978 (+1522938us) service_pool.cc:224] Handling call
0709 13:19:04.313068 (+444090us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:04.358546 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:04.359156 15519 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:19:04.359987 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:04.360631 15519 raft_consensus.cc:2393] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:03.488571 17662 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:03.540202 17426 raft_consensus.cc:697] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:19:04.362164 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 58601us
I20260709 13:19:04.362614 17426 consensus_queue.cc:237] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:03.875785 17670 consensus_meta.cc:287] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 1.407s	user 0.009s	sys 0.002s
I20260709 13:19:04.364550 17670 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.366694 17670 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:04.309134 15494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.029s	sys 0.065s
W20260709 13:19:04.367223 15494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.844s	user 0.029s	sys 0.066s
W20260709 13:19:04.223953 17763 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
W20260709 13:19:04.219811 17757 log.cc:927] Time spent T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 13:19:04.368250 17755 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.369146 15675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 1689) took 1877 ms. Trace:
I20260709 13:19:04.369529 15675 rpcz_store.cc:276] 0709 13:19:02.491777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.491916 (+   139us) service_pool.cc:224] Handling call
0709 13:19:02.492255 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:02.764499 (+272244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:02.764536 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:02.765060 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 13:19:03.244843 (+479783us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:03.244884 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:03.245654 (+   770us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:04.368942 (+1123288us) raft_consensus.cc:1685] finished
0709 13:19:04.368962 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:04.369093 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":215231,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":15683,"thread_start_us":309753,"threads_started":2,"wal-append.queue_time_us":577612}
I20260709 13:19:03.815580 17582 raft_consensus.cc:3055] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:04.370889 17582 raft_consensus.cc:740] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:19:04.305344 17572 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.016s	user 0.000s	sys 0.013s
W20260709 13:19:04.337728 15677 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.050s	user 0.008s	sys 0.024s
W20260709 13:19:04.216827 17746 log.cc:927] Time spent T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
W20260709 13:19:04.252604 17429 consensus_meta.cc:287] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 1.720s	user 0.007s	sys 0.004s
I20260709 13:19:04.371721 17582 consensus_queue.cc:260] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:03.718467 17630 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:04.372406 17429 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:04.307849 17688 consensus_meta.cc:287] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 1.071s	user 0.008s	sys 0.001s
I20260709 13:19:04.372970 17630 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.375053 17630 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:04.375334 16239 raft_consensus.cc:2393] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 3: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:04.375830 15524 raft_consensus.cc:2410] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:04.375053 17429 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:04.252903 17604 consensus_meta.cc:287] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 1.328s	user 0.007s	sys 0.001s
I20260709 13:19:04.342324 15528 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 2.
I20260709 13:19:03.874671 17593 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:03.682936 17737 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:04.377022 17737 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.379236 17737 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:04.379487 15528 raft_consensus.cc:1240] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:04.380237 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1523) took 2034 ms. Trace:
I20260709 13:19:04.308634 17482 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:04.355553 17722 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.303543 15523 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:19:04.370428 17755 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:03.866824 17758 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:03.862272 17715 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:04.337599 15521 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:19:04.344452 17728 raft_consensus.cc:740] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:04.397302 17482 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.407496 17482 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:04.390782 17684 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.386319 17734 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:03.867259 17742 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.307577 17578 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.398145 17747 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.350402 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1551) took 1855 ms. Trace:
I20260709 13:19:04.311638 17660 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:04.309330 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:04.433645 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.388521 15528 rpcz_store.cc:276] 0709 13:19:02.345728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.379010 (+2033282us) service_pool.cc:224] Handling call
0709 13:19:04.379330 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.380002 (+   672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.380185 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:04.389976 15641 leader_election.cc:336] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.816s (SENT)
W20260709 13:19:04.436954 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 47046.9us
I20260709 13:19:03.867530 15525 raft_consensus.cc:2376] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4. Re-sending same reply.
I20260709 13:19:04.394380 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1530) took 1906 ms. Trace:
W20260709 13:19:04.395279 16205 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.004s	sys 0.013s
I20260709 13:19:03.863680 17227 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:04.397523 15642 connection.cc:256] RPC call timeout handler was delayed by -2.8050021609999476s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1898s for a response
I20260709 13:19:04.361593 15519 raft_consensus.cc:1240] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:04.443482 15523 rpcz_store.cc:276] 0709 13:19:02.487946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.865737 (+1377791us) service_pool.cc:224] Handling call
0709 13:19:04.394317 (+528580us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:04.445505 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 441) took 2149 ms. Trace:
I20260709 13:19:04.392012 17688 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:04.387305 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 46972.2us
I20260709 13:19:04.394546 17630 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:04.385835 17593 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.414322 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1533) took 1925 ms. Trace:
I20260709 13:19:04.420336 17734 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:04.420856 17571 raft_consensus.cc:3055] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:04.420642 17717 consensus_queue.cc:237] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:04.422322 17741 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.410305 17684 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.423491 17745 consensus_queue.cc:237] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:04.217056 17748 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:04.398571 17429 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.302142 17687 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.341120 17733 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.429215 15526 rpcz_store.cc:276] 0709 13:19:02.495427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.348986 (+1853559us) service_pool.cc:224] Handling call
0709 13:19:04.350335 (+  1349us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:04.429960 16204 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.010s	sys 0.030s
W20260709 13:19:04.480450 16204 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.011s	sys 0.030s
I20260709 13:19:04.351737 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:04.434937 15601 raft_consensus.cc:1240] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:04.435587 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:04.482992 15600 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
W20260709 13:19:04.444619 16205 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.004s	sys 0.014s
W20260709 13:19:04.484115 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 615149us
I20260709 13:19:04.445850 15519 rpcz_store.cc:276] 0709 13:19:02.296148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.361123 (+2064975us) service_pool.cc:224] Handling call
0709 13:19:04.361435 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.445237 (+ 83802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.445449 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:04.449654 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.452358 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:04.347991 17732 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.450881 15602 raft_consensus.cc:1240] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:04.458577 17571 raft_consensus.cc:740] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:04.463057 17771 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.463346 15594 raft_consensus.cc:1240] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:04.439924 17747 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:03.862039 15599 rpcz_store.cc:276] 0709 13:19:01.458184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.680153 (+221969us) service_pool.cc:224] Handling call
0709 13:19:01.707002 (+ 26849us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:01.718769 (+ 11767us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:01.718806 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:01.719334 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 13:19:02.217105 (+497771us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:02.224395 (+  7290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.236368 (+ 11973us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:03.375928 (+1139560us) raft_consensus.cc:1685] finished
0709 13:19:03.376231 (+   303us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:03.394693 (+ 18462us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3682,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":106,"thread_start_us":483604,"threads_started":1,"wal-append.queue_time_us":686043,"wal-append.run_cpu_time_us":10252,"wal-append.run_wall_time_us":1957124}
I20260709 13:19:04.467113 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:04.453404 15525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1562) took 1208 ms. Trace:
W20260709 13:19:04.519768 15642 connection.cc:256] RPC call timeout handler was delayed by -2.5927793379999002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3184s for a response
I20260709 13:19:04.526459 15522 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:04.536881 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:04.437736 17578 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:04.528976 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.455152 15521 rpcz_store.cc:276] 0709 13:19:02.488700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.022066 (+1533366us) service_pool.cc:224] Handling call
0709 13:19:04.412226 (+390160us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:04.538372 15642 connection.cc:256] RPC call timeout handler was delayed by -2.4267051960000572s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30950000000000005s for a response
I20260709 13:19:04.537470 15526 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:19:04.529364 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:04.542439 15527 raft_consensus.cc:2393] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:04.529269 15677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 1694) took 1263 ms. Trace:
I20260709 13:19:04.472841 17748 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.552790 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:04.447289 17660 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:04.542832 15527 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1541) took 2050 ms (client timeout 1911 ms). Trace:
W20260709 13:19:04.541738 15642 connection.cc:256] RPC call timeout handler was delayed by -2.3571914650001418s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1542s for a response
I20260709 13:19:04.542968 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:04.462494 17688 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.573848 17571 consensus_queue.cc:260] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.575119 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:04.544210 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:03.851495 15673 rpcz_store.cc:276] 0709 13:19:01.943807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.943915 (+   108us) service_pool.cc:224] Handling call
0709 13:19:01.944232 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:01.944433 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:19:02.281249 (+336816us) log.cc:844] Serialized 14 byte log entry
0709 13:19:03.289271 (+1008022us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:03.315611 (+ 26340us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:03.321200 (+  5589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:03.380079 (+ 58879us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:03.395744 (+ 15665us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":187124,"prepare.queue_time_us":700,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":397544,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":2900,"thread_start_us":822234,"threads_started":3,"wal-append.queue_time_us":980261}
I20260709 13:19:04.473235 17728 consensus_queue.cc:260] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.543233 15677 rpcz_store.cc:276] 0709 13:19:03.265445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.265579 (+   134us) service_pool.cc:224] Handling call
0709 13:19:03.265859 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:03.266094 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:19:04.371649 (+1105555us) log.cc:844] Serialized 14 byte log entry
0709 13:19:04.528814 (+157165us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:04.528874 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:04.529027 (+   153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.529095 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:04.529211 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4278,"prepare.queue_time_us":950016,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":350,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":5683,"thread_start_us":1239829,"threads_started":2}
W20260709 13:19:04.639833 15642 connection.cc:256] RPC call timeout handler was delayed by -2.1849912820000554s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1632s for a response
I20260709 13:19:04.538736 17734 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.605393 15601 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:04.561693 17670 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.605434 15597 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:04.586162 17777 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:04.606272 15528 raft_consensus.cc:1275] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:04.532562 17767 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.607702 15526 raft_consensus.cc:1240] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:04.636663 15602 raft_consensus.cc:2393] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 4: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:04.679226 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1525) took 2332 ms. Trace:
I20260709 13:19:04.679610 15528 rpcz_store.cc:276] 0709 13:19:02.346289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.539110 (+2192821us) service_pool.cc:224] Handling call
0709 13:19:04.605936 (+ 66826us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.678958 (+ 73022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.679166 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:04.680830 15601 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:19:04.565991 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:04.637892 15527 rpcz_store.cc:269] 0709 13:19:02.492652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.725509 (+232857us) service_pool.cc:224] Handling call
0709 13:19:04.526801 (+1801292us) spinlock_profiling.cc:243] Waited 4.66 ms on lock 0x7b5800003698. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99d10a0 00007f5ce99d0984 00007f5ce2f4b6d4 00007f5ce2f4b662
0709 13:19:04.542774 (+ 15973us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10713984}
W20260709 13:19:04.640257 15642 connection.cc:256] RPC call timeout handler was delayed by -2.0469109759999355s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1908s for a response
I20260709 13:19:04.681967 15522 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
W20260709 13:19:04.682034 15642 connection.cc:256] RPC call timeout handler was delayed by -1.8108244810000542s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1673s for a response
W20260709 13:19:04.530709 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 173876us
I20260709 13:19:04.641034 15519 raft_consensus.cc:2410] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:04.682507 15527 raft_consensus.cc:1240] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:04.682998 15522 raft_consensus.cc:1240] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:04.683178 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1537) took 2193 ms. Trace:
W20260709 13:19:04.641319 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 203799us
I20260709 13:19:04.477689 17429 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.551043 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:04.562827 17754 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:04.682472 15642 connection.cc:256] RPC call timeout handler was delayed by -1.4768989470001088s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1897s for a response
I20260709 13:19:04.684507 15527 rpcz_store.cc:276] 0709 13:19:02.489723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.682051 (+2192328us) service_pool.cc:224] Handling call
0709 13:19:04.682349 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.682957 (+   608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.683128 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:04.684028 15641 leader_election.cc:336] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.874s (SENT)
W20260709 13:19:04.693480 15642 connection.cc:256] RPC call timeout handler was delayed by -1.4174645670000245s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1935s for a response
I20260709 13:19:04.538233 17426 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.688542 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1544) took 2195 ms. Trace:
W20260709 13:19:04.693897 15642 connection.cc:256] RPC call timeout handler was delayed by -1.4153675830000338s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1622s for a response
I20260709 13:19:04.693738 15641 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:04.694309 15642 connection.cc:256] RPC call timeout handler was delayed by -1.2214616840001327s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2675s for a response
W20260709 13:19:04.694626 15642 connection.cc:256] RPC call timeout handler was delayed by -1.1507607920000282s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2086s for a response
I20260709 13:19:04.689754 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 443) took 2328 ms. Trace:
I20260709 13:19:04.694707 15522 rpcz_store.cc:276] 0709 13:19:02.493496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.682595 (+2189099us) service_pool.cc:224] Handling call
0709 13:19:04.682850 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.688327 (+  5477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.688492 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:04.694720 15491 connection.cc:256] RPC call timeout handler was delayed by -1.1250757060001888s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2303s for a response
I20260709 13:19:04.695117 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 444) took 1316 ms. Trace:
W20260709 13:19:04.692212 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 84356.5us
I20260709 13:19:04.482081 17741 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.621872 17748 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:04.695748 15642 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:04.695101 15526 rpcz_store.cc:276] 0709 13:19:02.360916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.542467 (+2181551us) service_pool.cc:224] Handling call
0709 13:19:04.607514 (+ 65047us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.689511 (+ 81997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.689696 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:04.696406 15522 raft_consensus.cc:1275] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:04.697841 16205 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:04.696434 15527 rpcz_store.cc:276] 0709 13:19:03.379043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.694364 (+1315321us) service_pool.cc:224] Handling call
0709 13:19:04.694661 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.694831 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:04.694854 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:04.694894 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:04.694925 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.694975 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:04.695069 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:04.696084 15491 connection.cc:256] RPC call timeout handler was delayed by -1.0922934489999534s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1778s for a response
I20260709 13:19:04.676441 17669 raft_consensus.cc:3055] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:04.702234 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 445) took 1320 ms. Trace:
W20260709 13:19:04.702771 15491 connection.cc:256] RPC call timeout handler was delayed by -1.081875706000119s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2351s for a response
I20260709 13:19:04.703172 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:04.702665 15641 leader_election.cc:336] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.880s (SENT)
I20260709 13:19:04.704860 15600 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
W20260709 13:19:04.705052 15641 leader_election.cc:336] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.896s (SENT)
I20260709 13:19:04.536262 15525 rpcz_store.cc:276] 0709 13:19:03.245273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.319607 (+ 74334us) service_pool.cc:224] Handling call
0709 13:19:04.446830 (+1127223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:04.687875 17728 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:04.705870 15641 leader_election.cc:336] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.908s (SENT)
I20260709 13:19:04.537245 17771 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.706277 17669 raft_consensus.cc:740] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:04.728749 15641 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:04.708139 17715 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:19:04.708922 17768 log.cc:927] Time spent T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:19:04.703827 15528 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:19:04.705904 15522 rpcz_store.cc:276] 0709 13:19:03.381375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.695826 (+1314451us) service_pool.cc:224] Handling call
0709 13:19:04.696101 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.701889 (+  5788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.702178 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:04.721886 15519 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:19:04.735455 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1554) took 2221 ms. Trace:
I20260709 13:19:04.735792 15519 rpcz_store.cc:276] 0709 13:19:02.514848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.693140 (+2178292us) service_pool.cc:224] Handling call
0709 13:19:04.721519 (+ 28379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.735185 (+ 13666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.735403 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:04.737417 15519 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:19:04.688822 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:04.738276 15521 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:19:04.707968 17782 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:04.730029 15641 leader_election.cc:336] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.951s (SENT)
I20260709 13:19:04.696667 17722 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:04.739701 17722 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:04.740053 17722 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.740417 17722 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:04.740916 15641 leader_election.cc:336] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.958s (SENT)
I20260709 13:19:04.734762 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:04.742184 15641 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:04.737802 15519 pending_rounds.cc:85] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 5
I20260709 13:19:04.706609 17716 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:04.743705 15519 pending_rounds.cc:107] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:19:04.709331 17630 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:04.750142 17630 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.677623 17670 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.752473 17670 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:04.754627 17733 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:04.757025 17742 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:04.743863 17778 consensus_queue.cc:1059] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:19:04.760433 17227 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:04.761157 17593 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:04.761343 17722 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.748158 17747 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:04.700863 17754 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.762647 17669 consensus_queue.cc:260] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.763906 17669 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:04.764478 17754 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:04.765019 17722 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:19:04.765390 15641 connection.cc:256] RPC call timeout handler was delayed by -1.2528244529999029s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.273s for a response
W20260709 13:19:04.767844 15641 connection.cc:256] RPC call timeout handler was delayed by -1.0796442380001281s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1542s for a response
I20260709 13:19:04.750624 17630 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:04.746769 15519 raft_consensus.cc:2889] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:04.748972 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:04.771524 15525 raft_consensus.cc:2393] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 3: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:04.750438 17771 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:04.683102 17777 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.773948 17777 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:04.775517 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:04.733361 17755 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:04.781100 15519 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:19:04.782421 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:04.783047 15522 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:19:04.748796 17780 log.cc:927] Time spent T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:19:04.720973 17684 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:04.786605 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:04.687458 17734 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:04.794842 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.706990 17426 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.788808 15519 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:04.765085 17754 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:19:04.806013 17787 log.cc:927] Time spent T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:19:04.806527 17716 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.739281 17782 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.807082 17716 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:04.807399 17741 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:04.809188 17782 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:04.794909 17687 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:04.809250 17716 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:04.811200 17429 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:04.812373 15525 raft_consensus.cc:1275] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:19:04.812117 17748 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.813772 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:04.814646 15525 raft_consensus.cc:2393] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:04.722844 17611 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:04.691972 17767 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.793830 17578 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:04.836957 15603 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:04.834074 17744 consensus_meta.cc:287] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.950s	user 0.008s	sys 0.003s
I20260709 13:19:04.801296 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:04.851346 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 153576us
I20260709 13:19:04.871366 15603 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:04.874876 17777 consensus_queue.cc:1059] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:04.875546 17734 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:04.877137 16205 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:04.881788 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:04.882846 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:04.883476 15595 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:04.894078 17700 log.cc:927] Time spent T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 2.334s	user 0.018s	sys 0.013s
I20260709 13:19:04.894991 17660 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:04.898742 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:04.915246 15602 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:19:04.922650 15522 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:19:04.913578 17755 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:04.853399 17744 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:04.896900 17611 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:04.899132 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:05.097364 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.897492 17688 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:05.098057 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.098162 16205 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:05.099115 16205 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:05.099359 15641 leader_election.cc:336] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.533s (SENT)
I20260709 13:19:05.099828 17427 raft_consensus.cc:3055] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:05.100185 17427 raft_consensus.cc:740] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:05.097211 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:05.100345 15641 leader_election.cc:336] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.542s (SENT)
I20260709 13:19:05.100772 15641 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:04.898110 17594 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:05.100840 15599 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:05.022921 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:05.023795 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.103021 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 446) took 1721 ms. Trace:
W20260709 13:19:05.103567 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.095299 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:05.111227 15520 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1540) took 2618 ms (client timeout 2233 ms). Trace:
W20260709 13:19:05.111671 15520 rpcz_store.cc:269] 0709 13:19:02.492371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.111026 (+2618655us) service_pool.cc:211] Skipping call since client already timed out
0709 13:19:05.111164 (+   138us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:19:05.102350 17594 raft_consensus.cc:697] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:05.113097 17510 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:05.112967 17594 consensus_queue.cc:237] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:05.105094 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 384414us
I20260709 13:19:05.095492 17732 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:05.095808 17426 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:04.895336 17692 raft_consensus.cc:3055] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:05.116731 17692 raft_consensus.cc:740] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:05.111578 15602 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:05.103519 15526 rpcz_store.cc:276] 0709 13:19:03.381717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.102234 (+1720517us) service_pool.cc:224] Handling call
0709 13:19:05.102573 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:05.102739 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:05.102763 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:05.102800 (+    37us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:05.102829 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:05.102879 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:05.102969 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:05.088092 16231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 410) took 3028 ms (client timeout 3205 ms). Trace:
I20260709 13:19:05.119274 16205 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:04.985672 16235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 832) took 2598 ms. Trace:
I20260709 13:19:05.021616 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:05.099431 17582 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:05.126603 15491 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:05.121232 15595 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:05.122607 15596 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:05.121419 15642 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:05.120747 16235 rpcz_store.cc:276] 0709 13:19:02.384449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.469392 (+ 84943us) service_pool.cc:224] Handling call
0709 13:19:02.477987 (+  8595us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:02.599906 (+121919us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:02.599936 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:02.600365 (+   429us) log.cc:844] Serialized 26 byte log entry
0709 13:19:03.200356 (+599991us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:03.200402 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.974055 (+1773653us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b5400720e54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:04.974114 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:04.974172 (+    58us) raft_consensus.cc:1685] finished
0709 13:19:04.974187 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:04.982744 (+  8557us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":249901,"raft.queue_time_us":305910,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":17940,"spinlock_wait_cycles":263451392,"thread_start_us":499910,"threads_started":1,"wal-append.queue_time_us":621319}
I20260709 13:19:05.129454 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:05.130752 15642 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:05.092468 17326 raft_consensus.cc:3055] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:05.132973 15642 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:05.119509 15641 leader_election.cc:336] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.632s (SENT)
I20260709 13:19:05.132635 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:05.118412 17604 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:05.178133 15601 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:19:04.953126 17748 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:05.178759 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:04.980521 17630 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:05.179634 15675 raft_consensus.cc:2393] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 5: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:05.179922 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.180538 15674 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:19:05.180672 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.181464 17224 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.137250 17729 raft_consensus.cc:2804] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:05.181635 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.137591 17569 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:05.119757 17684 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:05.182154 17744 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:05.182432 17684 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.118634 17775 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.182600 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:05.183035 17775 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:05.178368 17604 raft_consensus.cc:697] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:05.185113 17775 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:05.138365 17427 consensus_queue.cc:260] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:05.139186 17623 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:05.185844 17623 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:05.186271 17623 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:05.122958 17476 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.189774 17476 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:05.190888 15669 raft_consensus.cc:2393] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 3: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:05.191967 17792 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.192716 17792 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:05.193254 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.193982 15673 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:05.136644 17688 consensus_queue.cc:1059] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:05.176607 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.195165 17688 raft_consensus.cc:3055] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:05.177510 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.122689 17745 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:19:05.185599 17604 consensus_queue.cc:237] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:05.139801 17326 raft_consensus.cc:740] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:19:05.178496 17793 log.cc:927] Time spent T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 13:19:05.189297 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.132366 17725 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:05.139230 17739 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:05.150319 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.198179 15525 raft_consensus.cc:2376] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3. Re-sending same reply.
I20260709 13:19:05.180881 17630 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:05.166855 16205 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:05.134091 15519 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
W20260709 13:19:05.201256 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.195442 15672 raft_consensus.cc:2393] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:05.249111 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:05.250648 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.250890 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.252943 15528 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:19:05.202318 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 69445.6us
W20260709 13:19:05.253546 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 86816.6us
W20260709 13:19:05.251312 15567 connection.cc:256] RPC call timeout handler was delayed by -2.4071205460002147s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15200000000000002s for a response
I20260709 13:19:05.192050 17476 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:05.264971 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.248495 17326 consensus_queue.cc:260] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:05.252067 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.255641 16205 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:05.249399 17427 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:05.200914 17684 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:05.254662 15642 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:05.252934 15676 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
W20260709 13:19:05.302976 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 47416.3us
I20260709 13:19:05.292114 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:05.247059 17669 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.195542 17688 raft_consensus.cc:740] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:05.240320 17224 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:05.293430 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.309195 15527 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:05.294159 15528 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:19:05.121429 17611 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:19:05.119871 16231 rpcz_store.cc:269] 0709 13:19:02.059441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.071998 (+ 12557us) service_pool.cc:224] Handling call
0709 13:19:05.087842 (+3015844us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54005370d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:05.088020 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2514002176}
I20260709 13:19:05.182042 17729 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:05.137647 17754 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:19:05.252802 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 133323us
W20260709 13:19:05.299739 15567 connection.cc:256] RPC call timeout handler was delayed by -2.3126853349999692s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21030000000000004s for a response
W20260709 13:19:05.315528 15567 connection.cc:256] RPC call timeout handler was delayed by -2.2779425500000343s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2183s for a response
I20260709 13:19:05.240769 17744 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:05.301004 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:05.319769 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.320351 15526 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:19:05.308010 17669 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:05.321165 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.312775 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.137970 17716 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:05.132213 17767 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:05.252583 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.322773 17739 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:05.322537 17688 consensus_queue.cc:260] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:05.323490 15526 raft_consensus.cc:1240] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:05.323727 17688 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:05.324627 17744 raft_consensus.cc:697] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:05.325089 17571 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:05.302218 15519 raft_consensus.cc:2393] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:05.325948 17744 consensus_queue.cc:237] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:05.326836 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.323657 17669 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:05.327991 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:05.328542 17224 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:05.322507 15671 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:05.307246 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:05.332093 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.248859 17721 consensus_queue.cc:1048] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:19:05.295012 17732 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:05.310588 15527 raft_consensus.cc:1240] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: []
W20260709 13:19:05.332518 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:05.332926 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.333395 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.334026 15527 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:19:05.334896 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.335922 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:05.336388 15527 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:05.338510 15523 raft_consensus.cc:1240] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:19:05.254873 17725 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.340610 17725 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:05.342109 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.342429 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.343089 15672 raft_consensus.cc:2410] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:19:05.343261 15673 raft_consensus.cc:2410] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:05.343601 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.344342 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.345170 15668 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 5: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:05.347108 17732 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.326721 15675 raft_consensus.cc:2410] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:19:05.347589 17732 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:05.351481 17570 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:19:05.357067 16239 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 418) took 3063 ms (client timeout 1622 ms). Trace:
I20260709 13:19:05.357360 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.329649 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.359252 15674 raft_consensus.cc:2410] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:05.360589 17749 consensus_queue.cc:1048] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:19:05.362388 16239 rpcz_store.cc:269] 0709 13:19:02.293778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.406168 (+112390us) service_pool.cc:224] Handling call
0709 13:19:05.356829 (+2950661us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b5400241b54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:05.356985 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4341709056}
I20260709 13:19:05.362823 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.363502 15521 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:19:05.364464 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.365185 15521 raft_consensus.cc:2368] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 3. Current term is 4.
I20260709 13:19:05.366539 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.366752 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:05.367480 15526 raft_consensus.cc:2393] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 3: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:05.367657 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.368180 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.314913 17729 raft_consensus.cc:3060] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:05.251365 17692 consensus_queue.cc:260] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:05.303825 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 49269.5us
I20260709 13:19:05.323304 17739 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:05.369207 17692 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:05.371187 17739 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:05.372524 17716 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:05.308291 17662 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:05.374344 15526 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:19:05.374575 17662 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.374977 17662 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:05.374745 17716 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:05.375239 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.375502 17326 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:05.316207 15568 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:05.377106 15524 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1518) took 3054 ms (client timeout 1908 ms). Trace:
W20260709 13:19:05.316787 15641 leader_election.cc:336] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.673s (SENT)
W20260709 13:19:05.377496 15524 rpcz_store.cc:269] 0709 13:19:02.322554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.617266 (+294712us) service_pool.cc:224] Handling call
0709 13:19:05.376930 (+2759664us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b54003ce6d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:05.377045 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3934578432}
I20260709 13:19:05.378890 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:05.379400 15524 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:05.368372 15520 raft_consensus.cc:2393] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:05.381883 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:05.305222 16205 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:05.376375 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.382640 15520 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:05.377770 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:05.327414 15519 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:05.389786 15523 raft_consensus.cc:1240] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:05.312580 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:19:05.416465 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.417297 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:05.418138 15525 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:05.365994 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.420817 15521 raft_consensus.cc:2393] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:05.419893 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:05.422487 15674 raft_consensus.cc:2368] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4.
I20260709 13:19:05.421734 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.423834 15521 raft_consensus.cc:2410] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 4 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:19:05.424172 17729 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:05.425276 15521 raft_consensus.cc:1240] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
W20260709 13:19:05.432685 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 115929us
I20260709 13:19:05.427301 15520 raft_consensus.cc:1275] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:19:05.427280 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.452373 15522 raft_consensus.cc:2393] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:05.453147 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.427587 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.446836 15524 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
W20260709 13:19:05.419720 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 57863.8us
I20260709 13:19:05.454598 16204 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:05.454782 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.438506 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:19:05.451228 15641 leader_election.cc:336] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.723s (SENT)
I20260709 13:19:05.455996 15524 raft_consensus.cc:2393] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:05.419893 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.456923 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:19:05.445591 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 58813.4us
I20260709 13:19:05.450376 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.425998 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.430757 17754 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:05.431878 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:05.460479 15526 raft_consensus.cc:2376] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6. Re-sending same reply.
I20260709 13:19:05.307789 17792 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:05.458326 15674 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:19:05.458297 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.462178 15520 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:05.460196 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.453817 15522 raft_consensus.cc:2368] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 3. Current term is 4.
W20260709 13:19:05.462975 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 146840us
I20260709 13:19:05.463387 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:05.463532 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.463862 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.463946 15523 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:05.313489 17630 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:05.466655 16204 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:05.433296 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 128190us
I20260709 13:19:05.442106 17721 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.467968 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:05.468587 15601 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:05.468530 16239 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:05.463006 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.469019 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:05.470551 15520 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:05.433172 17570 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:05.469066 16205 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:05.471943 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.457149 15519 raft_consensus.cc:2393] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:05.440219 15672 raft_consensus.cc:1240] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:05.459909 15670 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:19:05.477336 15520 raft_consensus.cc:2393] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:05.434124 15671 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:19:05.466313 15528 raft_consensus.cc:2393] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 3: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:05.431391 17725 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:05.432363 17729 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:05.471684 15566 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:05.488889 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:05.475232 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:05.490770 15595 raft_consensus.cc:2393] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:05.480250 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:05.480505 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.492000 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:05.492774 15601 raft_consensus.cc:2393] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 3: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:05.492434 15677 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:05.481295 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.456972 15527 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:05.470914 16234 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:05.456141 15641 leader_election.cc:336] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.730s (SENT)
I20260709 13:19:05.483362 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:05.496284 15641 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:05.465209 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.464520 17427 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:05.488284 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:05.488407 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:05.485072 16203 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:05.496531 15595 raft_consensus.cc:1275] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:19:05.509284 16238 raft_consensus.cc:2393] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:05.466899 17630 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:05.497103 15596 raft_consensus.cc:1240] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:05.497084 15522 raft_consensus.cc:2393] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:19:05.497696 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.499091 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.465611 17574 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:05.503733 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 47619.1us
W20260709 13:19:05.519914 15641 leader_election.cc:336] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 3.001s (SENT)
I20260709 13:19:05.496778 15603 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:05.509800 16235 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:05.510190 15595 raft_consensus.cc:1240] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:19:05.493836 17355 raft_consensus.cc:2804] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:05.503580 15521 raft_consensus.cc:2393] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:05.482144 17721 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:05.514436 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.522855 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.523406 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.521842 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:05.525010 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:05.525200 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:05.526023 15594 raft_consensus.cc:2368] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4.
I20260709 13:19:05.526238 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.526854 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:05.528072 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:05.528664 15594 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:19:05.529516 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:05.530438 17779 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:05.521898 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:05.531687 17576 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:05.532188 17576 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.532522 17576 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:05.533185 17779 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.533665 17779 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:05.535013 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.535718 15526 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
W20260709 13:19:05.513888 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 44892.7us
I20260709 13:19:05.515955 15527 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:05.516321 16203 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:05.519033 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:05.537173 15522 raft_consensus.cc:2376] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4. Re-sending same reply.
I20260709 13:19:05.538203 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.511970 15523 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:19:05.538197 16237 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:05.536361 15528 raft_consensus.cc:2368] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 3. Current term is 4.
I20260709 13:19:05.539608 15525 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:05.536998 16205 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:05.541884 16205 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:05.542445 15528 raft_consensus.cc:1240] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:19:05.542685 16203 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:05.539228 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.543736 15522 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:05.539819 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:05.541576 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.546360 16204 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:05.544816 15522 raft_consensus.cc:1275] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:19:05.541535 15527 raft_consensus.cc:1240] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:05.523730 15521 raft_consensus.cc:2410] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:05.553659 15523 raft_consensus.cc:2393] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:05.514591 17427 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.494522 17570 raft_consensus.cc:697] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:05.558907 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.597312 16239 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:05.521953 17355 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.458473 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:05.525753 17721 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:05.534595 16232 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:19:05.565809 16203 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:19:05.568276 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.518653 17630 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:05.598429 15675 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:05.620651 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.606644 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:05.606918 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:05.607872 15596 raft_consensus.cc:1240] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:05.624018 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:05.603737 15524 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
W20260709 13:19:05.627195 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 61497.4us
I20260709 13:19:05.547941 17749 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:05.627036 15526 raft_consensus.cc:1275] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:19:05.548355 17224 raft_consensus.cc:2804] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:05.557997 17669 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:05.634374 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.560725 17735 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:05.554508 15594 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:05.641983 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:05.641784 15675 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:05.650887 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:05.651621 15675 raft_consensus.cc:2368] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4.
I20260709 13:19:05.601953 17427 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:05.643875 16234 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:05.679405 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:05.601708 17221 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:05.600945 17777 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:05.681217 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:05.607347 17355 raft_consensus.cc:3060] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:05.625108 16232 pending_rounds.cc:85] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9: Aborting all ops after (but not including) 4
I20260709 13:19:05.649672 15670 raft_consensus.cc:2376] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3. Re-sending same reply.
I20260709 13:19:05.658776 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:05.660806 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.640672 17721 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:05.700242 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.700390 16239 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:05.680239 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:05.700893 15670 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:19:05.649821 17224 raft_consensus.cc:697] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:05.682585 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:05.642145 17669 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.692679 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:05.690397 17777 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.647171 17735 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.689332 16203 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:05.701565 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.701718 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.704820 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.740813 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.742488 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:05.744405 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 55163us
I20260709 13:19:05.746628 16203 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:05.743136 15675 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:19:05.771458 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.642962 17732 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:05.693291 17582 consensus_meta.cc:287] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.553s	user 0.008s	sys 0.003s
I20260709 13:19:05.790798 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:05.793754 15670 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:05.704103 16232 pending_rounds.cc:107] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9: Aborting uncommitted NO_OP operation due to leader change: 2.5
W20260709 13:19:05.797618 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 51081.3us
I20260709 13:19:05.798457 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:05.795531 15675 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:05.725190 16236 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:05.806420 15675 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:05.716359 17380 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.810585 16203 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:05.810941 15670 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:05.709229 17721 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:05.715298 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:05.666612 17623 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:05.797245 16232 raft_consensus.cc:2889] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:05.737565 17669 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:05.814445 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:05.738251 17777 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:05.751497 17735 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:05.816466 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:05.832931 15603 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:05.816442 15567 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.024s	sys 0.068s
I20260709 13:19:05.797951 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:05.812772 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:05.809116 17594 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:05.813321 17604 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:05.833540 15567 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.024s	sys 0.068s
I20260709 13:19:05.822446 17721 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:05.837100 15566 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:05.737650 17570 consensus_queue.cc:237] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:05.811973 17224 consensus_queue.cc:237] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:05.847857 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 531845us
I20260709 13:19:05.848641 15566 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:05.852082 15491 leader_election.cc:400] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4.
I20260709 13:19:05.853184 17569 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:05.869297 15603 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:05.848464 17725 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:05.870471 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 349763us
W20260709 13:19:05.899338 15641 leader_election.cc:336] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 3.154s (SENT)
I20260709 13:19:05.880451 15675 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 13:19:05.838596 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:05.881458 15566 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:05.825459 17380 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:05.851994 15594 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:05.849900 16236 raft_consensus.cc:1240] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:19:05.883001 15491 leader_election.cc:403] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:19:05.918547 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 73672.3us
I20260709 13:19:05.900130 15675 pending_rounds.cc:85] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting all ops after (but not including) 6
W20260709 13:19:05.846545 15642 leader_election.cc:336] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Timed out: RequestConsensusVote RPC to 127.14.23.68:39627 timed out after 1.622s (SENT)
I20260709 13:19:05.847229 17754 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:19:05.872484 15567 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.927s (SENT) [suppressed 68 similar messages]
I20260709 13:19:05.893069 17569 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:05.923190 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 50737.2us
W20260709 13:19:05.923691 15567 connection.cc:256] RPC call timeout handler was delayed by -2.21793508199994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1897s for a response
W20260709 13:19:05.924022 15567 connection.cc:256] RPC call timeout handler was delayed by -2.1239809800001694s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24680000000000002s for a response
W20260709 13:19:05.924351 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:05.924680 15567 connection.cc:256] RPC call timeout handler was delayed by -2.06250460199999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1885s for a response
W20260709 13:19:05.925107 15567 connection.cc:256] RPC call timeout handler was delayed by -1.6034281960000953s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3004s for a response
W20260709 13:19:05.925400 15567 connection.cc:256] RPC call timeout handler was delayed by -1.3947308060000978s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20579999999999998s for a response
W20260709 13:19:05.925771 15567 connection.cc:256] RPC call timeout handler was delayed by -1.3617029710001134s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1569s for a response
W20260709 13:19:05.926491 15567 connection.cc:256] RPC call timeout handler was delayed by -1.3308308060002219s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2129s for a response
W20260709 13:19:05.926798 15567 connection.cc:256] RPC call timeout handler was delayed by -1.1316502019999461s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1756s for a response
W20260709 13:19:05.927079 15567 connection.cc:256] RPC call timeout handler was delayed by -1.0842858460000571s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2284s for a response
W20260709 13:19:05.927390 15567 connection.cc:256] RPC call timeout handler was delayed by -1.0289196080000238s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16080000000000003s for a response
W20260709 13:19:05.928578 15567 leader_election.cc:336] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.520s (SENT)
W20260709 13:19:05.929378 15567 leader_election.cc:336] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.569s (SENT)
I20260709 13:19:05.908703 17576 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:05.919811 15675 pending_rounds.cc:107] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:19:05.931653 15675 raft_consensus.cc:2889] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:05.930621 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:05.932701 15603 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
W20260709 13:19:05.934077 15641 leader_election.cc:336] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 3.095s (SENT)
I20260709 13:19:05.929762 15567 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:05.936321 15670 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:19:05.889705 17687 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:05.940212 17732 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:05.940834 17380 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:05.942512 17669 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:05.937222 17426 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:05.930548 17569 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:05.943656 17426 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.944118 17426 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:05.945053 17669 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:05.950485 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:05.951354 15603 raft_consensus.cc:2393] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 3: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:05.951840 17426 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:05.953579 17744 consensus_queue.cc:1048] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:05.954964 17576 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:05.957782 17729 consensus_queue.cc:1059] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:19:05.959373 17611 consensus_queue.cc:1059] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:05.968688 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:05.960705 17717 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:05.963145 17716 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:05.952497 15603 raft_consensus.cc:1240] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:05.970777 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:05.975484 15670 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:05.921013 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 74521.5us
W20260709 13:19:05.973785 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 44437.5us
W20260709 13:19:05.976629 15567 leader_election.cc:336] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.608s (SENT)
I20260709 13:19:05.977052 15567 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:05.979444 17426 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:05.968804 17569 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:05.930680 17482 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:05.971148 17687 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.986419 17716 consensus_queue.cc:1048] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:19:05.987610 17801 log.cc:927] Time spent T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:19:05.981180 17684 consensus_meta.cc:287] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.669s	user 0.003s	sys 0.007s
I20260709 13:19:05.980603 17571 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:05.977725 17660 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:05.997426 15603 raft_consensus.cc:1275] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:19:06.004462 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.965488 17355 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:05.988559 17735 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:05.978590 15673 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:05.980201 17426 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:05.993026 17687 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:06.044273 16240 raft_consensus.cc:2368] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 3. Current term is 4.
I20260709 13:19:05.982592 17807 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:19:06.001853 17684 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.027684 17630 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.071630 17623 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:06.125232 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 148618us
W20260709 13:19:06.127118 15602 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.000s	sys 0.029s
W20260709 13:19:06.127576 15602 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.000s	sys 0.029s
I20260709 13:19:06.128703 15675 raft_consensus.cc:1240] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4. Ops: []
W20260709 13:19:06.130318 15567 leader_election.cc:336] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.885s (SENT)
I20260709 13:19:06.132581 15567 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:06.120806 17662 consensus_meta.cc:287] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.745s	user 0.008s	sys 0.009s
W20260709 13:19:06.139200 15567 leader_election.cc:336] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.058s (SENT)
I20260709 13:19:06.141222 15567 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:06.142902 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 178895us
W20260709 13:19:06.144975 15567 leader_election.cc:336] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.129s (SENT)
I20260709 13:19:06.145507 15567 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:06.129094 17623 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.147737 17623 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.148252 17355 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:06.138443 17426 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:06.150952 15567 leader_election.cc:336] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.284s (SENT)
W20260709 13:19:06.154531 15567 leader_election.cc:336] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 3.076s (SENT)
I20260709 13:19:06.156008 15567 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:06.146116 17660 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:19:06.162844 15567 leader_election.cc:336] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 3.121s (SENT)
I20260709 13:19:06.153676 17580 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:06.142082 17662 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.167387 15670 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:06.168016 15567 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:06.168109 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 186945us
I20260709 13:19:06.170569 15566 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:06.162119 17569 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:06.175819 15642 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:06.163769 17576 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:06.180776 16236 raft_consensus.cc:1240] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:19:06.181198 17660 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:19:06.189832 17815 log.cc:927] Time spent T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:19:06.191447 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:06.198611 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:06.169279 17777 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.201800 17576 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:06.202281 17630 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:06.202672 17660 raft_consensus.cc:697] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:06.202898 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:06.204401 17660 consensus_queue.cc:237] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:06.206820 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:06.207607 15677 raft_consensus.cc:2393] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:06.211395 17777 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:06.211725 17810 log.cc:927] Time spent T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:19:06.211457 17827 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:06.211470 17823 consensus_queue.cc:1059] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:19:06.212777 17827 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:06.213239 17827 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:06.214411 17777 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.215001 17777 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.217227 17721 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:06.221122 17427 consensus_meta.cc:287] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.511s	user 0.009s	sys 0.000s
I20260709 13:19:06.228529 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:06.229151 15526 raft_consensus.cc:3060] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:06.231052 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:06.221367 17739 raft_consensus.cc:1064] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:19:06.223230 17777 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:19:06.224020 17816 log.cc:927] Time spent T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:19:06.224468 17623 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:06.225235 17688 consensus_meta.cc:287] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.901s	user 0.012s	sys 0.000s
I20260709 13:19:06.179394 17735 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:06.227290 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:06.296933 15528 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:06.228094 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:06.282284 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:06.231786 15522 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:06.238938 17355 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.186036 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:06.251260 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:06.242921 17427 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.290939 17662 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:06.309188 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:06.309887 16240 raft_consensus.cc:2368] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 5.
I20260709 13:19:06.310645 16238 raft_consensus.cc:2393] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:06.310613 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:06.311249 16240 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:19:06.280205 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:06.242947 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:06.318302 15521 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:19:06.294325 15520 raft_consensus.cc:2410] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:19:06.286274 17669 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.322707 17669 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.325084 17669 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:06.287626 17749 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.295239 17721 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:06.296065 17779 consensus_meta.cc:287] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.762s	user 0.002s	sys 0.009s
I20260709 13:19:06.326758 17779 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.325827 17749 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.328919 17779 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:06.282984 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:06.300450 16204 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:06.329335 17749 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:06.329679 15519 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:06.329988 16234 raft_consensus.cc:2368] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5.
I20260709 13:19:06.330653 17721 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:06.331661 16204 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:06.301844 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.334596 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:06.301024 16203 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:06.338790 17779 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:06.339241 17779 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:06.339578 17779 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:06.340888 16204 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:06.253284 17777 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.341740 17777 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.311420 17427 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:06.346339 17582 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:06.346825 17582 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:06.347162 17582 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:06.348507 17779 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.350099 17777 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:06.333164 17224 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:06.352478 17224 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:06.352949 17224 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:06.353128 17779 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:06.303726 15526 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:06.335707 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:06.336470 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:06.354578 15522 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:06.337155 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:06.355512 15523 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:06.366976 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:06.368108 16240 raft_consensus.cc:2393] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 3: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:06.369760 16204 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:06.370349 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:06.370844 16204 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:06.371779 17669 raft_consensus.cc:2804] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:06.372331 17669 raft_consensus.cc:697] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:06.374137 17662 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:06.374624 17662 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:06.374993 17662 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:06.373282 17669 consensus_queue.cc:237] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.387224 16231 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:19:06.389111 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:06.392863 15521 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:06.400321 15528 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:06.395421 16204 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:06.402292 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:06.402297 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 206015us
I20260709 13:19:06.403035 16240 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:19:06.403090 16204 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:06.404744 15641 leader_election.cc:336] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.558s (SENT)
I20260709 13:19:06.296486 17482 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4.
I20260709 13:19:06.407152 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:19:06.407418 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 228088us
I20260709 13:19:06.282087 17684 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:06.295258 17744 raft_consensus.cc:3055] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:19:06.409300 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 241665us
I20260709 13:19:06.411257 16240 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:06.287941 15673 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:06.416673 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:06.307958 15594 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:06.253765 17611 consensus_queue.cc:1048] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.155s
I20260709 13:19:06.395617 15671 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:06.436352 15493 connection.cc:256] RPC call timeout handler was delayed by -2.96910461199991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1512s for a response
I20260709 13:19:06.438730 15567 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:06.454447 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:06.455873 15642 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:06.445470 16240 raft_consensus.cc:1240] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:19:06.406731 17777 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:06.457199 17777 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:06.457608 17777 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:06.457705 17828 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:06.458324 17828 raft_consensus.cc:697] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:06.459196 17828 consensus_queue.cc:237] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.460851 17630 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:06.461542 17630 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:06.461962 17630 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:06.463202 17660 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:06.465226 17717 raft_consensus.cc:3055] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:06.465628 17717 raft_consensus.cc:740] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:06.466499 17717 consensus_queue.cc:260] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:06.467720 17717 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:06.446931 17576 consensus_queue.cc:1048] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:19:06.474673 16236 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:06.441226 17224 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:06.450663 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:06.453178 15493 connection.cc:256] RPC call timeout handler was delayed by -2.9565046119998897s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1526s for a response
W20260709 13:19:06.477516 15493 connection.cc:256] RPC call timeout handler was delayed by -2.9241046119998373s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1562s for a response
W20260709 13:19:06.477826 15493 connection.cc:256] RPC call timeout handler was delayed by -2.8773046119997616s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16140000000000002s for a response
W20260709 13:19:06.478111 15493 connection.cc:256] RPC call timeout handler was delayed by -2.5434046119999039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1985s for a response
W20260709 13:19:06.478466 15493 connection.cc:256] RPC call timeout handler was delayed by -2.4696046119997845s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20670000000000002s for a response
W20260709 13:19:06.478576 15567 leader_election.cc:336] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 3.214s (SENT)
I20260709 13:19:06.479153 15567 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:06.394394 17355 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.436012 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:06.485523 16235 raft_consensus.cc:1240] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4. Ops: []
W20260709 13:19:06.478785 15493 connection.cc:256] RPC call timeout handler was delayed by -2.4336046119997263s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21070000000000003s for a response
I20260709 13:19:06.478039 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:06.441061 17749 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:19:06.486707 15493 connection.cc:256] RPC call timeout handler was delayed by -2.4228046119999362s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21190000000000003s for a response
I20260709 13:19:06.486621 15566 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:06.440042 17744 raft_consensus.cc:740] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
W20260709 13:19:06.489462 15493 connection.cc:256] RPC call timeout handler was delayed by -2.418304611999929s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21240000000000003s for a response
W20260709 13:19:06.490095 15493 connection.cc:256] RPC call timeout handler was delayed by -2.386804611999878s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21589999999999998s for a response
W20260709 13:19:06.490391 15493 connection.cc:256] RPC call timeout handler was delayed by -2.376904611999862s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.217s for a response
W20260709 13:19:06.490657 15493 connection.cc:256] RPC call timeout handler was delayed by -2.2590046119998988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23010000000000003s for a response
W20260709 13:19:06.490921 15493 connection.cc:256] RPC call timeout handler was delayed by -2.214004611999826s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2351s for a response
W20260709 13:19:06.491190 15493 connection.cc:256] RPC call timeout handler was delayed by -1.8270046119998824s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2781s for a response
W20260709 13:19:06.491442 15493 connection.cc:256] RPC call timeout handler was delayed by -1.7919046119998256s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.282s for a response
W20260709 13:19:06.491688 15493 connection.cc:256] RPC call timeout handler was delayed by -1.712704611999925s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2908s for a response
W20260709 13:19:06.491932 15493 connection.cc:256] RPC call timeout handler was delayed by -1.5885046119997241s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3046s for a response
W20260709 13:19:06.492192 15493 connection.cc:256] RPC call timeout handler was delayed by -1.5246046119998482s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31170000000000003s for a response
W20260709 13:19:06.492431 15493 connection.cc:256] RPC call timeout handler was delayed by -1.3329046119997656s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.333s for a response
I20260709 13:19:06.500407 15519 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:06.476519 17221 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.479270 17825 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.507982 15642 leader_election.cc:400] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 3. Current term is 4.
I20260709 13:19:06.507181 15493 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:06.515749 15642 leader_election.cc:403] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:19:06.516121 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:06.485670 17729 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:19:06.517225 15491 leader_election.cc:336] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.826s (SENT)
I20260709 13:19:06.439790 17826 consensus_queue.cc:1059] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:19:06.519366 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:06.485396 17807 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:06.520592 15675 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:06.502516 17833 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:19:06.529948 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.533416 15677 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:06.536772 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:06.505601 17749 raft_consensus.cc:697] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:06.567654 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:06.568486 17749 consensus_queue.cc:237] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.520177 17221 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.571700 17221 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:06.568456 15668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 1667) took 1115 ms. Trace:
I20260709 13:19:06.572814 15668 rpcz_store.cc:276] 0709 13:19:05.453529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.522360 (+1068831us) service_pool.cc:224] Handling call
0709 13:19:06.568391 (+ 46031us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:06.575062 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 82151.1us
W20260709 13:19:06.575695 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.576767 15493 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:06.577880 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:06.541803 17355 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:06.544082 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:06.577908 15493 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:06.580474 15524 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:06.544842 17582 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.554138 17744 consensus_queue.cc:260] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:06.524868 17834 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:06.519016 17825 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.502260 17687 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:06.533102 17824 consensus_queue.cc:1059] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:19:06.524523 17482 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:06.533805 17826 raft_consensus.cc:3060] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:06.520067 17729 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:06.559690 17630 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.565263 17224 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:06.567123 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:06.583063 17830 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:19:06.610054 15491 leader_election.cc:336] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.924s (SENT)
I20260709 13:19:06.610688 15491 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:06.580580 15600 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:06.550733 15671 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:06.581139 15493 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:06.581252 15676 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:06.574971 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:06.613143 15528 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:06.582094 15675 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:06.582298 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:06.614462 15602 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:06.480449 17662 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.590695 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:06.594394 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:06.602667 15525 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:06.579223 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:06.583657 17835 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:19:06.612362 15641 leader_election.cc:336] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.827s (SENT)
I20260709 13:19:06.610136 15595 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:06.559724 17827 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:06.620389 15596 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:06.595057 17834 raft_consensus.cc:697] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:06.625559 17482 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:06.600406 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:19:06.626574 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 45514.6us
I20260709 13:19:06.626443 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:06.622615 15641 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:06.646836 15596 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:06.646871 15493 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:06.647758 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:06.648368 15596 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:06.649318 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:06.620576 17744 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:06.661237 15641 leader_election.cc:336] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.897s (SENT)
I20260709 13:19:06.630254 17630 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:06.619415 17623 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:06.625694 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:06.632634 17662 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:06.635831 15525 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:06.637382 15602 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:06.669121 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:06.669550 15602 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:06.603519 17729 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:06.648151 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:06.670326 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:06.671047 15602 raft_consensus.cc:2393] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 5: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:06.671976 17224 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.672509 17224 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.628425 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:06.660916 17828 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:06.674216 15669 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:06.640372 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:06.675534 15519 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:06.641520 15600 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:06.642391 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:06.627350 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:06.651535 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:06.652228 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 144408us
I20260709 13:19:06.628705 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:06.661760 15641 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:06.677945 15493 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:06.662719 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:06.677038 15675 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:06.678841 15596 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
W20260709 13:19:06.680064 15641 leader_election.cc:336] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.898s (SENT)
I20260709 13:19:06.680512 15641 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:06.676626 15599 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:06.681545 15641 leader_election.cc:336] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.911s (SENT)
I20260709 13:19:06.682544 17717 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:06.687064 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:06.687726 15596 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:06.689045 16205 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:06.691221 15600 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:06.662025 17830 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:06.693289 15603 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:06.670984 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:06.696902 17582 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:06.677111 15598 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:06.700271 15594 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 5: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:06.698304 17827 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:06.693962 17688 raft_consensus.cc:2804] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:06.694914 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:06.701187 17688 raft_consensus.cc:697] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:06.701658 15672 raft_consensus.cc:2393] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:06.700080 15566 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:06.702441 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1559) took 1269 ms. Trace:
I20260709 13:19:06.702087 17688 consensus_queue.cc:237] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.703183 17510 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:06.703666 17510 raft_consensus.cc:697] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:06.704514 17510 consensus_queue.cc:237] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.705066 16234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 424) took 1027 ms. Trace:
I20260709 13:19:06.705452 17834 consensus_queue.cc:237] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:06.693413 15601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 1584) took 1205 ms. Trace:
I20260709 13:19:06.702806 15520 rpcz_store.cc:276] 0709 13:19:05.432924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.526012 (+ 93088us) service_pool.cc:224] Handling call
0709 13:19:06.702264 (+1176252us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b54002ddcd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:06.702387 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1764337536}
I20260709 13:19:06.706552 15601 rpcz_store.cc:276] 0709 13:19:05.487553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.497160 (+  9607us) service_pool.cc:224] Handling call
0709 13:19:05.497579 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:05.541628 (+ 44049us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:05.541657 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:05.661850 (+120193us) spinlock_profiling.cc:243] Waited 1.45 ms on lock 0x7b2c000fb608. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce3304a11 00007f5ce3314372 00007f5ce4a46a01 00007f5ce4a0b43a 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2
0709 13:19:05.738900 (+ 77050us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.001557 (+262657us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:06.001593 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.692998 (+691405us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b54000ea954. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:06.693053 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:06.693113 (+    60us) raft_consensus.cc:1685] finished
0709 13:19:06.693132 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:06.693336 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19086,"raft.queue_time_us":23063,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":12635,"spinlock_wait_cycles":59842688,"thread_start_us":181613,"threads_started":1,"wal-append.queue_time_us":300251}
I20260709 13:19:06.690807 17835 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:06.709980 17687 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:06.710189 16205 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:06.710909 17727 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:06.711337 17727 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:06.705423 16234 rpcz_store.cc:276] 0709 13:19:05.677266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.679149 (+  1883us) service_pool.cc:224] Handling call
0709 13:19:06.704878 (+1025729us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b540041b4d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:06.705012 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1494437120}
I20260709 13:19:06.714764 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:06.715466 16234 raft_consensus.cc:2393] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:06.716987 16205 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:06.719200 15525 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:06.722298 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:06.728562 17735 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:06.729060 17735 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:06.729389 17735 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:06.730716 17825 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:06.731420 17717 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:06.732947 17224 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:06.734433 15525 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:06.734476 17611 raft_consensus.cc:3055] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:06.717519 17571 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:06.735333 17611 raft_consensus.cc:740] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:06.735785 17571 raft_consensus.cc:697] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:06.736228 17611 consensus_queue.cc:260] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.736673 17571 consensus_queue.cc:237] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.737411 17611 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:06.714395 17727 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:06.747336 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 61273.9us
W20260709 13:19:06.762760 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 84989.9us
I20260709 13:19:06.763500 15669 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:06.763633 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:06.764941 15493 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:06.767158 15493 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:06.769107 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:06.769590 17355 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:06.769824 15601 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:19:06.771634 17355 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:06.773733 15642 leader_election.cc:400] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5.
I20260709 13:19:06.774094 15642 leader_election.cc:403] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:19:06.764966 15675 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:06.783689 17833 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:06.783864 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:06.784619 15672 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:19:06.785635 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:06.786013 17482 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:06.786235 15525 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:06.787003 17729 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.789094 17729 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:06.789515 15596 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:06.766197 17716 consensus_queue.cc:1059] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:19:06.790766 15566 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:06.790769 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:06.791821 15676 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:06.783706 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:06.793385 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:06.793757 15603 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:06.795316 15519 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:06.791134 15568 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:06.795967 15493 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:06.796377 15669 raft_consensus.cc:2393] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 5: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:06.797201 15493 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:06.769301 17823 raft_consensus.cc:1064] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:19:06.793831 15525 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:06.799925 15568 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:06.802695 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:06.803449 15669 raft_consensus.cc:2393] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:06.804082 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:19:06.809557 15641 leader_election.cc:336] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.973s (SENT)
W20260709 13:19:06.810635 15641 leader_election.cc:336] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.986s (SENT)
I20260709 13:19:06.811053 15641 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:06.813059 16234 raft_consensus.cc:1240] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:06.813202 16237 raft_consensus.cc:1240] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:06.808564 15491 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:06.794866 17574 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.818812 15491 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:06.798545 17825 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.824726 15676 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:06.798899 17623 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:06.827370 15566 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:06.800923 17660 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:06.798530 17830 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:06.798767 17835 raft_consensus.cc:2764] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 13:19:06.835587 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:19:06.839704 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.804937 17837 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:06.805274 17807 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:19:06.806093 17823 raft_consensus.cc:1068] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4: Unable to promote non-voter dd1a14c3173d49dc9b55bb116e3783c9: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:19:06.836463 15603 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:06.841022 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:06.841524 15642 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:06.841801 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:06.837755 17754 raft_consensus.cc:3055] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:06.843349 15670 raft_consensus.cc:1240] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:06.838421 17825 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.786015 17777 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.825851 17688 raft_consensus.cc:2764] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
W20260709 13:19:06.842507 15641 leader_election.cc:336] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.026s (SENT)
I20260709 13:19:06.837653 17660 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:06.880239 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 83129.8us
I20260709 13:19:06.811854 17827 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5.
I20260709 13:19:06.880798 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:06.881863 17717 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:06.856254 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:06.840010 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:06.844856 17482 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:06.851236 17623 raft_consensus.cc:2804] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:06.814206 17834 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:19:06.854826 17716 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:06.888477 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 45993.8us
I20260709 13:19:06.890944 15493 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:06.892606 16203 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:06.837234 17574 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:06.931237 17582 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:06.931921 17582 raft_consensus.cc:697] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:06.931121 16203 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:06.932797 17582 consensus_queue.cc:237] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.883242 17754 raft_consensus.cc:740] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:06.901309 17845 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:06.909304 16232 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:06.934508 17754 consensus_queue.cc:260] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:06.935002 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 44149.5us
I20260709 13:19:06.935374 17574 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:06.935626 17355 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:06.935711 15493 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:06.907212 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 88464.7us
I20260709 13:19:06.933840 16203 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:06.936492 17688 raft_consensus.cc:2764] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 13:19:06.936829 15493 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:06.937130 17688 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:06.899574 17717 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:06.912464 17837 raft_consensus.cc:697] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:06.893457 17660 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:06.909584 17482 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:06.914994 17623 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:06.925333 16233 raft_consensus.cc:2393] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 6: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:06.926662 17576 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:19:06.930532 17851 log.cc:927] Time spent T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 13:19:06.911994 17807 raft_consensus.cc:697] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:06.938449 16203 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:06.948328 17576 raft_consensus.cc:697] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:06.946038 17623 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:06.946174 17221 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:19:06.956378 17846 log.cc:927] Time spent T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.329s	user 0.001s	sys 0.002s
I20260709 13:19:06.898470 17594 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:19:06.961689 16203 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:06.908433 17744 consensus_queue.cc:1059] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:19:06.942201 17717 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:06.910830 17611 consensus_queue.cc:1048] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:19:06.982069 17749 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:19:07.052488 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 249434us
I20260709 13:19:06.908823 17735 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.945070 17482 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:07.050535 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 223239us
W20260709 13:19:07.051429 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 89820.2us
W20260709 13:19:07.039151 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.053407 15568 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:07.057173 16203 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:07.058315 16203 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:06.933521 17739 consensus_queue.cc:237] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.947356 15599 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:06.942503 17355 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:19:07.064460 15669 raft_consensus.cc:1275] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:19:07.061391 17224 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:07.067010 15672 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:07.067476 17662 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:07.068897 17754 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:07.070088 17835 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:07.086493 17835 raft_consensus.cc:697] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:07.086267 17807 consensus_queue.cc:237] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:07.087531 17835 consensus_queue.cc:237] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:06.982933 17858 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.129s
I20260709 13:19:07.078450 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:07.090389 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:07.091727 17576 consensus_queue.cc:237] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:07.094035 17837 consensus_queue.cc:237] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:07.094571 17727 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:07.053014 17571 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:19:07.104380 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 207858us
W20260709 13:19:07.105085 15641 leader_election.cc:336] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.233s (SENT)
I20260709 13:19:07.105574 15641 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:07.105520 15642 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:07.108560 15641 leader_election.cc:336] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.253s (SENT)
I20260709 13:19:06.945096 17857 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.137s
W20260709 13:19:07.123551 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 186773us
I20260709 13:19:07.124794 15493 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
W20260709 13:19:07.127041 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:07.127241 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 73923.4us
I20260709 13:19:07.121407 15642 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:07.077351 17749 raft_consensus.cc:697] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:07.134255 17630 consensus_queue.cc:1059] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:07.135465 17630 raft_consensus.cc:3055] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:07.135872 17630 raft_consensus.cc:740] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:07.136744 17630 consensus_queue.cc:260] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:07.137754 17630 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:07.147521 15642 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:07.148959 17630 consensus_queue.cc:1059] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:07.101311 17727 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:07.120235 17844 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:19:07.171511 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 63489.8us
I20260709 13:19:07.127040 17827 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:07.174487 15567 leader_election.cc:336] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.820s (SENT)
I20260709 13:19:07.183620 15567 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:07.150693 17832 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:07.171160 17630 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:07.169332 17688 raft_consensus.cc:697] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
W20260709 13:19:07.230893 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:07.231217 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 103226us
I20260709 13:19:07.172562 17777 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:07.166949 17861 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:07.202961 17825 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:07.274503 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 337048us
I20260709 13:19:07.223603 17835 consensus_queue.cc:1059] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
W20260709 13:19:07.250768 17843 log.cc:927] Time spent T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.622s	user 0.001s	sys 0.003s
I20260709 13:19:07.292198 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:07.230702 17832 raft_consensus.cc:697] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:07.173184 17867 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:19:07.295256 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 170521us
W20260709 13:19:07.295449 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 120985us
I20260709 13:19:07.298364 15596 raft_consensus.cc:1275] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:07.309249 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:07.261339 17864 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:07.306288 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:07.305333 15600 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:07.310391 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:07.339938 15596 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:07.266920 15599 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:07.328037 15493 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:07.328758 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.330363 15603 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:19:07.293004 17865 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:07.334851 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:07.288784 17426 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:19:07.350960 15603 raft_consensus.cc:1275] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:19:07.349294 17826 consensus_meta.cc:287] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.745s	user 0.011s	sys 0.000s
I20260709 13:19:07.363643 17826 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 3. Current term is 4.
I20260709 13:19:07.364329 17844 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:07.329701 17735 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:07.366590 17739 raft_consensus.cc:3055] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:07.367019 17739 raft_consensus.cc:740] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:07.367831 17739 consensus_queue.cc:260] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:07.339977 17744 consensus_queue.cc:1048] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:19:07.353228 17827 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
W20260709 13:19:07.373250 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 45364us
I20260709 13:19:07.374370 15493 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:07.352468 17863 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:07.375473 15493 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:07.329100 17857 consensus_queue.cc:1059] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:19:07.299306 17749 consensus_queue.cc:237] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:07.369021 17739 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:07.397019 15596 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:07.383714 16231 raft_consensus.cc:1240] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:19:07.346382 17870 raft_consensus.cc:3055] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:07.398630 15642 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:07.383770 17877 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:07.402818 17835 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:07.422461 15600 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:07.437021 16238 raft_consensus.cc:1275] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEARNER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:19:07.348316 17688 consensus_queue.cc:237] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:07.403247 17832 consensus_queue.cc:237] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:07.408363 17870 raft_consensus.cc:740] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:07.434101 17824 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:07.401571 16232 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:07.329452 17867 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:07.450907 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 75506us
I20260709 13:19:07.452515 15493 leader_election.cc:400] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4.
I20260709 13:19:07.452831 15493 leader_election.cc:403] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:07.463914 15493 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:07.442919 17877 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:07.474165 15493 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:07.472935 17874 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4.
I20260709 13:19:07.459914 17426 consensus_queue.cc:1059] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:19:07.477150 15493 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
W20260709 13:19:07.464236 17871 log.cc:927] Time spent T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 13:19:07.457556 17824 raft_consensus.cc:697] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:07.489140 15674 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:07.456138 17510 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:19:07.501269 15603 raft_consensus.cc:1240] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:07.502398 17828 raft_consensus.cc:3055] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:07.502828 17828 raft_consensus.cc:740] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:07.502986 17876 consensus_queue.cc:1059] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:07.503842 17876 consensus_queue.cc:1243] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer 5498ef3e89a941e39c25f68cf64e6b0f log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:19:07.506386 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.507947 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:07.508203 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:07.508534 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:07.509033 15519 raft_consensus.cc:2393] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:07.509883 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:07.510169 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:07.487609 17877 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:07.510587 15528 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:07.511631 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:07.509052 17830 raft_consensus.cc:2764] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:19:07.512641 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:07.513391 15520 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:07.514608 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:07.538347 15676 raft_consensus.cc:1275] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:07.540318 17735 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.540896 17735 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:07.502519 17824 consensus_queue.cc:237] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:07.543083 17735 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:07.543087 15671 raft_consensus.cc:1275] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:07.553508 15520 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:19:07.555160 17867 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:07.549532 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:07.544597 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:19:07.541692 17623 consensus_meta.cc:287] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.556s	user 0.006s	sys 0.002s
I20260709 13:19:07.563763 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:07.564042 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:07.565148 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:07.566239 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:07.597653 15522 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:07.566669 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:07.573355 15567 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:07.567674 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:07.594996 15674 raft_consensus.cc:2410] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:07.595762 15673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57710 (request call id 478) took 1080 ms. Trace:
I20260709 13:19:07.598768 15526 raft_consensus.cc:2393] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:07.497480 17857 raft_consensus.cc:2764] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:19:07.598925 15527 raft_consensus.cc:2368] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 3. Current term is 4.
I20260709 13:19:07.598903 15673 rpcz_store.cc:276] 0709 13:19:06.515157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.516945 (+  1788us) service_pool.cc:224] Handling call
0709 13:19:07.125889 (+608944us) spinlock_profiling.cc:243] Waited 6.45 ms on lock 0x7b5800012398. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99d10a0 00007f5ce99d0984 00007f5ce2f4b6d4 00007f5ce2f4b662
0709 13:19:07.544465 (+418576us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b5800012398. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6e9c 00007f5ce18e6a5b 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99d10a0 00007f5ce99d0984 00007f5ce2f4b6d4 00007f5ce2f4b662
0709 13:19:07.595706 (+ 51241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18254976}
I20260709 13:19:07.599108 15523 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:07.603924 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:07.604117 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:07.604780 15519 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:07.604987 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:07.605796 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:07.606253 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:07.606940 15526 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:07.597245 15528 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 3: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:07.608207 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:07.609287 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:07.610033 15528 raft_consensus.cc:2368] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5.
I20260709 13:19:07.611136 15528 raft_consensus.cc:1275] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:07.612464 15528 raft_consensus.cc:1275] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:19:07.604704 15523 raft_consensus.cc:2368] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 3. Current term is 4.
I20260709 13:19:07.606941 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:07.620483 15528 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:19:07.537523 17482 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:07.652962 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:07.654199 16234 raft_consensus.cc:1240] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:07.623854 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:19:07.575901 17717 consensus_meta.cc:287] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.500s	user 0.006s	sys 0.004s
I20260709 13:19:07.624569 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:07.626920 17882 log.cc:927] Time spent T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:19:07.575706 17582 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:07.627470 17623 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:07.627895 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 54627.9us
I20260709 13:19:07.498672 17870 consensus_queue.cc:260] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:07.628476 17867 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:19:07.633602 17660 consensus_meta.cc:287] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.687s	user 0.003s	sys 0.004s
I20260709 13:19:07.545506 17835 raft_consensus.cc:3055] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:07.549372 17828 consensus_queue.cc:260] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:07.635921 17716 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:07.637203 17852 log.cc:927] Time spent T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.752s	user 0.007s	sys 0.004s
I20260709 13:19:07.644325 16239 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:07.646901 16240 raft_consensus.cc:1240] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 3. Current term is 4. Ops: []
W20260709 13:19:07.649726 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.651366 15642 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:07.654429 16237 raft_consensus.cc:1275] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:19:07.656358 16204 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
W20260709 13:19:07.714118 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 62842.7us
I20260709 13:19:07.666662 16238 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEARNER]: Advancing to term 4
I20260709 13:19:07.545228 17221 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.130s
I20260709 13:19:07.682601 17582 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:07.692600 17835 raft_consensus.cc:740] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:07.716678 15670 raft_consensus.cc:1240] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:19:07.666729 17865 raft_consensus.cc:697] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:07.717882 15674 raft_consensus.cc:1240] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:19:07.721599 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 65358.3us
I20260709 13:19:07.709864 16231 raft_consensus.cc:2393] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:07.665874 17717 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:07.724565 16204 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:07.727958 16204 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:07.729907 16235 raft_consensus.cc:1240] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:19:07.719808 17735 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.723969 17754 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:07.726536 17355 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:07.725730 17828 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:07.718732 17716 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:07.744916 17582 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:07.744935 17355 raft_consensus.cc:697] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:07.747884 17355 consensus_queue.cc:237] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:07.735049 17224 consensus_meta.cc:287] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.651s	user 0.011s	sys 0.000s
I20260709 13:19:07.741851 17749 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:07.742177 17777 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:07.693581 17660 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:07.747756 17887 log.cc:927] Time spent T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:19:07.752415 17754 raft_consensus.cc:697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:07.750146 17735 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:07.760651 17777 raft_consensus.cc:697] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:07.838812 17482 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:07.840298 17777 consensus_queue.cc:237] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:07.842713 17623 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:07.800557 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.665803 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:07.595894 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:07.641568 15524 raft_consensus.cc:2393] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:07.864063 15519 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:07.865417 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:07.663784 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:07.865999 15524 raft_consensus.cc:1240] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:07.669137 15521 raft_consensus.cc:2376] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4. Re-sending same reply.
I20260709 13:19:07.806880 15522 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:07.808444 15525 raft_consensus.cc:2368] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5.
W20260709 13:19:07.873695 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.808614 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:19:07.814566 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.815454 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:19:07.817152 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:07.818539 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.825623 16240 raft_consensus.cc:1275] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:07.827359 16238 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEARNER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:19:07.829139 16239 raft_consensus.cc:1275] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:19:07.834929 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:07.752156 17224 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:07.766371 17754 consensus_queue.cc:237] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:07.763330 17749 raft_consensus.cc:697] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:07.751065 17835 consensus_queue.cc:260] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:07.764914 17865 consensus_queue.cc:237] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:07.888245 15528 raft_consensus.cc:1275] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:07.878007 15670 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:07.893488 15493 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:07.899037 17660 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:07.901335 17749 consensus_queue.cc:237] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:07.902187 15528 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:07.904060 15524 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:19:07.905400 15526 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:07.907034 17863 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.907933 17863 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:07.912046 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.923204 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:07.927614 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:07.943818 15599 raft_consensus.cc:1275] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:19:07.947671 17870 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:07.948359 15599 raft_consensus.cc:1240] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:07.949747 15675 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
W20260709 13:19:07.951407 17878 log.cc:927] Time spent T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.413s	user 0.001s	sys 0.000s
I20260709 13:19:07.952265 15675 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:07.957266 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:19:07.958180 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 65088.6us
I20260709 13:19:07.959401 17482 consensus_queue.cc:1059] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:07.961164 17482 consensus_queue.cc:1059] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:07.962325 17889 raft_consensus.cc:3055] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:07.962769 17889 raft_consensus.cc:740] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:07.964306 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:07.964986 15493 leader_election.cc:400] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 5.
I20260709 13:19:07.965348 15493 leader_election.cc:403] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:19:07.966120 17877 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 5.
I20260709 13:19:07.971635 17835 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:07.971690 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:07.972621 15669 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:07.973610 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 51260.3us
I20260709 13:19:07.976246 17574 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:07.976855 16239 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:07.976112 16237 raft_consensus.cc:1240] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:19:07.937016 17623 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:07.937770 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:08.007272 15670 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:07.964273 17889 consensus_queue.cc:260] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:08.025949 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:08.028112 15599 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:08.031946 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 49800.8us
I20260709 13:19:08.025959 17729 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.957301 17863 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:08.026818 17889 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:08.032918 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.026103 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:08.052215 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:08.028179 15567 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:08.072269 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 107554us
W20260709 13:19:08.074281 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.075559 17832 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:08.076596 17832 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:08.082383 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:08.082774 17824 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.002s	sys 0.013s
I20260709 13:19:08.083603 17832 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:08.088622 15677 raft_consensus.cc:1275] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:08.020455 17660 consensus_queue.cc:1048] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.069s
W20260709 13:19:08.090530 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.101815 17729 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:08.107049 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 379067us
I20260709 13:19:08.115289 15493 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:08.115625 17716 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:08.115320 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:08.108741 15595 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:08.131752 15669 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
W20260709 13:19:08.134629 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:08.135488 15641 leader_election.cc:336] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.608s (SENT)
I20260709 13:19:08.136830 15602 raft_consensus.cc:1275] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:08.139945 15595 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:08.141855 17582 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:19:08.165515 17717 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:19:08.169414 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 112616us
I20260709 13:19:08.183089 15526 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:08.189942 17482 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:08.184568 17660 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.213295 15600 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:08.205787 15668 raft_consensus.cc:1275] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:19:08.235334 17807 consensus_queue.cc:1048] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:19:08.237459 17844 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:08.238056 17898 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:08.238399 17844 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:08.241372 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:08.242190 15596 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:08.243232 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:08.243999 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:08.245076 17900 log.cc:927] Time spent T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.322s	user 0.002s	sys 0.000s
I20260709 13:19:08.246115 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:08.247372 15491 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:19:08.222164 17823 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.014s	sys 0.005s
I20260709 13:19:08.266499 17623 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:08.267519 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 239496us
I20260709 13:19:08.267432 17623 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:08.268059 17867 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:08.268770 15491 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:08.269737 17867 consensus_queue.cc:1059] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:08.270098 17482 raft_consensus.cc:2804] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:08.270735 17482 raft_consensus.cc:697] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:08.271037 17717 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.271955 17877 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:08.271705 17717 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:08.271847 15675 raft_consensus.cc:1240] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:19:08.271057 17510 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:08.300328 15675 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:19:08.279158 17844 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:08.271862 17482 consensus_queue.cc:237] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:08.280578 17835 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:08.282797 17729 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:08.283360 16204 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:08.246133 17660 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:08.284107 17716 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:08.284767 17865 consensus_queue.cc:1059] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:19:08.285028 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 169933us
W20260709 13:19:08.273193 17823 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.015s	sys 0.006s
I20260709 13:19:08.293046 17872 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:08.300160 17717 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:08.277982 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 55562.9us
W20260709 13:19:08.304051 17900 log.cc:927] Time spent T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
W20260709 13:19:08.335719 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:08.339388 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 79780.5us
W20260709 13:19:08.372947 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:08.389156 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.377523 15493 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:08.392699 15602 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:08.308516 15599 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
W20260709 13:19:08.414275 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 145158us
W20260709 13:19:08.410962 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 143677us
I20260709 13:19:08.454931 17867 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:08.489817 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.483191 16204 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:08.409564 17877 raft_consensus.cc:697] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:08.405953 17582 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 13:19:08.412343 17889 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.392086 17835 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:08.572403 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 160231us
I20260709 13:19:08.573181 17574 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.455236 17865 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:19:08.588956 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 181452us
I20260709 13:19:08.410231 17716 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:08.616016 17580 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:08.631620 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 315461us
I20260709 13:19:08.557106 17777 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.547405 17898 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:19:08.652197 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.651968 15596 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
W20260709 13:19:08.658965 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 64739.8us
W20260709 13:19:08.653203 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 54774us
I20260709 13:19:08.573951 17574 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:08.659806 17901 log.cc:927] Time spent T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:19:08.603685 17717 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.656675 17865 raft_consensus.cc:697] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:08.589663 17867 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:08.720557 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 146494us
I20260709 13:19:08.632124 17872 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:08.739053 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.691697 17889 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:08.747259 15566 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:08.706446 17777 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:08.705875 17580 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:09.218884 15569 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.007s	sys 0.008s
W20260709 13:19:09.396435 15569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.009s	sys 0.009s
W20260709 13:19:09.382087 16203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.016s	sys 0.042s
I20260709 13:19:08.909593 17827 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:09.080931 15567 leader_election.cc:336] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.756s (SENT)
I20260709 13:19:08.754400 15493 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:09.398128 16203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.016s	sys 0.044s
W20260709 13:19:09.218194 15603 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.000s	sys 0.016s
W20260709 13:19:09.396569 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 675246us
W20260709 13:19:09.357059 15491 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.014s	sys 0.005s
W20260709 13:19:08.798843 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:08.732581 17870 consensus_meta.cc:287] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.784s	user 0.009s	sys 0.006s
I20260709 13:19:09.428054 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:09.013981 17914 log.cc:927] Time spent T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.447s	user 0.000s	sys 0.000s
W20260709 13:19:08.821599 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 155281us
W20260709 13:19:09.044625 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.398918 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:09.081519 17932 log.cc:927] Time spent T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.378s	user 0.000s	sys 0.000s
W20260709 13:19:08.921068 17882 log.cc:927] Time spent T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
W20260709 13:19:09.602073 15491 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.022s	sys 0.010s
I20260709 13:19:09.016367 17716 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:09.084833 17510 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:09.620097 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:09.620664 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.620491 17827 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.083889 17865 consensus_queue.cc:237] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.622246 17952 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.622906 17827 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:09.080047 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 282587us
I20260709 13:19:08.754276 17929 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.574741 17946 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:09.602252 15603 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.984s	user 0.002s	sys 0.029s
W20260709 13:19:09.086342 17893 log.cc:927] Time spent T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:19:09.082446 17933 log.cc:927] Time spent T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:19:09.629844 15520 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:09.012048 17913 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:09.630112 17926 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:09.630870 17913 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:09.492085 15492 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.003s	sys 0.008s
W20260709 13:19:09.562585 16233 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.000s	sys 0.011s
W20260709 13:19:09.631768 15492 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.006s	sys 0.008s
I20260709 13:19:08.921260 17355 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:08.767740 17623 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:09.091609 15672 raft_consensus.cc:1275] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:19:09.013028 17569 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:09.637715 17569 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.639035 17580 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:09.639998 17569 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:09.493608 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:09.602654 17574 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:09.615917 17935 consensus_queue.cc:1048] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:09.617087 17940 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:09.640900 15598 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:19:09.641151 17940 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:09.629701 17716 raft_consensus.cc:1064] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:19:09.641564 17940 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:09.642683 17867 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:09.636111 15603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 1615) took 1389 ms. Trace:
I20260709 13:19:09.279685 17835 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:09.622798 17952 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:09.606930 15567 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:09.602026 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 929331us
I20260709 13:19:08.797405 17660 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:09.013480 17930 log.cc:927] Time spent T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 13:19:09.105253 17872 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:08.731607 17717 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.563448 17823 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.086105 17928 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:09.633335 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.626005 17827 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:09.635129 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:09.633786 16233 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.000s	sys 0.012s
I20260709 13:19:09.630237 16205 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:09.629621 17948 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.636722 15672 pending_rounds.cc:85] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting all ops after (but not including) 2
I20260709 13:19:09.086695 17877 consensus_queue.cc:237] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.646865 17355 raft_consensus.cc:697] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:09.656577 17827 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.657028 17576 raft_consensus.cc:3055] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:09.657482 17576 raft_consensus.cc:740] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:09.658571 17576 consensus_queue.cc:260] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.659271 16233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 857) took 1070 ms. Trace:
W20260709 13:19:09.661689 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 990792us
I20260709 13:19:09.620025 17510 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.659708 16233 rpcz_store.cc:276] 0709 13:19:08.589170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.603008 (+ 13838us) service_pool.cc:224] Handling call
0709 13:19:08.614669 (+ 11661us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:08.677092 (+ 62423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:08.682539 (+  5447us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:08.922485 (+239946us) log.cc:844] Serialized 52 byte log entry
0709 13:19:09.657391 (+734906us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:09.657446 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:09.657524 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:09.658998 (+  1474us) raft_consensus.cc:1685] finished
0709 13:19:09.659022 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:09.659191 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70423,"raft.queue_time_us":101290,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":78509,"spinlock_wait_cycles":19328,"thread_start_us":728308,"threads_started":1,"wal-append.queue_time_us":708498}
I20260709 13:19:09.657174 17355 consensus_queue.cc:237] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:09.663839 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 582957us
I20260709 13:19:09.665162 17827 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:09.665774 17827 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.662746 17929 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.647733 17870 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:09.666507 17827 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.647606 17946 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.667060 17948 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.667783 17576 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:09.667929 17913 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:09.668594 17913 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:09.669036 17946 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:09.656126 17777 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:09.648787 17623 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.669905 17948 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:09.670380 17823 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:09.670790 15493 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:09.670555 17623 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.643642 15603 rpcz_store.cc:276] 0709 13:19:08.242603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.245688 (+  3085us) service_pool.cc:224] Handling call
0709 13:19:08.246092 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:08.246518 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:19:08.539114 (+292596us) log.cc:844] Serialized 14 byte log entry
0709 13:19:09.631839 (+1092725us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:09.631891 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:09.631935 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:09.632000 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:09.632111 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4149,"prepare.queue_time_us":470210,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":17028,"raft.run_cpu_time_us":48270,"raft.run_wall_time_us":1393151,"spinlock_wait_cycles":68215424,"thread_start_us":1293384,"threads_started":2,"wal-append.queue_time_us":875324}
I20260709 13:19:09.655740 15672 pending_rounds.cc:107] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:19:09.643733 17716 consensus_queue.cc:237] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:09.676851 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 46778.4us
I20260709 13:19:09.672811 16237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 856) took 1693 ms. Trace:
I20260709 13:19:09.684881 15493 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:09.654605 17867 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:09.666219 15567 leader_election.cc:336] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.897s (SENT)
I20260709 13:19:09.667929 17938 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.684191 17953 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.673589 17717 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:09.680357 16204 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:09.699947 16237 rpcz_store.cc:276] 0709 13:19:07.979376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.979529 (+   153us) service_pool.cc:224] Handling call
0709 13:19:07.979984 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:07.981118 (+  1134us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:07.981159 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:07.982032 (+   873us) log.cc:844] Serialized 26 byte log entry
0709 13:19:08.215724 (+233692us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:08.215769 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:08.215846 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:09.672598 (+1456752us) raft_consensus.cc:1685] finished
0709 13:19:09.672619 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:09.672744 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":162,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":287,"thread_start_us":188932,"threads_started":1,"wal-append.queue_time_us":396472}
I20260709 13:19:09.679351 17510 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:09.679836 17929 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:09.687970 17823 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:09.717321 15567 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:09.684731 17835 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:09.674384 17827 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:09.700860 15672 raft_consensus.cc:2889] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:09.679766 17777 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:09.685209 17870 consensus_queue.cc:1059] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:19:09.727061 17717 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:09.727864 15520 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1603) took 2158 ms (client timeout 1578 ms). Trace:
I20260709 13:19:09.728374 17623 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:09.728794 15671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 498) took 2427 ms. Trace:
W20260709 13:19:09.731352 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 46569.8us
I20260709 13:19:09.731472 17510 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:09.731760 17899 log.cc:927] Time spent T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 1.826s	user 0.017s	sys 0.015s
I20260709 13:19:09.745997 17688 raft_consensus.cc:3055] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:09.746479 17688 raft_consensus.cc:740] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:09.747550 17688 consensus_queue.cc:260] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.748315 17944 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:09.732849 17952 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:09.733992 17951 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.712546 17872 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:09.749910 17951 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:09.752319 17951 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:09.748915 17688 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:09.753401 17955 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.754117 17955 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:09.754588 15595 raft_consensus.cc:1275] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:19:09.756830 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 495) took 2102 ms. Trace:
I20260709 13:19:09.735107 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:09.757252 15522 rpcz_store.cc:276] 0709 13:19:07.654876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.755844 (+2100968us) service_pool.cc:224] Handling call
0709 13:19:09.756274 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:09.756500 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:09.756526 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:09.756564 (+    38us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:09.756590 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:09.756646 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:09.756756 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:09.720840 17938 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.760361 17889 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:09.761556 17938 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:09.762349 17716 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:09.763098 17943 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:09.762996 17716 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.765255 17716 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:09.669181 17913 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:09.737846 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 57636.7us
W20260709 13:19:09.740084 15520 rpcz_store.cc:269] 0709 13:19:07.569015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.595641 (+ 26626us) service_pool.cc:224] Handling call
0709 13:19:09.727634 (+2131993us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b54003cd7d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:09.727773 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4036815232}
W20260709 13:19:09.740667 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 139280us
I20260709 13:19:09.742826 15671 rpcz_store.cc:276] 0709 13:19:07.301860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.303405 (+  1545us) service_pool.cc:224] Handling call
0709 13:19:07.542741 (+239336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:07.602876 (+ 60135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:09.728406 (+2125530us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b5400460850. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:09.728620 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":467267840}
I20260709 13:19:09.688719 17945 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
W20260709 13:19:09.730451 17941 log.cc:927] Time spent T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:19:09.765872 17827 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:09.771679 15566 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:09.772749 17777 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.773300 17950 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:09.738802 17867 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:09.784432 17867 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:09.756711 17955 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:09.669993 17928 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:09.824783 17928 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:09.825366 17928 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.774443 15493 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:09.826924 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 52564.5us
I20260709 13:19:09.828987 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:09.775290 16205 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:09.830070 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:19:09.830526 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 55323.3us
I20260709 13:19:09.830821 15521 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:09.831432 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:09.832227 15522 raft_consensus.cc:2393] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:09.832751 17735 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:09.781821 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 115645us
I20260709 13:19:09.774260 17929 consensus_queue.cc:1048] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:19:09.783694 17824 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:09.834144 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:09.834261 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:09.834718 17938 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:09.835098 15524 raft_consensus.cc:2410] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:19:09.833460 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 61880.2us
I20260709 13:19:09.705077 17953 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.835664 17872 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:09.836707 15600 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:07.170750 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:09.838310 15672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49094 (request call id 1688) took 1185 ms. Trace:
I20260709 13:19:09.838790 15672 rpcz_store.cc:276] 0709 13:19:08.652840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.656881 (+  4041us) service_pool.cc:224] Handling call
0709 13:19:08.731518 (+ 74637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:09.779563 (+1048045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:09.838220 (+ 58657us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:09.840952 17940 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.817072 17951 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:09.817621 17889 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.819439 17913 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:09.819921 17777 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.785892 17827 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:09.825736 17950 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.833320 17735 raft_consensus.cc:2804] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:09.833889 17824 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.833264 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:09.785171 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1604) took 2215 ms. Trace:
I20260709 13:19:09.815661 17958 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:09.826839 15520 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:19:09.843214 15566 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:09.847132 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:09.847430 15421 catalog_manager.cc:5697] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 2 to 3, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 3 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:09.848729 17915 log.cc:927] Time spent T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 1.228s	user 0.004s	sys 0.016s
I20260709 13:19:09.850301 16205 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:09.743062 17717 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.851770 17721 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:09.837682 17953 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:09.850353 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:09.825871 17928 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.860435 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:09.861318 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:09.862010 17929 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.780332 17482 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.902343 15524 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:09.864449 16237 raft_consensus.cc:1240] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 4. Current term is 5. Ops: []
W20260709 13:19:09.865346 15641 leader_election.cc:336] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.618s (SENT)
I20260709 13:19:09.771010 17943 raft_consensus.cc:697] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:09.779942 17623 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.780650 17716 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:09.847322 15567 leader_election.cc:336] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.927s (SENT)
I20260709 13:19:09.875142 15520 pending_rounds.cc:85] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 4
I20260709 13:19:09.922132 15520 pending_rounds.cc:107] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:19:09.922449 15520 raft_consensus.cc:2889] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:09.922080 15519 raft_consensus.cc:2368] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 4. Current term is 5.
W20260709 13:19:09.887465 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 44334.9us
I20260709 13:19:09.898627 15525 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:09.873565 17735 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:09.873930 15527 rpcz_store.cc:276] 0709 13:19:07.569267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.617859 (+ 48592us) service_pool.cc:224] Handling call
0709 13:19:07.866980 (+249121us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:07.897454 (+ 30474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:07.897491 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:07.899257 (+  1766us) log.cc:844] Serialized 26 byte log entry
0709 13:19:07.984034 (+ 84777us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:07.984076 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.984144 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:08.291809 (+307665us) raft_consensus.cc:1685] finished
0709 13:19:08.291832 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:09.784757 (+1492925us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b54003cdcd0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:09.784988 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12067,"raft.queue_time_us":29902,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":256,"spinlock_wait_cycles":571970944,"thread_start_us":84313,"threads_started":1,"wal-append.queue_time_us":285333,"wal-append.run_cpu_time_us":2544,"wal-append.run_wall_time_us":1102028}
I20260709 13:19:09.921793 15567 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:09.927289 15526 raft_consensus.cc:2368] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 4. Current term is 5.
I20260709 13:19:09.901464 17582 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:09.909667 17872 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:09.944222 15669 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:19:09.944692 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 79436us
I20260709 13:19:09.945531 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:09.933817 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 83648.7us
I20260709 13:19:09.937376 17510 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:09.948211 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:09.938433 17885 log.cc:927] Time spent T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 2.400s	user 0.018s	sys 0.025s
I20260709 13:19:09.938577 17717 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:09.951226 16205 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:09.928267 17735 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:09.940265 17824 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:09.901616 17951 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:09.941066 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:09.940109 17948 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:09.940193 17920 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.942219 17947 consensus_queue.cc:1048] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:19:09.942988 17946 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:09.945693 17957 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:09.936297 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 89067.5us
I20260709 13:19:09.944140 17942 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:09.930840 17716 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:10.012166 15523 raft_consensus.cc:2410] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:19:09.945567 15641 leader_election.cc:336] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.707s (SENT)
I20260709 13:19:09.914956 17953 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.954310 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:09.913620 17938 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:09.911258 17576 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:09.960222 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:09.904246 17889 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.937255 17876 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.958323 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:10.024362 15567 leader_election.cc:336] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.991s (SENT)
I20260709 13:19:09.964303 15528 raft_consensus.cc:2393] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:19:09.941864 17925 log.cc:927] Time spent T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 1.326s	user 0.010s	sys 0.013s
I20260709 13:19:09.911109 17950 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:09.968432 15525 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:09.999307 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:09.959537 17940 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:09.921474 17482 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:10.034144 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:09.921665 17929 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:10.006011 15675 raft_consensus.cc:3055] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:09.948320 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:09.960225 17688 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:10.016228 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:10.012663 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 61511.1us
I20260709 13:19:09.944269 17872 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:10.019330 15641 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:10.024860 15567 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:09.964932 17660 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:19:09.962555 17623 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:10.026641 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:10.007943 15521 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:10.025419 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:10.039156 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:10.039361 15519 raft_consensus.cc:2393] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:10.036949 15675 raft_consensus.cc:740] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:10.038089 16205 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:10.028914 17729 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.041121 15676 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:10.029455 17952 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.043228 16239 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:10.010571 17582 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:19:10.038717 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:10.044385 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:10.041059 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:10.045275 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:10.051254 16238 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:10.026732 17938 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
W20260709 13:19:10.043890 17898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.309s	user 0.010s	sys 0.018s
I20260709 13:19:10.041119 17355 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:19:10.046828 17961 log.cc:927] Time spent T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:19:10.047912 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:10.048059 17950 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:10.048595 17867 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:10.049459 17482 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:10.046201 16235 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEARNER]: Advancing to term 4
I20260709 13:19:10.045706 15599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 1619) took 1101 ms. Trace:
I20260709 13:19:10.005448 17920 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:10.063264 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:10.063515 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:10.105161 15675 consensus_queue.cc:260] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:10.105734 17951 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:10.063130 17729 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:10.106631 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:10.107054 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:10.108168 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:10.108309 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:10.108566 17729 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:10.109346 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:10.034240 17943 consensus_queue.cc:237] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:10.067413 17929 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:10.068064 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:10.069263 17623 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:10.113061 16237 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:09.926415 17958 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:10.114078 16205 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:10.098594 17952 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:10.116420 15642 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:19:10.113184 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 94034.6us
I20260709 13:19:10.070302 17754 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:10.140358 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:10.070817 17967 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:10.071231 17827 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:10.046320 17876 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:10.072005 17716 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:10.072360 17777 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:10.059834 15528 raft_consensus.cc:2376] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6. Re-sending same reply.
W20260709 13:19:10.069280 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 123797us
W20260709 13:19:10.087010 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:10.151871 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.033500 17953 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:10.159191 15567 connection.cc:256] RPC call timeout handler was delayed by -1.4920657010000014s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2158s for a response
I20260709 13:19:10.159755 17953 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:10.051090 17940 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:10.161087 17940 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:10.161510 17920 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:10.162186 17939 log.cc:927] Time spent T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.669s	user 0.007s	sys 0.005s
I20260709 13:19:10.162442 16231 raft_consensus.cc:1240] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:19:10.049520 17928 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:10.092896 15524 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:10.159641 17835 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:10.165273 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:10.165678 17777 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.166301 17777 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:10.167573 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:10.168390 17777 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:10.169358 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:10.085899 16236 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:19:10.170099 15669 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:10.170718 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:10.171636 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:10.171823 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:10.172122 17952 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:10.031360 17942 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:10.145723 17872 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:10.140930 17744 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:19:10.159663 15567 connection.cc:256] RPC call timeout handler was delayed by -1.4689596379998875s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24680000000000002s for a response
I20260709 13:19:10.173758 15525 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:19:10.168313 16234 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:10.074177 17898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.343s	user 0.010s	sys 0.019s
I20260709 13:19:10.070874 17889 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:10.033293 17946 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:10.141505 15421 catalog_manager.cc:5697] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 1 to 2, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 2 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:10.102913 15602 raft_consensus.cc:1240] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:10.060918 17688 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:10.144631 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:10.145233 17947 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.119695 17355 raft_consensus.cc:697] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:10.119448 17935 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:19:10.138758 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:10.114104 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:10.138247 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:10.106539 15675 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:10.144654 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:10.148602 15670 raft_consensus.cc:2393] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 5: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:10.132174 15521 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:10.106722 15601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 1597) took 2800 ms. Trace:
I20260709 13:19:10.157240 15595 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:19:10.041585 17957 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:10.158043 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.174866 16239 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:10.095031 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:10.176954 15568 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:10.177983 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:10.178790 15524 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:10.204813 15596 raft_consensus.cc:2410] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:10.153347 15642 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:10.229713 16235 raft_consensus.cc:1275] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEARNER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:19:10.234630 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:10.235110 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:10.235512 15641 connection.cc:256] RPC call timeout handler was delayed by -1.1756381119998878s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.221s for a response
W20260709 13:19:10.235831 15641 connection.cc:256] RPC call timeout handler was delayed by -1.1267943620000551s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3151s for a response
W20260709 13:19:10.236112 15641 connection.cc:256] RPC call timeout handler was delayed by -1.0709817729998576s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2222s for a response
W20260709 13:19:10.234022 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 57136.7us
I20260709 13:19:10.143326 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:10.238931 15597 raft_consensus.cc:1240] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:19:10.159621 17876 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:10.139694 17938 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:19:10.239857 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:10.142778 17945 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:19:10.240957 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 90961.8us
W20260709 13:19:10.249941 15567 leader_election.cc:336] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.103s (SENT)
I20260709 13:19:10.250452 15567 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:10.253489 17872 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.259747 17872 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:10.255769 17827 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:10.256196 17953 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.232359 15601 rpcz_store.cc:276] 0709 13:19:07.297949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.331636 (+ 33687us) service_pool.cc:224] Handling call
0709 13:19:07.331942 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:07.338872 (+  6930us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:07.338902 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:07.351232 (+ 12330us) log.cc:844] Serialized 26 byte log entry
0709 13:19:07.440731 (+ 89499us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:07.440771 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.440836 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:09.886723 (+2445887us) spinlock_profiling.cc:243] Waited 56.6 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:19:10.073093 (+186370us) raft_consensus.cc:1685] finished
0709 13:19:10.073114 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:10.098078 (+ 24964us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6567,"raft.queue_time_us":14416,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"spinlock_wait_cycles":130202368,"thread_start_us":55376,"threads_started":1,"wal-append.queue_time_us":143677}
I20260709 13:19:10.258972 15521 raft_consensus.cc:1275] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:19:10.178362 17744 raft_consensus.cc:697] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:10.326651 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:10.327880 15676 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:10.177999 17898 raft_consensus.cc:697] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:10.335206 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:10.328451 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:10.326265 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 76391.2us
I20260709 13:19:10.327996 16234 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:10.327852 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:10.205338 15598 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:19:10.338275 17355 consensus_queue.cc:237] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:10.340400 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.341784 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:10.342072 15524 raft_consensus.cc:1275] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:19:10.342693 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:10.343326 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:10.343900 15521 raft_consensus.cc:1240] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:10.261883 17872 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:10.344544 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:10.343341 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:10.346835 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:10.347397 15601 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:19:10.348065 16233 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:19:10.348667 15567 leader_election.cc:336] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.183s (SENT)
I20260709 13:19:10.349205 15567 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:10.345589 15521 raft_consensus.cc:1275] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:19:10.343823 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:10.348335 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:10.262028 15603 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
W20260709 13:19:10.262879 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.262463 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:10.372200 17876 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:10.372474 15641 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:10.372639 17716 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:10.372826 17876 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:10.373194 17716 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:10.249676 17826 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.274s	user 0.011s	sys 0.038s
W20260709 13:19:10.375017 17826 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.470s	user 0.012s	sys 0.039s
I20260709 13:19:10.375967 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:10.263119 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:10.161799 17940 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:10.375413 17716 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:10.098682 15599 rpcz_store.cc:276] 0709 13:19:08.942739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.994826 (+ 52087us) service_pool.cc:224] Handling call
0709 13:19:09.828296 (+833470us) spinlock_profiling.cc:243] Waited 83.8 ms on lock 0x7b5800113498. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99e4fc0 00007f5ce99cf929 00007f5ce99cf3d2 00007f5ce2f46bd4
0709 13:19:09.828516 (+   220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:09.828805 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:09.829141 (+   336us) log.cc:844] Serialized 14 byte log entry
0709 13:19:10.025781 (+196640us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:10.028985 (+  3204us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:10.031861 (+  2876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.035792 (+  3931us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:10.043518 (+  7726us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2751,"prepare.queue_time_us":5838,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":10360,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":11365,"spinlock_wait_cycles":228083840,"thread_start_us":181982,"threads_started":1,"wal-append.queue_time_us":289527}
I20260709 13:19:10.496840 15675 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:10.240599 15672 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:19:10.434238 15642 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:10.242231 17867 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:19:10.507296 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:10.442287 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 63983.7us
W20260709 13:19:10.486068 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 113726us
W20260709 13:19:10.442068 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 93456.7us
W20260709 13:19:10.531589 15567 connection.cc:256] RPC call timeout handler was delayed by -1.286493672000006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2136s for a response
I20260709 13:19:10.494983 15493 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:10.518807 17951 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.525753 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:10.526520 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 92361us
W20260709 13:19:10.372072 17949 log.cc:927] Time spent T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.878s	user 0.006s	sys 0.007s
I20260709 13:19:10.444723 17623 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.506325 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:10.507252 17942 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:10.325642 17735 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:10.442948 17569 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.164s
I20260709 13:19:10.268286 17947 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:10.505754 17920 consensus_queue.cc:1059] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.102s
I20260709 13:19:10.441376 17889 consensus_queue.cc:1048] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.180s
I20260709 13:19:10.529181 15601 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:19:10.240559 15668 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
W20260709 13:19:10.648012 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:10.532330 15567 connection.cc:256] RPC call timeout handler was delayed by -1.0973897529997885s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
W20260709 13:19:10.702135 15567 connection.cc:256] RPC call timeout handler was delayed by -1.04562339600011s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1816s for a response
I20260709 13:19:10.625448 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:10.262639 17827 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:10.442687 17660 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:10.443272 17948 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:10.394603 17872 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.695478 15675 raft_consensus.cc:2410] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:19:10.500941 17929 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.441012 17835 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.715476 17872 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:10.716224 17355 raft_consensus.cc:3055] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:10.716080 17835 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:10.716378 17623 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:10.717760 17872 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:10.718667 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.705564 17826 consensus_queue.cc:1048] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:19:10.718600 17872 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:10.547047 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:10.718986 15567 leader_election.cc:336] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.786s (SENT)
I20260709 13:19:10.705696 17827 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:10.720036 15567 leader_election.cc:336] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.816s (SENT)
I20260709 13:19:10.720459 15567 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:10.699790 16237 raft_consensus.cc:1240] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:10.703310 15670 raft_consensus.cc:2410] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:10.443140 17688 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:10.723443 15670 raft_consensus.cc:1275] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:19:10.723587 16232 raft_consensus.cc:2393] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:10.440759 17952 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:10.724110 15674 raft_consensus.cc:1275] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:19:10.721455 17569 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:10.724323 17952 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:10.725536 15567 leader_election.cc:336] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.894s (SENT)
I20260709 13:19:10.726063 15567 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:10.727218 17569 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:19:10.728077 15567 leader_election.cc:336] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.966s (SENT)
I20260709 13:19:10.716675 17355 raft_consensus.cc:740] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:10.425137 17611 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:10.607849 17940 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:10.262961 17953 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:10.729822 17355 consensus_queue.cc:260] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:10.731240 17355 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:10.731542 17953 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:10.731689 17957 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:10.732239 15676 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:10.520316 17946 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:10.598846 17876 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:10.546630 17716 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:10.744359 15677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 1732) took 2831 ms. Trace:
I20260709 13:19:10.714929 17929 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:10.718194 17835 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:10.409103 17945 raft_consensus.cc:697] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:10.727928 17735 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:10.601279 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:10.748792 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 188217us
I20260709 13:19:10.595414 16231 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEARNER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:10.748054 15677 rpcz_store.cc:276] 0709 13:19:07.913455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.930512 (+ 17057us) service_pool.cc:224] Handling call
0709 13:19:07.931006 (+   494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:08.089088 (+158082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.732985 (+2643897us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54001540d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:10.744288 (+ 11303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":361241472}
I20260709 13:19:10.604743 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:10.441167 17928 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:10.722196 17947 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:10.745163 17623 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:10.700183 17867 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:10.825821 17958 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:10.829360 15527 raft_consensus.cc:1275] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:19:10.799144 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:10.859481 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 507) took 1001 ms. Trace:
I20260709 13:19:10.828083 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:19:10.829891 15600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53310 (request call id 1621) took 1513 ms (client timeout 1659 ms). Trace:
I20260709 13:19:10.865083 15527 rpcz_store.cc:276] 0709 13:19:09.857620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.828683 (+971063us) service_pool.cc:224] Handling call
0709 13:19:10.829007 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:10.859200 (+ 30193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.859422 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:10.865855 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 61858.1us
W20260709 13:19:10.865401 15600 rpcz_store.cc:269] 0709 13:19:09.316757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.316911 (+   154us) service_pool.cc:224] Handling call
0709 13:19:09.734911 (+418000us) spinlock_profiling.cc:243] Waited 3.82 ms on lock 0x7b5800113498. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99d10a0 00007f5ce99d0984 00007f5ce2f4b6d4 00007f5ce2f4b662
0709 13:19:10.829681 (+1094770us) spinlock_profiling.cc:243] Waited 77.9 ms on lock 0x7b5400439754. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:10.829808 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188043392}
I20260709 13:19:10.852406 17827 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:10.867519 16205 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:10.747644 17716 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:10.857993 15599 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:10.858798 17867 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:10.880777 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:10.602128 17898 consensus_queue.cc:237] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:10.889386 15597 raft_consensus.cc:2410] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:10.865880 15596 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:10.889568 15641 leader_election.cc:400] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5.
I20260709 13:19:10.745750 17951 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:10.888469 17716 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:10.894199 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:10.894578 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:10.868101 17928 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:10.900880 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 508) took 1043 ms. Trace:
I20260709 13:19:10.901293 15527 rpcz_store.cc:276] 0709 13:19:09.857877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.878088 (+1020211us) service_pool.cc:224] Handling call
0709 13:19:10.892211 (+ 14123us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:10.900561 (+  8350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:10.900588 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:10.900630 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:10.900661 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.900717 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:10.900827 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:10.895423 17942 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:10.602480 17744 consensus_queue.cc:237] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:10.895676 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:10.904222 15597 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:19:10.905161 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:10.905753 15597 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:10.906682 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:10.895453 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:10.858888 17945 consensus_queue.cc:237] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:10.908236 15600 raft_consensus.cc:2410] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:19:10.897506 17929 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:10.870374 15672 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:19:10.898283 17827 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:19:10.921639 15642 connection.cc:256] RPC call timeout handler was delayed by -1.0450370630001089s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17s for a response
W20260709 13:19:10.889475 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 160184us
I20260709 13:19:10.894457 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:10.888652 17947 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:10.897312 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:10.922729 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.894590 17951 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:10.929200 15596 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:10.925705 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 511) took 1066 ms. Trace:
I20260709 13:19:10.930621 15527 rpcz_store.cc:276] 0709 13:19:09.859126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.902630 (+1043504us) service_pool.cc:224] Handling call
0709 13:19:10.911375 (+  8745us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:10.922416 (+ 11041us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:10.922442 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:10.925437 (+  2995us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:10.925480 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.925536 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:10.925651 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:10.932669 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.938297 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:10.938388 15567 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.158s (SENT) [suppressed 22 similar messages]
I20260709 13:19:10.905596 17942 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:10.936094 15598 raft_consensus.cc:2410] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:19:10.944056 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:10.945343 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 73219.5us
I20260709 13:19:10.925274 17947 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:10.971994 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:10.973146 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 512) took 1113 ms. Trace:
I20260709 13:19:10.982889 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:10.983878 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:10.984377 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:10.985863 17947 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:11.018072 17952 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:10.985616 17953 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:10.986303 17826 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:10.987316 15641 leader_election.cc:400] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 3. Current term is 4.
I20260709 13:19:10.989017 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:10.990031 17876 consensus_queue.cc:1048] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:10.990669 17754 consensus_queue.cc:1048] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:10.974999 15599 raft_consensus.cc:2410] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:19:10.995168 17928 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:10.997900 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:11.002728 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1568) took 1174 ms. Trace:
W20260709 13:19:11.006649 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 557335us
I20260709 13:19:11.007594 15527 rpcz_store.cc:276] 0709 13:19:09.859411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.934086 (+1074675us) service_pool.cc:224] Handling call
0709 13:19:10.942372 (+  8286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:10.947852 (+  5480us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:10.972814 (+ 24962us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:10.972864 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:10.972901 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.972962 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:10.973090 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:11.015035 15567 leader_election.cc:336] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.468s (SENT)
I20260709 13:19:11.016096 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:11.016273 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:10.972892 17978 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.926707 15597 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:19:11.018791 15598 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:11.056474 15523 rpcz_store.cc:276] 0709 13:19:09.828235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.833767 (+  5532us) service_pool.cc:224] Handling call
0709 13:19:11.002418 (+1168651us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b5400260f54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:11.002626 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":403378432}
I20260709 13:19:10.974661 17611 consensus_queue.cc:1059] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:19:11.059031 15597 raft_consensus.cc:606] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:11.078752 15421 catalog_manager.cc:5697] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 4 to 5, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 5 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:11.079979 15597 raft_consensus.cc:614] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:11.081281 17942 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:11.082409 17978 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:11.083851 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:11.084496 15599 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:11.084584 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1585) took 1233 ms. Trace:
I20260709 13:19:11.084919 17978 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:11.029368 17729 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:11.066713 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:11.088064 16236 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:11.069665 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:11.071233 17754 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:11.081198 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:11.089679 15597 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
W20260709 13:19:11.066457 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 79292.1us
I20260709 13:19:11.086082 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:11.095109 15493 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:11.101011 15527 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:11.101820 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 513) took 1242 ms. Trace:
I20260709 13:19:11.102167 15527 rpcz_store.cc:276] 0709 13:19:09.859688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.063125 (+1203437us) service_pool.cc:224] Handling call
0709 13:19:11.100635 (+ 37510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.101536 (+   901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.101757 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:11.110266 15600 raft_consensus.cc:2393] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:11.091022 15519 rpcz_store.cc:276] 0709 13:19:09.851168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.933729 (+ 82561us) service_pool.cc:224] Handling call
0709 13:19:11.084357 (+1150628us) spinlock_profiling.cc:243] Waited 71 ms on lock 0x7b5400262854. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:11.084518 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":163389696}
I20260709 13:19:11.113374 15493 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:19:11.121642 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.121824 15493 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:11.110152 17990 consensus_queue.cc:1059] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:11.125551 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:11.128255 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:11.131479 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:11.145605 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 54861.8us
W20260709 13:19:11.146293 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 134192us
I20260709 13:19:11.149024 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:11.147853 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:11.149744 16238 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:11.121343 17952 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:11.157184 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:11.158713 15519 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:11.124778 17951 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:11.183113 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 201621us
I20260709 13:19:11.157153 17952 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:11.131886 17623 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:11.147275 17889 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:11.128113 17929 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:11.192227 15602 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:11.192929 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:11.134433 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:11.198904 15603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53310 (request call id 1624) took 1239 ms. Trace:
I20260709 13:19:11.189352 17951 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:11.229084 15669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49100 (request call id 1741) took 1262 ms. Trace:
I20260709 13:19:11.232889 15603 rpcz_store.cc:276] 0709 13:19:09.959826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.959945 (+   119us) service_pool.cc:224] Handling call
0709 13:19:11.198733 (+1238788us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b540032c3d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:11.198839 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":556223872}
I20260709 13:19:11.239013 15603 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:11.240489 15603 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:11.190011 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:11.232717 15567 leader_election.cc:336] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 3.004s (SENT)
I20260709 13:19:11.242007 15567 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:11.196035 15568 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:11.235605 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:11.232965 17929 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:11.242620 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:11.243062 17929 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:11.160486 17826 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:11.248457 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:11.194321 17960 log.cc:927] Time spent T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 1.080s	user 0.002s	sys 0.001s
I20260709 13:19:11.237458 15669 rpcz_store.cc:276] 0709 13:19:09.941277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.945264 (+  3987us) service_pool.cc:224] Handling call
0709 13:19:11.203697 (+1258433us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54001a8954. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:11.203814 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":504632064}
I20260709 13:19:11.247627 15523 raft_consensus.cc:2376] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4. Re-sending same reply.
I20260709 13:19:11.213485 17952 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:11.244139 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:11.265513 15677 raft_consensus.cc:2410] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:19:11.194077 17876 consensus_queue.cc:1048] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 13:19:11.213773 17623 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:11.233069 17889 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:11.272848 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:11.274560 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 78590.8us
I20260709 13:19:11.248807 17826 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:11.278952 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:11.244110 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1589) took 1384 ms. Trace:
I20260709 13:19:11.278254 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:11.234575 17978 consensus_queue.cc:1048] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:19:11.280483 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:11.286463 15676 raft_consensus.cc:2410] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:11.081570 15421 catalog_manager.cc:5697] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 1 to 2, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 2 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:11.244508 17735 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:19:11.234290 17947 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:11.290591 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:11.274225 17623 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:11.277348 17889 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:11.291610 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 58959us
I20260709 13:19:11.282835 17826 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:11.287670 17965 log.cc:927] Time spent T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 1.169s	user 0.006s	sys 0.000s
W20260709 13:19:11.289230 17355 consensus_meta.cc:287] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.558s	user 0.008s	sys 0.003s
I20260709 13:19:11.301203 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:11.303766 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:11.306941 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:11.294329 17936 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:19:11.314433 17992 log.cc:927] Time spent T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 13:19:11.314644 15675 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:19:11.315765 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:11.314474 17936 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:11.316820 17936 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:11.318027 17739 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:19:11.311296 17995 log.cc:927] Time spent T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 13:19:11.320133 17991 log.cc:927] Time spent T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:19:11.320566 15524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 505) took 1464 ms. Trace:
I20260709 13:19:11.310592 15421 catalog_manager.cc:5697] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 2 to 3, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 3 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:11.304615 15668 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:11.321854 15421 catalog_manager.cc:5697] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 2 to 3, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 3 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:11.322652 15668 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:11.324385 15668 raft_consensus.cc:1275] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:19:11.320958 15524 rpcz_store.cc:276] 0709 13:19:09.855995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.519559 (+663564us) service_pool.cc:224] Handling call
0709 13:19:10.597886 (+ 78327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:10.911106 (+313220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:10.911134 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:10.923966 (+ 12832us) log.cc:844] Serialized 26 byte log entry
0709 13:19:11.055510 (+131544us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:11.055549 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.055621 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:11.320360 (+264739us) raft_consensus.cc:1685] finished
0709 13:19:11.320383 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.320509 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":96580,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":87,"thread_start_us":379155,"threads_started":2,"wal-append.queue_time_us":203752}
I20260709 13:19:11.326447 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:11.327032 15524 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:11.327967 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:11.329680 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:11.329843 17947 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:11.330854 15524 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:11.332679 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:11.324582 17951 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:11.337030 15421 catalog_manager.cc:5697] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 2 to 3, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 3 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:11.337805 17716 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:11.338946 15421 catalog_manager.cc:5697] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 3 to 4, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 4 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:11.340684 17716 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:11.288847 17867 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:11.341050 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:11.343535 15519 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:11.345608 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:11.346076 16234 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:11.347716 15675 raft_consensus.cc:2393] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 5: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:19:11.310487 15424 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49140 (request call id 359) took 13125 ms (client timeout 15000 ms). Trace:
W20260709 13:19:11.354691 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 120403us
I20260709 13:19:11.355020 16203 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:11.356376 15669 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:11.354992 15676 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:11.370983 16203 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:11.382354 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:11.382627 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:11.392949 15525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56840 (request call id 509) took 1534 ms. Trace:
I20260709 13:19:11.393447 15525 rpcz_store.cc:276] 0709 13:19:09.858436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.033852 (+175416us) service_pool.cc:224] Handling call
0709 13:19:11.392785 (+1358933us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b5400070dd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:11.392882 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":319274368}
I20260709 13:19:11.395057 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:11.396796 15525 raft_consensus.cc:1275] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:19:11.282181 15521 rpcz_store.cc:276] 0709 13:19:09.852384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.443766 (+591382us) service_pool.cc:224] Handling call
0709 13:19:10.444143 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:10.444580 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:10.444613 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:10.525833 (+ 81220us) log.cc:844] Serialized 26 byte log entry
0709 13:19:10.923711 (+397878us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:11.066295 (+142584us) log.cc:844] Serialized 14 byte log entry
0709 13:19:11.122873 (+ 56578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.128518 (+  5645us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:11.195452 (+ 66934us) raft_consensus.cc:1685] finished
0709 13:19:11.195479 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.236607 (+ 41128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3980,"prepare.queue_time_us":262238,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":8484,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":19304,"thread_start_us":465904,"threads_started":2,"wal-append.queue_time_us":589104}
I20260709 13:19:11.400349 15521 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:19:11.400714 15521 pending_rounds.cc:85] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 6
I20260709 13:19:11.400981 15521 pending_rounds.cc:107] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:19:11.401244 15521 raft_consensus.cc:2889] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:11.403096 17952 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:11.403620 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:11.404733 17867 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:11.404958 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:11.406450 18000 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:11.406931 15566 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:11.408154 15566 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:11.409731 18009 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:11.410207 18009 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:11.410576 18009 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:11.411042 15599 raft_consensus.cc:2393] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 4: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
W20260709 13:19:11.412359 15641 leader_election.cc:336] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.578s (SENT)
I20260709 13:19:11.418481 15641 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:11.362071 15668 pending_rounds.cc:85] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting all ops after (but not including) 5
I20260709 13:19:11.409996 16237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57258 (request call id 601) took 1310 ms. Trace:
I20260709 13:19:11.419638 16237 rpcz_store.cc:276] 0709 13:19:10.099488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.113739 (+ 14251us) service_pool.cc:224] Handling call
0709 13:19:10.114065 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:10.722127 (+608062us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.409734 (+687607us) spinlock_profiling.cc:243] Waited 585 ms on lock 0x7b54001e50d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:11.409934 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1346597248}
I20260709 13:19:11.380947 17947 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:11.427547 17739 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:11.405676 17929 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:11.429662 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 517) took 1258 ms. Trace:
I20260709 13:19:11.430153 15527 rpcz_store.cc:276] 0709 13:19:10.171159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.102932 (+931773us) service_pool.cc:224] Handling call
0709 13:19:11.103296 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:11.120246 (+ 16950us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:11.120295 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:11.127625 (+  7330us) log.cc:844] Serialized 26 byte log entry
0709 13:19:11.276591 (+148966us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:11.276633 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.276693 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:11.429391 (+152698us) raft_consensus.cc:1685] finished
0709 13:19:11.429417 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.429573 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9298,"raft.queue_time_us":15048,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":13487,"thread_start_us":71788,"threads_started":1,"wal-append.queue_time_us":278032}
I20260709 13:19:11.432310 15527 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:11.383203 15523 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
W20260709 13:19:11.434321 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 49407.2us
I20260709 13:19:11.434446 15566 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:11.428236 17739 raft_consensus.cc:697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:11.438210 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:11.438862 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:11.439172 15524 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:11.447289 16203 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:11.448990 18008 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:11.451478 15601 raft_consensus.cc:1240] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:11.452147 16203 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:11.452677 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:11.452920 17952 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:11.454807 17929 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:11.455565 15421 catalog_manager.cc:5697] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 1 to 2, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 2 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:11.457976 17721 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:19:11.459427 18000 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:11.459924 15525 raft_consensus.cc:1240] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:11.460997 17826 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:11.462319 15676 raft_consensus.cc:2393] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 7: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:11.463156 17826 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:11.467168 15519 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:11.451362 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:11.434890 15668 pending_rounds.cc:107] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:19:11.459082 18009 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:11.470510 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 223618us
I20260709 13:19:11.474435 16203 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:11.436216 17967 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:11.484637 15642 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:11.486212 15642 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:11.490064 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:11.488795 16203 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:11.495421 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:11.496701 15642 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:11.476534 15668 raft_consensus.cc:2889] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:11.485590 17944 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:11.477432 15598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 408) took 1137 ms. Trace:
W20260709 13:19:11.498234 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 220315us
I20260709 13:19:11.503342 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:11.497641 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:11.488003 17777 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:11.520706 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:11.497457 18000 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:19:11.350176 15424 rpcz_store.cc:269] 0709 13:18:58.066452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.066707 (+   255us) service_pool.cc:224] Handling call
0709 13:19:11.183055 (+13116348us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:03.265838 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:03.640661 (+374823us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:03.681976 (+ 41315us) write_op.cc:435] Acquired schema lock
0709 13:19:03.711782 (+ 29806us) tablet.cc:662] Decoding operations
0709 13:19:03.712978 (+  1196us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:03.713067 (+    89us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:03.713105 (+    38us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:19:03.776842 (+ 63737us) tablet.cc:701] Row locks acquired
0709 13:19:03.801036 (+ 24194us) write_op.cc:260] PREPARE: finished
0709 13:19:03.862046 (+ 61010us) write_op.cc:270] Start()
0709 13:19:04.106706 (+244660us) write_op.cc:276] Timestamp: P: 1783603143834928 usec, L: 0
0709 13:19:04.106765 (+    59us) op_driver.cc:348] REPLICATION: starting
0709 13:19:04.190656 (+ 83891us) log.cc:844] Serialized 2891 byte log entry
0709 13:19:05.094612 (+903956us) op_driver.cc:464] REPLICATION: finished
0709 13:19:05.545602 (+450990us) write_op.cc:301] APPLY: starting
0709 13:19:05.558618 (+ 13016us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:05.649141 (+ 90523us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:05.697434 (+ 48293us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:05.989654 (+292220us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:05.989902 (+   248us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 13:19:05.989972 (+    70us) write_op.cc:312] APPLY: finished
0709 13:19:05.990365 (+   393us) log.cc:844] Serialized 59 byte log entry
0709 13:19:06.184181 (+193816us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:06.244688 (+ 60507us) write_op.cc:454] Released schema lock
0709 13:19:06.250562 (+  5874us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:06.286487 (+ 35925us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:06.286521 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":25576,"reported_tablets":11,"spinlock_wait_cycles":17024,"child_traces":[["op",{"apply.queue_time_us":399946,"apply.run_cpu_time_us":50871,"apply.run_wall_time_us":772562,"num_ops":7,"prepare.queue_time_us":708525,"prepare.run_cpu_time_us":37100,"prepare.run_wall_time_us":1218758,"raft.queue_time_us":289795,"raft.run_cpu_time_us":143749,"raft.run_wall_time_us":206008,"replication_time_us":959336,"thread_start_us":896295,"threads_started":5,"wal-append.queue_time_us":646806,"wal-append.run_cpu_time_us":13052,"wal-append.run_wall_time_us":2415776}]]}
I20260709 13:19:11.460997 17936 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:11.495887 17744 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:11.526896 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:11.498039 17735 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:11.486647 17978 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:19:11.543462 15599 raft_consensus.cc:1240] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:11.536633 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:11.544965 15601 raft_consensus.cc:2393] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 5: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:11.544399 16237 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:19:11.546594 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:11.544627 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 45684.2us
I20260709 13:19:11.502904 17953 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:11.503520 17998 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:11.550874 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:11.542296 17943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.430s	user 0.009s	sys 0.021s
I20260709 13:19:11.497887 17947 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:19:11.566048 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:11.568171 15599 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:19:11.538076 18000 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:11.569959 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:11.543952 17744 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:11.578354 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:11.543533 17735 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:11.578816 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 144463us
I20260709 13:19:11.550483 17823 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:11.551395 17953 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:11.565418 17998 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:11.582546 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:11.567618 17943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.455s	user 0.009s	sys 0.024s
I20260709 13:19:11.585399 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:11.591838 15641 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:11.593029 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:11.621774 15599 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:19:11.579133 18000 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:11.626031 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 149823us
I20260709 13:19:11.578227 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:11.582168 17744 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:11.589035 17823 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:11.629261 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:11.621312 15602 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:11.586483 17735 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:11.592752 16232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57244 (request call id 871) took 1443 ms. Trace:
I20260709 13:19:11.634855 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:11.640275 15599 raft_consensus.cc:1240] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 13:19:11.641104 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 114748us
I20260709 13:19:11.594421 17953 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:11.643857 15601 raft_consensus.cc:2393] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 5: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:11.622483 17998 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:11.647580 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:11.650480 15603 raft_consensus.cc:2410] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:19:11.651229 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:11.586290 17889 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:11.546921 17739 consensus_queue.cc:237] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:11.638110 17823 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:11.645037 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 94498.3us
I20260709 13:19:11.671887 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 503) took 1845 ms. Trace:
W20260709 13:19:11.673406 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 90358.9us
I20260709 13:19:11.673173 15520 rpcz_store.cc:276] 0709 13:19:09.819630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.819830 (+   200us) service_pool.cc:224] Handling call
0709 13:19:09.820151 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:09.923153 (+103002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.662103 (+1738950us) spinlock_profiling.cc:243] Waited 6.26 ms on lock 0x7b540029c2d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:11.665260 (+  3157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14396672}
I20260709 13:19:11.676640 16234 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:19:11.668147 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 76398.9us
I20260709 13:19:11.643944 18013 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:11.679535 15641 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:11.666932 16231 raft_consensus.cc:2393] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:11.686465 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:11.686836 16234 raft_consensus.cc:1275] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:11.672490 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:11.690419 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 47462.8us
W20260709 13:19:11.693764 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:11.703884 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 56420.8us
I20260709 13:19:11.706386 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:11.663605 18021 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:19:11.640939 16232 rpcz_store.cc:276] 0709 13:19:10.137387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.259071 (+121684us) service_pool.cc:224] Handling call
0709 13:19:11.563978 (+1304907us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b5400496954. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:11.580565 (+ 16587us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63958912}
W20260709 13:19:11.708911 17969 log.cc:927] Time spent T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 1.590s	user 0.000s	sys 0.005s
I20260709 13:19:11.702543 18009 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:11.685127 17623 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:11.722898 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:11.723451 16240 raft_consensus.cc:1240] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:19:11.726595 16235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57258 (request call id 606) took 1586 ms. Trace:
W20260709 13:19:11.717721 18010 log.cc:927] Time spent T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:19:11.734774 16235 rpcz_store.cc:276] 0709 13:19:10.139990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.240747 (+100757us) service_pool.cc:224] Handling call
0709 13:19:11.726400 (+1485653us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54005f0fd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:11.726538 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2951873280}
I20260709 13:19:11.744477 16240 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57258 (request call id 607) took 1568 ms. Trace:
I20260709 13:19:11.803750 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:11.803279 16240 rpcz_store.cc:276] 0709 13:19:10.176053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.717409 (+1541356us) service_pool.cc:224] Handling call
0709 13:19:11.723232 (+  5823us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:11.744127 (+ 20895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.744404 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:11.722198 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:11.814889 16231 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:11.815065 18018 log.cc:927] Time spent T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:19:11.727677 17777 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:11.818598 17777 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:11.523789 15598 rpcz_store.cc:276] 0709 13:19:10.337721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.104234 (+766513us) service_pool.cc:224] Handling call
0709 13:19:11.104567 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:11.104965 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:11.105003 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:11.105498 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 13:19:11.303318 (+197820us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:11.340360 (+ 37042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.340499 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:11.436785 (+ 96286us) raft_consensus.cc:1685] finished
0709 13:19:11.436806 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.475007 (+ 38201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6611,"raft.run_cpu_time_us":991,"raft.run_wall_time_us":29052,"thread_start_us":188423,"threads_started":1,"wal-append.queue_time_us":324995}
W20260709 13:19:11.826222 18011 log.cc:927] Time spent T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:19:11.716850 18028 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:19:11.725399 17976 log.cc:927] Time spent T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 1.127s	user 0.005s	sys 0.000s
I20260709 13:19:11.725366 17826 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:11.800246 18015 log.cc:927] Time spent T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:19:11.799470 17623 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:11.721855 17936 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:11.722908 17889 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:11.842150 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 167959us
I20260709 13:19:11.832123 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:11.828887 18028 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:11.742354 18024 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:11.848361 15673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 521) took 1780 ms. Trace:
W20260709 13:19:11.848806 18017 log.cc:927] Time spent T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
W20260709 13:19:11.831516 18014 log.cc:927] Time spent T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 13:19:11.729905 18026 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 13:19:11.796700 18021 consensus_queue.cc:1048] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:19:11.796977 17867 consensus_queue.cc:1048] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:19:11.851416 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:11.848510 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:11.855672 15566 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:11.866024 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 186566us
I20260709 13:19:11.852676 15673 rpcz_store.cc:276] 0709 13:19:10.067877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.226774 (+1158897us) service_pool.cc:224] Handling call
0709 13:19:11.227119 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:11.249135 (+ 22016us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:11.249169 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:11.271983 (+ 22814us) log.cc:844] Serialized 26 byte log entry
0709 13:19:11.539340 (+267357us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:11.539383 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.539460 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:11.848004 (+308544us) raft_consensus.cc:1685] finished
0709 13:19:11.848024 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.848169 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10710,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":3599,"thread_start_us":239410,"threads_started":1,"wal-append.queue_time_us":362638}
I20260709 13:19:11.723263 18025 consensus_queue.cc:1059] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.135s
I20260709 13:19:11.863677 15671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 514) took 1937 ms. Trace:
I20260709 13:19:11.875564 15671 rpcz_store.cc:276] 0709 13:19:09.926416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.933793 (+  7377us) service_pool.cc:224] Handling call
0709 13:19:09.941833 (+  8040us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:10.025229 (+ 83396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:10.027110 (+  1881us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:10.064164 (+ 37054us) log.cc:844] Serialized 26 byte log entry
0709 13:19:10.249553 (+185389us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:10.328637 (+ 79084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.342302 (+ 13665us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:11.724812 (+1382510us) spinlock_profiling.cc:243] Waited 9.84 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:19:11.863351 (+138539us) raft_consensus.cc:1685] finished
0709 13:19:11.863380 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.863613 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42245,"raft.queue_time_us":46994,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":9105,"spinlock_wait_cycles":22621952,"thread_start_us":138035,"threads_started":1,"wal-append.queue_time_us":430075}
I20260709 13:19:11.878273 17729 raft_consensus.cc:3055] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:11.878720 17729 raft_consensus.cc:740] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:11.879511 17729 consensus_queue.cc:260] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:11.880941 17729 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:11.862079 18009 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:19:11.865981 18023 consensus_queue.cc:1048] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:19:11.854961 18024 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:11.864369 15670 raft_consensus.cc:1240] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:19:11.854456 17826 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.726470 18027 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.130s
I20260709 13:19:11.861028 16231 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:11.843241 18007 consensus_queue.cc:1048] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:19:11.906332 17739 raft_consensus.cc:3055] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:11.906819 17739 raft_consensus.cc:740] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:11.907604 17739 consensus_queue.cc:260] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:11.908761 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:11.908805 17739 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:11.913098 17735 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:11.915700 17735 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:11.923350 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1578) took 2074 ms. Trace:
I20260709 13:19:11.923790 15528 rpcz_store.cc:276] 0709 13:19:09.849127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.151344 (+302217us) service_pool.cc:224] Handling call
0709 13:19:10.157613 (+  6269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:10.325713 (+168100us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:10.325743 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:10.505151 (+179408us) log.cc:844] Serialized 26 byte log entry
0709 13:19:10.903640 (+398489us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:11.126152 (+222512us) log.cc:844] Serialized 14 byte log entry
0709 13:19:11.126450 (+   298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.922982 (+796532us) spinlock_profiling.cc:243] Waited 93.8 ms on lock 0x7b54002724d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:11.923031 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:11.923075 (+    44us) raft_consensus.cc:1685] finished
0709 13:19:11.923156 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.923289 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12238,"prepare.queue_time_us":252867,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":1636,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":13469,"spinlock_wait_cycles":220837632,"thread_start_us":600013,"threads_started":2,"wal-append.queue_time_us":521252}
I20260709 13:19:11.932883 15594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 1610) took 3705 ms. Trace:
I20260709 13:19:11.935513 17953 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:11.870340 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 191235us
I20260709 13:19:11.854280 18028 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:11.941538 16205 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:11.942509 15677 raft_consensus.cc:1240] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:11.966308 15597 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:11.982934 15566 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:11.990684 15596 raft_consensus.cc:1275] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:19:11.978210 18022 log.cc:927] Time spent T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.152s	user 0.006s	sys 0.006s
I20260709 13:19:11.944399 18033 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:11.881906 18000 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:11.979161 17978 raft_consensus.cc:2804] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:11.998466 17739 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:11.937811 17826 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:12.008931 17823 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:11.988382 17946 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:11.998165 17984 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:12.031430 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 92894.3us
I20260709 13:19:11.994957 15598 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:19:12.007850 18033 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:12.067166 15673 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:19:12.074414 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.995409 17744 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:12.087709 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 50995.5us
I20260709 13:19:12.032955 17953 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:12.034673 17978 raft_consensus.cc:697] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:19:12.077732 18020 log.cc:927] Time spent T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.320s	user 0.001s	sys 0.001s
I20260709 13:19:12.100275 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:12.066015 18033 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:12.103762 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:12.026607 17735 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:19:12.080436 17777 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:12.025207 17739 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:12.070334 15598 raft_consensus.cc:606] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:12.104533 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:12.105306 15597 raft_consensus.cc:2393] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 7: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:12.032634 17355 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:19:12.109098 17944 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:12.110236 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:12.111106 18035 log.cc:927] Time spent T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.285s	user 0.001s	sys 0.000s
I20260709 13:19:12.114022 17958 raft_consensus.cc:3055] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:12.114464 18000 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:12.115418 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:19:12.116112 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 896379us
I20260709 13:19:12.138994 15491 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:12.116621 17688 consensus_queue.cc:1048] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:12.084486 18032 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.117767 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 510) took 2259 ms. Trace:
I20260709 13:19:12.118297 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:12.109277 17978 consensus_queue.cc:237] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:12.123057 15602 raft_consensus.cc:1275] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:19:12.035899 17946 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:12.148092 15568 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:12.157912 15673 raft_consensus.cc:2393] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:19:12.162158 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 201839us
W20260709 13:19:12.200959 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 52964.2us
I20260709 13:19:12.201786 15596 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:19:12.203538 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:12.204456 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 506) took 2347 ms. Trace:
I20260709 13:19:12.204651 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:12.130121 15598 raft_consensus.cc:614] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:12.206781 15674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 1745) took 2156 ms. Trace:
I20260709 13:19:12.130777 18026 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.208320 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:12.204870 15526 rpcz_store.cc:276] 0709 13:19:09.857334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.625692 (+768358us) service_pool.cc:224] Handling call
0709 13:19:11.085764 (+460072us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:11.094247 (+  8483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:11.094280 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:11.123950 (+ 29670us) log.cc:844] Serialized 26 byte log entry
0709 13:19:11.279181 (+155231us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:11.279219 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.279283 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:11.408783 (+129500us) raft_consensus.cc:1685] finished
0709 13:19:11.408801 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.166833 (+758032us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b5400262850. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:12.204403 (+ 37570us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1048,"raft.queue_time_us":17689,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":112,"spinlock_wait_cycles":1011551104,"thread_start_us":88983,"threads_started":1,"wal-append.queue_time_us":258062}
I20260709 13:19:12.209038 15524 raft_consensus.cc:2376] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6. Re-sending same reply.
I20260709 13:19:12.209312 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:12.202802 15522 rpcz_store.cc:276] 0709 13:19:09.858728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.898371 (+1039643us) service_pool.cc:224] Handling call
0709 13:19:10.898743 (+   372us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:10.899119 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:10.899148 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:10.908809 (+  9661us) log.cc:844] Serialized 52 byte log entry
0709 13:19:11.207194 (+298385us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:11.249735 (+ 42541us) log.cc:844] Serialized 14 byte log entry
0709 13:19:12.117469 (+867734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.117546 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:12.117587 (+    41us) raft_consensus.cc:1685] finished
0709 13:19:12.117600 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.117712 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":867484,"prepare.queue_time_us":44974,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":873393,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":17352,"thread_start_us":263523,"threads_started":1,"wal-append.queue_time_us":371107}
I20260709 13:19:12.209945 15523 raft_consensus.cc:2376] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4. Re-sending same reply.
I20260709 13:19:12.210171 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:12.211964 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:12.135985 17958 raft_consensus.cc:740] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:12.207129 15674 rpcz_store.cc:276] 0709 13:19:10.050417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.725152 (+674735us) service_pool.cc:224] Handling call
0709 13:19:11.074442 (+349290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:11.074814 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:11.074842 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:11.075314 (+   472us) log.cc:844] Serialized 26 byte log entry
0709 13:19:11.236430 (+161116us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:11.236468 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.236529 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:11.328401 (+ 91872us) raft_consensus.cc:1685] finished
0709 13:19:11.328421 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.158589 (+830168us) spinlock_profiling.cc:243] Waited 349 ms on lock 0x7b54001a7a50. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:12.206727 (+ 48138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25193,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"spinlock_wait_cycles":802475776,"thread_start_us":160740,"threads_started":1,"wal-append.queue_time_us":198381}
I20260709 13:19:12.215739 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:12.216341 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:12.217031 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:12.222548 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.156397 17984 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:12.225286 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 86358.7us
I20260709 13:19:12.226864 15522 raft_consensus.cc:1275] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:19:12.225006 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:12.232975 15491 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:12.235591 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:12.161262 17967 consensus_queue.cc:1059] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:19:12.239866 17984 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:12.210219 18032 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:12.242684 18045 log.cc:927] Time spent T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:19:12.262912 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:11.933363 15594 rpcz_store.cc:276] 0709 13:19:08.227120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.228214 (+  1094us) service_pool.cc:224] Handling call
0709 13:19:08.228636 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:08.301124 (+ 72488us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:19:08.534196 (+233072us) log.cc:844] Serialized 14 byte log entry
0709 13:19:08.791839 (+257643us) log.cc:844] Serialized 14 byte log entry
0709 13:19:09.747770 (+955931us) spinlock_profiling.cc:243] Waited 33.4 ms on lock 0x7b18000564a0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce191e522 00007f5ce19207c1 00007f5ce1920779 00007f5ce1920709 00007f5ce19206d1 00007f5ce191f9cd 000000000038f354 000000000038f199 00007f5ce190183a
0709 13:19:10.525641 (+777871us) spinlock_profiling.cc:243] Waited 8.49 ms on lock 0x7b18000564a0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191b455 00007f5ce4a03060 00007f5ce4a15716 00007f5ce4a26287 00007f5ce4a26208
0709 13:19:10.890176 (+364535us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:10.890235 (+    59us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:10.890311 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.932519 (+1042208us) spinlock_profiling.cc:243] Waited 72.1 ms on lock 0x7b54001dabd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:11.932569 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.932720 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":124946,"prepare.queue_time_us":565544,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":80,"raft.queue_time_us":1320869,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":10026,"spinlock_wait_cycles":262194560,"thread_start_us":587855,"threads_started":3}
I20260709 13:19:12.246097 18050 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.115s
I20260709 13:19:12.250257 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:12.268723 15520 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:19:12.252276 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:12.252532 17729 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:12.208424 17867 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:12.255066 15526 raft_consensus.cc:2393] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 6: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:12.255003 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:19:12.224247 18034 log.cc:927] Time spent T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:19:12.264029 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 483908us
I20260709 13:19:12.278365 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:12.223345 17623 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:12.226015 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:12.332612 15675 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:12.334815 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 111032us
I20260709 13:19:12.336838 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:12.337852 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:12.339439 15526 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:12.236470 17951 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:12.226269 16233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 467) took 1301 ms. Trace:
W20260709 13:19:12.346360 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 113437us
I20260709 13:19:12.351154 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:12.354386 15520 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:12.351871 15599 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:12.219869 18026 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:12.350451 15642 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:12.365618 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:12.268041 17952 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.268471 17928 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.370810 15642 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:12.260159 15598 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:19:12.366355 15522 raft_consensus.cc:2376] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6. Re-sending same reply.
I20260709 13:19:12.376886 15491 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:12.379868 15596 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:12.361035 17827 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:12.381244 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:12.382339 15675 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:12.385740 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:12.386574 15526 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:19:12.387933 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:12.350718 17688 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:12.359086 17951 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:12.363278 17823 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:12.414412 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1631) took 1079 ms. Trace:
I20260709 13:19:12.414301 16231 raft_consensus.cc:2393] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 7: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:12.432245 15521 raft_consensus.cc:2393] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:12.347388 17729 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:12.456133 15519 rpcz_store.cc:276] 0709 13:19:11.308866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.345337 (+ 36471us) service_pool.cc:224] Handling call
0709 13:19:12.387696 (+1042359us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b540029d1d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:12.387854 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":277510400}
I20260709 13:19:12.353665 16233 rpcz_store.cc:276] 0709 13:19:10.922599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.103647 (+1181048us) service_pool.cc:224] Handling call
0709 13:19:12.223508 (+119861us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:12.434212 17951 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:12.433007 17688 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:12.376041 17623 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:12.415518 17826 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:12.393901 17988 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:12.442803 17827 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:12.437297 17744 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:12.499998 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:12.435070 17958 consensus_queue.cc:260] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:12.515352 17739 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:12.434728 17952 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:12.523461 17729 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:12.433746 17928 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:12.525612 18032 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:12.526167 15670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49094 (request call id 1690) took 2497 ms. Trace:
I20260709 13:19:12.526978 15525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1621) took 1291 ms. Trace:
I20260709 13:19:12.530568 17946 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:12.534806 17984 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:12.378283 18049 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.104s
W20260709 13:19:12.575410 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 189025us
I20260709 13:19:12.614933 15525 rpcz_store.cc:276] 0709 13:19:11.235681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.397699 (+162018us) service_pool.cc:224] Handling call
0709 13:19:11.418206 (+ 20507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.460438 (+ 42232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.526729 (+1066291us) spinlock_profiling.cc:243] Waited 41.5 ms on lock 0x7b540029bdd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:12.526911 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95484800}
W20260709 13:19:12.591809 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:12.560457 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 182667us
I20260709 13:19:12.615007 15670 rpcz_store.cc:276] 0709 13:19:10.029266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.724237 (+694971us) service_pool.cc:224] Handling call
0709 13:19:11.864076 (+1139839us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:11.899744 (+ 35668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.525927 (+626183us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54003886d0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:12.526093 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2620741504}
I20260709 13:19:12.570992 18027 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.660880 15524 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:19:12.661892 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:12.624483 17867 raft_consensus.cc:697] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:12.599041 18033 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:12.660291 17984 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:12.663853 17984 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:12.558148 17777 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:12.540252 17688 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:12.615211 17827 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:12.627766 17988 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:12.516403 18047 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:12.655167 17721 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:12.687386 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:12.695667 15670 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:12.687116 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 189617us
W20260709 13:19:12.683768 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 294676us
W20260709 13:19:12.659015 17976 log.cc:927] Time spent T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:19:12.491014 18028 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:12.686523 17827 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:12.686677 18047 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:12.775061 18047 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:12.711850 18044 log.cc:927] Time spent T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:19:12.689519 17988 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:12.779831 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:12.689481 17826 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:12.781211 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:12.781062 17826 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:12.781628 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:12.682667 17867 consensus_queue.cc:237] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:12.780833 16240 raft_consensus.cc:1240] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:12.782248 16235 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:12.783746 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:12.674832 18027 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:12.785722 15668 raft_consensus.cc:1240] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:19:12.786172 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:12.787086 16232 raft_consensus.cc:2410] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:19:12.787920 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:12.789237 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:12.721248 18067 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.791707 18067 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:12.792680 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:12.793941 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:12.794474 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:12.795075 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:12.728366 18053 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:12.514631 18062 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.234s
I20260709 13:19:12.762298 18068 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:12.763368 17928 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:12.764736 18033 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:12.766148 17952 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:12.766563 17984 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:12.768203 18028 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:12.770989 18026 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:12.772315 17623 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:12.780666 15672 raft_consensus.cc:2393] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:12.741236 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:12.913074 15669 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:19:12.803678 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:12.816366 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:19:12.843171 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.839619 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:12.914142 15602 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:12.914430 15675 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:12.875532 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:19:12.816802 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 116761us
I20260709 13:19:12.824723 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:12.843688 16203 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:12.835373 15676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49100 (request call id 1753) took 1431 ms. Trace:
I20260709 13:19:12.916473 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:12.919226 15599 raft_consensus.cc:2393] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 5: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:12.916752 15676 rpcz_store.cc:276] 0709 13:19:11.404320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.404701 (+   381us) service_pool.cc:224] Handling call
0709 13:19:12.835155 (+1430454us) spinlock_profiling.cc:243] Waited 56.6 ms on lock 0x7b540042bb54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:12.835308 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":130133376}
I20260709 13:19:12.924681 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:12.928596 15493 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:12.932350 15523 raft_consensus.cc:1275] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:12.933424 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:12.936728 15493 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:12.938174 15493 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:12.708447 15642 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:12.961014 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 117421us
I20260709 13:19:12.967878 15525 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:12.923976 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:12.968583 16240 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:19:12.969946 15526 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:19:12.914677 18059 log.cc:927] Time spent T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.423s	user 0.004s	sys 0.000s
I20260709 13:19:12.818677 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:12.839620 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:12.988301 15675 raft_consensus.cc:1240] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:12.991515 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:12.992931 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:12.992326 15528 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:12.879040 16232 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:12.720317 17721 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:13.011134 15677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 540) took 1161 ms. Trace:
I20260709 13:19:13.011579 15677 rpcz_store.cc:276] 0709 13:19:11.850185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.919970 (+ 69785us) service_pool.cc:224] Handling call
0709 13:19:11.920229 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.952088 (+ 31859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:13.010893 (+1058805us) spinlock_profiling.cc:243] Waited 22.1 ms on lock 0x7b540009c4d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:13.011076 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50832640}
W20260709 13:19:13.053440 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 345050us
I20260709 13:19:12.931537 17355 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.121s
I20260709 13:19:13.014772 18072 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:13.057911 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:13.058911 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:12.992317 17950 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:13.061087 15522 raft_consensus.cc:1240] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:13.015782 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:12.971405 17623 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.063966 15670 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:12.834429 17889 consensus_queue.cc:1059] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.176s
W20260709 13:19:13.004539 17998 consensus_meta.cc:287] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 1.339s	user 0.015s	sys 0.000s
I20260709 13:19:13.066411 15642 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:13.072487 17716 consensus_queue.cc:1048] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:19:13.076748 17721 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:13.078281 18072 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:19:13.078578 18016 log.cc:927] Time spent T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 1.302s	user 0.007s	sys 0.000s
I20260709 13:19:13.079195 18027 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:13.015056 18056 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:13.089766 15642 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:13.067423 18028 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:19:13.103770 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.003513 17920 consensus_queue.cc:1059] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.085s
I20260709 13:19:13.097586 18072 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.113451 15674 raft_consensus.cc:1275] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:13.103920 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:13.120200 15522 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:13.112860 15642 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:12.918176 18068 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:13.125360 18072 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:13.100569 18026 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:13.144949 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:13.154253 15641 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.043s	sys 0.055s
W20260709 13:19:13.155773 15641 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.044s	sys 0.055s
I20260709 13:19:13.109827 18067 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:13.125427 18048 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:13.140354 17889 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:13.144949 15668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 545) took 1011 ms. Trace:
I20260709 13:19:13.147042 18074 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:13.093613 17623 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:13.163187 15567 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.804s	user 0.024s	sys 0.016s
I20260709 13:19:13.143106 17950 raft_consensus.cc:697] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:13.162595 15524 raft_consensus.cc:3055] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:13.172777 15524 raft_consensus.cc:740] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
W20260709 13:19:13.173671 15567 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.815s	user 0.024s	sys 0.016s
W20260709 13:19:13.154692 17823 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.017s	sys 0.033s
W20260709 13:19:13.254871 17823 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.017s	sys 0.033s
W20260709 13:19:13.256420 15567 leader_election.cc:336] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.671s (SENT)
W20260709 13:19:13.257380 15567 leader_election.cc:336] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.767s (SENT)
W20260709 13:19:13.260206 17951 consensus_meta.cc:287] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.696s	user 0.005s	sys 0.002s
I20260709 13:19:13.104534 17998 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:13.175168 18028 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:13.113677 18056 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:13.161427 18048 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:13.172019 18026 raft_consensus.cc:697] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:13.252261 17950 consensus_queue.cc:237] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:13.253841 15524 consensus_queue.cc:260] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:13.292711 15522 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
W20260709 13:19:13.327960 15603 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.013s	sys 0.052s
W20260709 13:19:13.332026 15603 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.013s	sys 0.052s
I20260709 13:19:13.333001 15603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 1646) took 1231 ms. Trace:
I20260709 13:19:13.333453 15603 rpcz_store.cc:276] 0709 13:19:12.101599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.107681 (+  6082us) service_pool.cc:224] Handling call
0709 13:19:12.133596 (+ 25915us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:12.133795 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:19:12.199968 (+ 66173us) log.cc:844] Serialized 14 byte log entry
0709 13:19:12.820274 (+620306us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b18003fb740. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab23a 00007f5ce4a15a2a
0709 13:19:13.332616 (+512342us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:13.332676 (+    60us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:13.332737 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:13.332809 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:13.332942 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":66607,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":3405,"spinlock_wait_cycles":30067968,"thread_start_us":512033,"threads_started":1}
W20260709 13:19:13.437721 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.453024 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:13.261077 18064 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.456043 15602 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:13.453828 15522 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
W20260709 13:19:13.267715 18058 log.cc:927] Time spent T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.638s	user 0.007s	sys 0.002s
W20260709 13:19:13.275871 18047 consensus_meta.cc:287] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.500s	user 0.006s	sys 0.002s
W20260709 13:19:13.263466 17688 consensus_meta.cc:287] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.544s	user 0.007s	sys 0.000s
W20260709 13:19:13.487738 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 210847us
I20260709 13:19:13.270488 18048 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:13.267710 17920 raft_consensus.cc:3055] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:13.260753 18072 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:13.269934 18056 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:13.270689 18068 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:13.264453 17998 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:13.275841 17623 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:13.274204 17826 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:13.330506 15672 raft_consensus.cc:1240] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:13.260684 17951 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:13.496978 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:13.503314 15522 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:13.519037 15600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 410) took 3180 ms. Trace:
I20260709 13:19:13.501598 15524 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:13.536290 15600 rpcz_store.cc:276] 0709 13:19:10.338406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.111103 (+772697us) service_pool.cc:224] Handling call
0709 13:19:11.111554 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:11.123726 (+ 12172us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:19:11.255140 (+131414us) log.cc:844] Serialized 14 byte log entry
0709 13:19:11.629853 (+374713us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:11.632607 (+  2754us) log.cc:844] Serialized 26 byte log entry
0709 13:19:11.633470 (+   863us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:11.633776 (+   306us) log.cc:844] Serialized 14 byte log entry
0709 13:19:11.641675 (+  7899us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.641768 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:11.936435 (+294667us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b48001c0158. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce9934aa1 00007f5ce992bb37 00007f5ce992b9ca 00007f5ce9a2326f 00007f5ce9a22f37 00007f5ce9a3755c 00007f5ce9a37508 00007f5ce9a37488 00007f5ce9a37443 00007f5ce9a366ec 00007f5ce8960a96
0709 13:19:12.886667 (+950232us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b24002c1a88. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce17a49c4 00007f5ce17a2acb 00007f5ce195ddc9 00007f5ce195da55 00007f5ce195d09d 00007f5ce458f10e 00007f5ce45a688c 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221
0709 13:19:13.518807 (+632140us) raft_consensus.cc:1685] finished
0709 13:19:13.518828 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:13.518976 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20424,"prepare.queue_time_us":39357,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":106,"raft.queue_time_us":221788,"raft.run_cpu_time_us":2297,"raft.run_wall_time_us":820416,"spinlock_wait_cycles":79175168,"thread_start_us":447779,"threads_started":3,"wal-append.queue_time_us":358963}
I20260709 13:19:13.505575 17920 raft_consensus.cc:740] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:19:13.576970 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 80054.2us
I20260709 13:19:13.577988 15642 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:13.170428 15668 rpcz_store.cc:276] 0709 13:19:12.129237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.969138 (+839901us) service_pool.cc:224] Handling call
0709 13:19:13.052771 (+ 83633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:13.074241 (+ 21470us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:13.088782 (+ 14541us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:13.091376 (+  2594us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:13.100147 (+  8771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:13.107580 (+  7433us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:13.140820 (+ 33240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:13.502568 18047 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:13.568359 18025 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.011s	sys 0.018s
I20260709 13:19:13.545095 18062 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:13.506356 17688 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:13.489938 18064 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:13.588585 17945 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.796s	user 0.013s	sys 0.031s
I20260709 13:19:13.588785 17716 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:19:13.623092 15519 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.005s	sys 0.014s
W20260709 13:19:13.623502 15519 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.005s	sys 0.014s
I20260709 13:19:13.596825 17826 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:19:13.604636 18025 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.011s	sys 0.018s
I20260709 13:19:13.627399 15524 raft_consensus.cc:1275] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 13:19:13.639714 15524 pending_rounds.cc:85] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 6
I20260709 13:19:13.640237 15524 pending_rounds.cc:107] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:19:13.640575 15524 raft_consensus.cc:2889] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:13.518484 18026 consensus_queue.cc:237] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:13.642836 15522 raft_consensus.cc:1240] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: []
W20260709 13:19:13.646783 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 487343us
W20260709 13:19:13.621031 17945 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.832s	user 0.014s	sys 0.031s
I20260709 13:19:13.648273 15600 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:13.651113 15524 raft_consensus.cc:1240] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: []
W20260709 13:19:13.624110 18051 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.373s	user 0.006s	sys 0.015s
W20260709 13:19:13.636849 17967 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.004s	sys 0.013s
W20260709 13:19:13.645761 17943 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.005s	sys 0.015s
I20260709 13:19:13.637882 17826 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:19:13.654632 18051 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.403s	user 0.007s	sys 0.015s
W20260709 13:19:13.658008 17967 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.005s	sys 0.013s
W20260709 13:19:13.650556 17988 consensus_meta.cc:287] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.871s	user 0.009s	sys 0.009s
I20260709 13:19:13.638132 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:13.656049 18080 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:19:13.618950 18082 consensus_queue.cc:1048] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.135s
W20260709 13:19:13.745229 17943 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.005s	sys 0.015s
W20260709 13:19:13.779577 16204 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.003s	sys 0.033s
W20260709 13:19:13.781014 16204 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.004s	sys 0.033s
W20260709 13:19:13.743590 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 87408.9us
I20260709 13:19:13.769495 17826 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:13.783427 15642 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:13.771749 18066 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.822268 18066 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:13.824671 18066 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:13.777949 18063 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.017s	sys 0.045s
I20260709 13:19:13.825889 15677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 1766) took 1289 ms. Trace:
W20260709 13:19:13.826310 18063 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.018s	sys 0.046s
W20260709 13:19:13.788583 17867 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.014s	sys 0.052s
I20260709 13:19:13.805734 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:13.830109 15669 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:19:13.778291 17988 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:13.821049 18087 log.cc:927] Time spent T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:19:13.821491 18046 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:13.819092 18051 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:13.826331 15677 rpcz_store.cc:276] 0709 13:19:12.536036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.013850 (+477814us) service_pool.cc:224] Handling call
0709 13:19:13.073307 (+ 59457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:13.091829 (+ 18522us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:13.091863 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:13.107262 (+ 15399us) log.cc:844] Serialized 26 byte log entry
0709 13:19:13.624370 (+517108us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:13.624414 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:13.624490 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:13.825694 (+201204us) raft_consensus.cc:1685] finished
0709 13:19:13.825712 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:13.825830 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1917,"raft.queue_time_us":2043,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":7331,"thread_start_us":457237,"threads_started":1,"wal-append.queue_time_us":639219}
W20260709 13:19:13.843750 18095 log.cc:927] Time spent T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:19:13.843987 17688 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:13.833812 17951 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:13.845144 15702 heartbeater.cc:646] Failed to heartbeat to 127.14.23.126:39251 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.14.23.126:39251 timed out after 15.000s (SENT)
I20260709 13:19:13.845645 18051 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:13.851351 15677 raft_consensus.cc:1240] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:19:13.815160 18072 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:13.851473 18048 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:13.853320 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:13.833360 17827 consensus_meta.cc:287] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 1.040s	user 0.013s	sys 0.000s
I20260709 13:19:13.854308 15668 raft_consensus.cc:2410] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:19:13.854241 17827 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:13.855566 15673 raft_consensus.cc:2410] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:19:13.856387 17827 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:13.807919 17984 raft_consensus.cc:2955] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 2 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:19:13.945693 15567 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.014s	sys 0.031s
I20260709 13:19:13.962433 15670 raft_consensus.cc:1240] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:19:13.987558 15567 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.014s	sys 0.032s
I20260709 13:19:13.865743 15673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49100 (request call id 1768) took 1271 ms. Trace:
I20260709 13:19:13.988096 15670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49094 (request call id 1713) took 1280 ms. Trace:
W20260709 13:19:13.865679 15676 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.766s	user 0.014s	sys 0.027s
W20260709 13:19:13.988840 15676 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.889s	user 0.015s	sys 0.035s
I20260709 13:19:13.856632 15526 raft_consensus.cc:1240] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:19:13.988543 15670 rpcz_store.cc:276] 0709 13:19:12.707398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.086534 (+379136us) service_pool.cc:224] Handling call
0709 13:19:13.961974 (+875440us) spinlock_profiling.cc:243] Waited 2.47 ms on lock 0x7b5800012398. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99e4fc0 00007f5ce99cf929 00007f5ce99cf3d2 00007f5ce2f46bd4
0709 13:19:13.962191 (+   217us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:13.987782 (+ 25591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:13.988028 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5691648}
I20260709 13:19:13.988997 15673 rpcz_store.cc:276] 0709 13:19:12.594230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.794843 (+200613us) service_pool.cc:224] Handling call
0709 13:19:13.856909 (+1062066us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540046e954. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:13.865653 (+  8744us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2436837760}
W20260709 13:19:13.831255 17867 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.014s	sys 0.053s
W20260709 13:19:13.987895 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 727450us
I20260709 13:19:13.853737 18048 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:13.853602 18064 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:13.908622 17777 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.022s	sys 0.033s
W20260709 13:19:13.967917 18106 log.cc:927] Time spent T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:19:14.466809 15598 raft_consensus.cc:606] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 LEADER]: Received request to transfer leadership to TS dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:13.851516 18089 log.cc:927] Time spent T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:19:14.392339 17945 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.008s	sys 0.018s
W20260709 13:19:14.468461 17945 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.008s	sys 0.018s
I20260709 13:19:14.221133 18046 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.470770 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 528) took 1539 ms. Trace:
W20260709 13:19:14.236639 15519 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.013s	sys 0.021s
W20260709 13:19:14.473387 16204 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.019s	sys 0.021s
W20260709 13:19:14.474073 16204 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.019s	sys 0.021s
I20260709 13:19:14.221554 15674 raft_consensus.cc:1240] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 5. Current term is 6. Ops: []
W20260709 13:19:13.865007 15493 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.926s	user 0.020s	sys 0.022s
W20260709 13:19:14.474960 15493 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.536s	user 0.021s	sys 0.058s
I20260709 13:19:14.467435 16238 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 883) took 1687 ms. Trace:
W20260709 13:19:14.475572 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 1.53736e+06us
I20260709 13:19:14.477819 15599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 424) took 1635 ms. Trace:
W20260709 13:19:13.870474 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.478477 15599 rpcz_store.cc:276] 0709 13:19:12.842286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.975220 (+132934us) service_pool.cc:224] Handling call
0709 13:19:12.975662 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:13.004732 (+ 29070us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:13.004771 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:13.062203 (+ 57432us) log.cc:844] Serialized 26 byte log entry
0709 13:19:13.573282 (+511079us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:13.573353 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:13.573479 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:14.477597 (+904118us) raft_consensus.cc:1685] finished
0709 13:19:14.477619 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.477760 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":295615,"raft.queue_time_us":19174,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":7901,"thread_start_us":471984,"threads_started":1,"wal-append.queue_time_us":573630}
W20260709 13:19:13.974990 18093 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 13:19:14.475862 16238 rpcz_store.cc:276] 0709 13:19:12.779651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.779767 (+   116us) service_pool.cc:224] Handling call
0709 13:19:12.780064 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:13.072987 (+292923us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:13.073020 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:13.073065 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:13.073102 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.467032 (+1393930us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b5400640ad4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:14.467123 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.467348 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":673158528}
W20260709 13:19:14.476954 15567 leader_election.cc:336] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.123s (SENT)
I20260709 13:19:14.471174 15523 rpcz_store.cc:276] 0709 13:19:12.976495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.976648 (+   153us) service_pool.cc:224] Handling call
0709 13:19:12.977015 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:13.008579 (+ 31564us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:13.008611 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:13.015478 (+  6867us) log.cc:844] Serialized 26 byte log entry
0709 13:19:13.544808 (+529330us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:13.544847 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:13.544942 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:14.470521 (+925579us) raft_consensus.cc:1685] finished
0709 13:19:14.470543 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.470698 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31156,"raft.queue_time_us":188233,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":3911,"thread_start_us":486601,"threads_started":1,"wal-append.queue_time_us":688831}
W20260709 13:19:13.969117 18091 log.cc:927] Time spent T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
W20260709 13:19:14.477116 15603 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.859s	user 0.012s	sys 0.004s
W20260709 13:19:14.474478 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 689128us
W20260709 13:19:14.396468 18000 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.998s	user 0.020s	sys 0.057s
W20260709 13:19:14.473230 15594 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.846s	user 0.013s	sys 0.034s
I20260709 13:19:13.852089 18047 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:13.960336 18024 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:14.488492 15603 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.870s	user 0.012s	sys 0.004s
I20260709 13:19:13.908813 17947 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:14.464864 15672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 1772) took 1568 ms. Trace:
I20260709 13:19:14.344771 17735 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:14.215914 17827 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:14.469105 17945 raft_consensus.cc:3055] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:19:14.466194 17777 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.360s	user 0.047s	sys 0.043s
I20260709 13:19:13.856381 17988 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:14.470032 18046 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:14.487924 18025 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.015s	sys 0.015s
I20260709 13:19:13.925029 18062 consensus_queue.cc:1059] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.567s
I20260709 13:19:14.484784 15567 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:14.495240 15596 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.322s	user 0.008s	sys 0.025s
W20260709 13:19:14.560359 15596 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.387s	user 0.020s	sys 0.030s
I20260709 13:19:13.981911 17826 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:14.546727 15524 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.012s	sys 0.021s
W20260709 13:19:14.561249 15524 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.012s	sys 0.021s
I20260709 13:19:14.548576 15603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 433) took 1653 ms. Trace:
W20260709 13:19:14.546586 15594 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.926s	user 0.013s	sys 0.035s
W20260709 13:19:14.545964 18000 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.294s	user 0.020s	sys 0.057s
W20260709 13:19:14.545704 17950 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.012s	sys 0.022s
I20260709 13:19:14.558315 17735 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:14.563295 17950 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.012s	sys 0.022s
I20260709 13:19:14.564050 17950 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:14.565809 17735 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:14.572549 18047 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:14.573315 18047 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:14.574414 17826 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:14.575537 18047 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:14.556138 17945 raft_consensus.cc:740] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:14.580294 17945 consensus_queue.cc:260] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:14.581372 17945 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:14.582175 18046 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:14.547884 18096 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.583315 18096 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:14.584965 17744 raft_consensus.cc:1064] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
W20260709 13:19:14.588136 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.588701 15676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 1760) took 2054 ms. Trace:
I20260709 13:19:14.589082 15676 rpcz_store.cc:276] 0709 13:19:12.534516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.926404 (+391888us) service_pool.cc:224] Handling call
0709 13:19:12.927220 (+   816us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:13.057637 (+130417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:13.057668 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:13.058163 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 13:19:13.989779 (+931616us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:13.989820 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:13.989887 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:14.588483 (+598596us) raft_consensus.cc:1685] finished
0709 13:19:14.588504 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.588645 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85564,"raft.queue_time_us":97191,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":15117,"thread_start_us":890080,"threads_started":1,"wal-append.queue_time_us":1503051}
I20260709 13:19:14.591197 18102 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.591768 18102 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:14.588014 15598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57344 (request call id 24) took 2335 ms. Trace:
I20260709 13:19:14.585623 18096 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:14.601233 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1611) took 2104 ms. Trace:
I20260709 13:19:14.583231 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1610) took 2102 ms. Trace:
I20260709 13:19:14.601225 15596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 1655) took 1798 ms. Trace:
I20260709 13:19:14.561729 15603 rpcz_store.cc:276] 0709 13:19:12.894942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.571438 (+676496us) service_pool.cc:224] Handling call
0709 13:19:13.597545 (+ 26107us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:13.597779 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:13.618307 (+ 20528us) log.cc:844] Serialized 14 byte log entry
0709 13:19:14.548230 (+929923us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:14.548290 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:14.548333 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.548398 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.548520 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20906,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":89,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":3412,"thread_start_us":929556,"threads_started":1}
I20260709 13:19:14.570158 17958 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:14.603307 15519 rpcz_store.cc:276] 0709 13:19:12.497121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.499232 (+  2111us) service_pool.cc:224] Handling call
0709 13:19:12.656280 (+157048us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:12.912822 (+256542us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:12.912852 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:12.927025 (+ 14173us) log.cc:844] Serialized 26 byte log entry
0709 13:19:13.623879 (+696854us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:13.968548 (+344669us) spinlock_profiling.cc:243] Waited 64.3 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:19:14.471961 (+503413us) log.cc:844] Serialized 14 byte log entry
0709 13:19:14.472226 (+   265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.472300 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:14.559452 (+ 87152us) raft_consensus.cc:1685] finished
0709 13:19:14.601021 (+ 41569us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.601162 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":882893,"prepare.run_cpu_time_us":6871,"prepare.run_wall_time_us":37898,"raft.queue_time_us":258661,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":44913,"spinlock_wait_cycles":147777408,"thread_start_us":1590592,"threads_started":3,"wal-append.queue_time_us":835711}
I20260709 13:19:14.603478 15527 rpcz_store.cc:276] 0709 13:19:12.481386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.481543 (+   157us) service_pool.cc:224] Handling call
0709 13:19:12.497036 (+ 15493us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:19:13.142905 (+645869us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:13.142939 (+    34us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:19:13.276103 (+133164us) log.cc:844] Serialized 608 byte log entry
0709 13:19:13.687443 (+411340us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:13.965168 (+277725us) spinlock_profiling.cc:243] Waited 6.06 ms on lock 0x7b4800002958. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce9934aa1 00007f5ce992bb37 00007f5ce992b9ca 00007f5ce9a2326f 00007f5ce9a22f37 00007f5ce9a3755c 00007f5ce9a37508 00007f5ce9a37488 00007f5ce9a37443 00007f5ce9a366ec 00007f5ce8960a96
0709 13:19:13.967176 (+  2008us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:19:14.235857 (+268681us) log.cc:844] Serialized 14 byte log entry
0709 13:19:14.464996 (+229139us) log.cc:844] Serialized 14 byte log entry
0709 13:19:14.466382 (+  1386us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.582895 (+116513us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b540041f854. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:14.582941 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:14.582993 (+    52us) raft_consensus.cc:1685] finished
0709 13:19:14.583007 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.583171 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8518,"prepare.queue_time_us":960179,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":69,"raft.queue_time_us":309038,"raft.run_cpu_time_us":11818,"raft.run_wall_time_us":1043506,"spinlock_wait_cycles":427326464,"thread_start_us":1308366,"threads_started":3,"wal-append.queue_time_us":570687}
I20260709 13:19:14.551241 15672 rpcz_store.cc:276] 0709 13:19:12.896209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.262085 (+365876us) service_pool.cc:224] Handling call
0709 13:19:13.262428 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:13.543949 (+281521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.464546 (+920597us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b54002cd154. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:14.464796 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37011712}
I20260709 13:19:14.601894 18102 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:14.626119 15641 leader_election.cc:336] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.628s (SENT)
W20260709 13:19:14.626407 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.626657 15641 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:14.558776 17827 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:14.627408 18046 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:14.628577 15641 leader_election.cc:336] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.978s (SENT)
I20260709 13:19:14.628988 15641 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:14.603116 18064 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:14.629537 15519 raft_consensus.cc:3055] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:14.629719 18046 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:14.549548 18100 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:19:14.558435 18025 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.015s	sys 0.015s
I20260709 13:19:14.637359 18100 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:14.630015 15519 raft_consensus.cc:740] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:14.640574 15519 consensus_queue.cc:260] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:14.641537 15519 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:14.553435 17947 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:14.643144 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 166218us
I20260709 13:19:14.601547 15598 rpcz_store.cc:276] 0709 13:19:12.252773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.252958 (+   185us) service_pool.cc:224] Handling call
0709 13:19:14.587828 (+2334870us) spinlock_profiling.cc:243] Waited 2.05 s on lock 0x7b540032c3d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a05853 00007f5ce99d360a 00007f5ce2f597d4 00007f5ce2f59762 00007f5ce2f59691 00007f5ce2f5960c 00007f5ce2f588c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:14.587928 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4717254912}
I20260709 13:19:14.628901 17827 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:14.645705 15567 connection.cc:256] RPC call timeout handler was delayed by -1.279012529000056s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1807s for a response
I20260709 13:19:14.647413 18100 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:14.636795 18047 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:19:14.653797 15567 leader_election.cc:336] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.136s (SENT)
I20260709 13:19:14.655658 15567 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:14.658871 15567 leader_election.cc:336] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.158s (SENT)
W20260709 13:19:14.697290 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 310215us
W20260709 13:19:14.710583 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 51740.3us
W20260709 13:19:14.718379 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.699870 18025 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:14.651846 18047 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:14.706763 18046 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:14.719449 17867 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:14.749446 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 265390us
I20260709 13:19:14.755554 15566 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:14.756937 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:14.753890 15493 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:14.751348 15676 raft_consensus.cc:2955] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:14.760475 15641 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:14.762837 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:19:14.754740 18063 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.928s	user 0.018s	sys 0.019s
I20260709 13:19:14.758463 17929 consensus_queue.cc:1048] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:19:14.781764 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:14.772380 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:14.756906 18047 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:14.765388 18025 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:14.784431 18025 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:14.740881 17867 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:14.784855 18025 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:14.778225 18063 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.951s	user 0.018s	sys 0.019s
I20260709 13:19:14.762531 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:14.786492 17867 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:14.741747 18046 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:14.789260 18046 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:14.790230 17827 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:14.790762 15671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 535) took 3295 ms. Trace:
I20260709 13:19:14.791013 17827 consensus_queue.cc:1048] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:14.791577 17826 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:19:14.792635 15641 leader_election.cc:336] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.984s (SENT)
I20260709 13:19:14.792891 17953 consensus_queue.cc:1059] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:14.793447 17947 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:14.758641 18024 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:14.793769 17953 raft_consensus.cc:3055] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:14.794332 17953 raft_consensus.cc:740] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:14.795244 17953 consensus_queue.cc:260] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:14.795990 15525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 531) took 1798 ms. Trace:
I20260709 13:19:14.796500 17953 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:14.747947 17744 consensus_queue.cc:237] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:14.794332 18024 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:14.800323 18024 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:14.800845 18024 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:14.802366 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:14.803273 15598 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:14.796363 15525 rpcz_store.cc:276] 0709 13:19:12.997290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.997406 (+   116us) service_pool.cc:224] Handling call
0709 13:19:14.795279 (+1797873us) spinlock_profiling.cc:243] Waited 3.93 ms on lock 0x7b5800003698. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99e4fc0 00007f5ce99cf929 00007f5ce99cf3d2 00007f5ce2f46bd4
0709 13:19:14.795482 (+   203us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:14.795659 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:14.795683 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:14.795735 (+    52us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:14.795763 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.795822 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.795935 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9039744}
I20260709 13:19:14.806942 15520 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 8. (term mismatch)
I20260709 13:19:14.807394 15520 pending_rounds.cc:85] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 7
I20260709 13:19:14.807605 15520 pending_rounds.cc:107] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 13:19:14.807837 15520 raft_consensus.cc:2889] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:14.809847 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 524) took 1882 ms. Trace:
I20260709 13:19:14.810478 15520 rpcz_store.cc:276] 0709 13:19:12.928267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.967261 (+ 38994us) service_pool.cc:224] Handling call
0709 13:19:13.007941 (+ 40680us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:14.808247 (+1800306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.809588 (+  1341us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b540007eed4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:14.809773 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4136467840}
I20260709 13:19:14.801337 18024 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:14.812551 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:14.813426 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:14.813720 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:14.814939 15603 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:14.805316 15524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 539) took 1023 ms. Trace:
I20260709 13:19:14.815806 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:14.816448 15599 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:14.792555 17827 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:19:14.822196 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:14.810062 17928 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.824132 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:14.818120 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:14.832715 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:14.833478 16238 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
W20260709 13:19:14.834360 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 286007us
I20260709 13:19:14.836683 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:14.837414 16238 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:14.845424 15493 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:14.847599 15493 connection.cc:256] RPC call timeout handler was delayed by -2.445489791999762s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27340000000000003s for a response
I20260709 13:19:14.867636 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:14.869294 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:14.874121 15493 connection.cc:256] RPC call timeout handler was delayed by -2.1619897919999858s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3049s for a response
W20260709 13:19:14.874558 15493 connection.cc:256] RPC call timeout handler was delayed by -1.9938641769999776s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1573s for a response
W20260709 13:19:14.874840 15493 connection.cc:256] RPC call timeout handler was delayed by -1.9497641769999063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1622s for a response
I20260709 13:19:14.875061 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:14.876317 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:19:14.875100 15493 connection.cc:256] RPC call timeout handler was delayed by -1.8372641769999518s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17470000000000002s for a response
W20260709 13:19:14.877135 15493 connection.cc:256] RPC call timeout handler was delayed by -1.6788641769999231s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19230000000000003s for a response
W20260709 13:19:14.877436 15493 connection.cc:256] RPC call timeout handler was delayed by -1.6500641770001039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1955s for a response
W20260709 13:19:14.877696 15493 connection.cc:256] RPC call timeout handler was delayed by -1.6005641770000238s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20099999999999998s for a response
I20260709 13:19:14.831907 17929 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:14.877978 15493 connection.cc:256] RPC call timeout handler was delayed by -1.5447641769999336s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20720000000000002s for a response
W20260709 13:19:14.886049 15493 connection.cc:256] RPC call timeout handler was delayed by -1.4745641770000475s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.215s for a response
I20260709 13:19:14.886584 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:14.886749 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:19:14.887234 15493 connection.cc:256] RPC call timeout handler was delayed by -1.3881641769999078s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22460000000000002s for a response
W20260709 13:19:14.887578 15493 connection.cc:256] RPC call timeout handler was delayed by -1.0911641769998823s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2576s for a response
I20260709 13:19:14.887832 16235 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:14.887545 16237 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:14.885951 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 114964us
I20260709 13:19:14.891145 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:14.893404 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 133906us
I20260709 13:19:14.828442 17928 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:14.888123 17958 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.908015 15493 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:14.834399 18104 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:19:14.878232 18063 consensus_queue.cc:1048] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.036s
W20260709 13:19:14.896991 15567 leader_election.cc:336] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.529s (SENT)
I20260709 13:19:14.811861 15671 rpcz_store.cc:276] 0709 13:19:11.495280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.877638 (+382358us) service_pool.cc:224] Handling call
0709 13:19:11.886545 (+  8907us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:12.098192 (+211647us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:12.104113 (+  5921us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:12.117177 (+ 13064us) log.cc:844] Serialized 26 byte log entry
0709 13:19:12.353881 (+236704us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:12.361193 (+  7312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.837384 (+476191us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:19:14.790443 (+1953059us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b540026b6d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:14.790499 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:14.790540 (+    41us) raft_consensus.cc:1685] finished
0709 13:19:14.790555 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.790689 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":40043,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":5162,"spinlock_wait_cycles":901063936,"thread_start_us":214059,"threads_started":1,"wal-append.queue_time_us":260048,"wal-append.run_cpu_time_us":5710,"wal-append.run_wall_time_us":1827198}
I20260709 13:19:14.866618 17944 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:19:14.908514 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:14.932708 15600 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:14.932726 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:14.922287 15493 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:14.881579 17978 consensus_queue.cc:1059] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:19:14.940058 17944 raft_consensus.cc:3055] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:14.923426 18062 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:14.940460 17944 raft_consensus.cc:740] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:14.923604 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:14.942034 16237 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:14.942225 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:14.942488 18024 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:14.942817 15601 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:19:14.942955 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:14.943763 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:14.941322 16240 raft_consensus.cc:2393] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:14.945837 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:14.946360 16240 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:14.924607 17958 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:14.927150 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:14.928133 15567 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:14.895269 18122 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.938493 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:14.954649 15600 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:19:14.948864 15493 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:14.950281 16237 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:14.950989 16238 raft_consensus.cc:2410] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:19:14.917434 16231 raft_consensus.cc:1240] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:14.951786 15525 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
W20260709 13:19:14.952142 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 55194.5us
I20260709 13:19:14.917739 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:14.921497 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:19:14.956794 15567 leader_election.cc:336] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.530s (SENT)
I20260709 13:19:14.968356 15567 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:14.957306 16235 raft_consensus.cc:1240] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: []
W20260709 13:19:14.949288 18042 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.432s	user 0.029s	sys 0.022s
I20260709 13:19:14.929301 18128 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:14.945022 15601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4fd164d74054954b750fe9958e66778"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:19:14.969980 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:14.941358 17944 consensus_queue.cc:260] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:14.979776 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:14.977838 15601 raft_consensus.cc:606] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:14.981896 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:14.955422 18062 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:14.983390 15603 raft_consensus.cc:2410] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
W20260709 13:19:14.978612 18042 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.459s	user 0.029s	sys 0.022s
I20260709 13:19:14.985450 18046 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:14.983639 18074 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:14.981453 17827 raft_consensus.cc:1064] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4: attempting to promote NON_VOTER dd1a14c3173d49dc9b55bb116e3783c9 to VOTER
I20260709 13:19:14.989322 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:14.987592 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 92424.4us
I20260709 13:19:14.995644 15525 raft_consensus.cc:2393] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 7: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:14.988047 15603 raft_consensus.cc:1240] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:14.944995 18024 consensus_queue.cc:1059] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:19:14.997485 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 540) took 1214 ms. Trace:
I20260709 13:19:14.998469 15523 rpcz_store.cc:276] 0709 13:19:13.782904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.485658 (+702754us) service_pool.cc:224] Handling call
0709 13:19:14.486100 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:14.489054 (+  2954us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:14.489084 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:14.489561 (+   477us) log.cc:844] Serialized 26 byte log entry
0709 13:19:14.824203 (+334642us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:14.831463 (+  7260us) log.cc:844] Serialized 14 byte log entry
0709 13:19:14.831663 (+   200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.831737 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:14.997265 (+165528us) raft_consensus.cc:1685] finished
0709 13:19:14.997283 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.997422 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14745,"prepare.queue_time_us":6846,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":109649,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":10290,"thread_start_us":274255,"threads_started":1,"wal-append.queue_time_us":422368}
W20260709 13:19:15.005834 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 57046.1us
I20260709 13:19:14.986793 18131 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:15.008073 15523 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:14.981500 18128 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:14.978480 18025 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:14.966013 18122 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:14.997745 18141 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:15.023469 17958 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:15.024215 17958 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:15.025046 15493 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:15.011075 18128 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:15.005329 17944 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:15.011945 18062 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:15.040989 15493 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:15.005234 18074 raft_consensus.cc:697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:15.044762 18074 consensus_queue.cc:237] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:15.018460 18046 raft_consensus.cc:697] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:15.049932 15493 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:19:15.051609 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 57996.4us
I20260709 13:19:15.052477 15493 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:15.056474 15567 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:15.061347 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.061800 18046 consensus_queue.cc:237] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:15.063448 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:15.064121 15668 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 2.
I20260709 13:19:15.066586 16240 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:15.025871 17958 consensus_queue.cc:1059] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:19:15.068055 15642 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:15.026018 18131 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:19:15.072652 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:15.073487 15641 leader_election.cc:336] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 3.180s (SENT)
I20260709 13:19:15.080595 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:15.080938 15641 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:15.081408 15675 raft_consensus.cc:2393] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:15.081722 15493 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:19:15.078814 18137 log.cc:927] Time spent T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:19:15.067216 18128 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:15.115783 15567 leader_election.cc:336] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.644s (SENT)
W20260709 13:19:15.133700 16204 leader_election.cc:336] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.143s (SENT)
I20260709 13:19:15.134289 16204 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:15.085179 15671 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:19:15.083114 18025 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:15.042727 18062 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:15.220350 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 86697.1us
I20260709 13:19:15.089056 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:15.098155 18000 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:19:15.259528 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.221207 18145 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:15.218664 17978 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:19:15.254540 18065 log.cc:927] Time spent T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 2.590s	user 0.021s	sys 0.024s
I20260709 13:19:15.223198 18139 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:19:15.252668 18105 log.cc:927] Time spent T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 13:19:15.252094 18008 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:15.249435 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:15.251665 18154 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:15.261648 15668 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:15.268788 18000 raft_consensus.cc:697] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:15.285079 18000 consensus_queue.cc:237] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:15.293016 18134 log.cc:927] Time spent T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
W20260709 13:19:15.294422 18133 log.cc:927] Time spent T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.252s	user 0.000s	sys 0.000s
I20260709 13:19:15.275188 17978 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:15.268571 18074 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:19:15.318079 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.290233 15672 raft_consensus.cc:2393] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:19:15.325692 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 63045.5us
I20260709 13:19:15.384871 17744 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.385488 17744 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:15.326269 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 288150us
W20260709 13:19:15.342900 15568 connection.cc:256] RPC call timeout handler was delayed by -1.090844165000135s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15510000000000002s for a response
I20260709 13:19:15.390926 15568 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:15.308902 18127 log.cc:927] Time spent T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.314s	user 0.001s	sys 0.000s
I20260709 13:19:15.324029 18024 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:15.399668 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:15.383448 18129 log.cc:927] Time spent T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
W20260709 13:19:15.382076 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 308608us
I20260709 13:19:15.284938 18062 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:15.449506 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 44791.6us
W20260709 13:19:15.444947 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.488247 17953 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:15.488886 17953 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:15.491447 17953 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:15.480829 15597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 423) took 3356 ms. Trace:
I20260709 13:19:15.501174 15597 rpcz_store.cc:276] 0709 13:19:12.130429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.136093 (+  5664us) service_pool.cc:224] Handling call
0709 13:19:12.136462 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:12.136670 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:12.136695 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:12.204291 (+ 67596us) log.cc:844] Serialized 26 byte log entry
0709 13:19:12.391237 (+186946us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:12.391281 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.391368 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:15.011332 (+2619964us) spinlock_profiling.cc:243] Waited 65.2 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:19:15.458154 (+446822us) raft_consensus.cc:1685] finished
0709 13:19:15.458179 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:15.480725 (+ 22546us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149951360,"thread_start_us":129781,"threads_started":1,"wal-append.queue_time_us":433079}
W20260709 13:19:15.512163 18156 log.cc:927] Time spent T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:19:15.512547 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:15.513439 15597 raft_consensus.cc:2393] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:15.441201 18024 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:15.722401 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 328273us
I20260709 13:19:15.605414 15595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 1650) took 3502 ms. Trace:
W20260709 13:19:15.514816 18140 log.cc:927] Time spent T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.435s	user 0.000s	sys 0.001s
I20260709 13:19:15.516525 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:15.732995 15603 raft_consensus.cc:2368] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5.
I20260709 13:19:15.730391 15568 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:15.719838 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:19:15.643193 15493 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.002s	sys 0.023s
W20260709 13:19:15.736567 15493 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.003s	sys 0.024s
I20260709 13:19:15.719405 15566 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:19:15.551932 18146 log.cc:927] Time spent T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
W20260709 13:19:15.501629 18143 log.cc:927] Time spent T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 13:19:15.742836 15568 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:15.730221 15595 rpcz_store.cc:276] 0709 13:19:12.102825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.105071 (+  2246us) service_pool.cc:224] Handling call
0709 13:19:12.107913 (+  2842us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:12.108244 (+   331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:12.108269 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:14.708083 (+2599814us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b4c00561240. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce2cd7a20 00007f5ce2ce8a29 00007f5ce2cae644 00007f5ce2cd5f2d 00007f5ce2cd5b93 00007f5ce4a0618b 00007f5ce4a06cb5 00007f5ce4a08004 00007f5ce4a07729 00007f5ce4a0cd96 00007f5ce4a01adb
0709 13:19:14.723086 (+ 15003us) log.cc:844] Serialized 556 byte log entry
0709 13:19:14.955240 (+232154us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:14.955276 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.955340 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:15.604811 (+649471us) raft_consensus.cc:1685] finished
0709 13:19:15.604830 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:15.605333 (+   503us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39695104,"thread_start_us":222495,"threads_started":1,"wal-append.queue_time_us":316503}
W20260709 13:19:15.788849 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 707220us
I20260709 13:19:15.733848 15527 raft_consensus.cc:2376] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5. Re-sending same reply.
I20260709 13:19:15.790338 15493 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:15.796222 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:15.797032 16240 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:15.800478 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 57743.3us
I20260709 13:19:15.802879 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:15.814528 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:15.818465 16238 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:15.820363 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:15.822082 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:15.828609 18155 log.cc:927] Time spent T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 13:19:15.830760 15599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 441) took 1001 ms. Trace:
I20260709 13:19:15.831236 15599 rpcz_store.cc:276] 0709 13:19:14.828908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.829078 (+   170us) service_pool.cc:224] Handling call
0709 13:19:14.830560 (+  1482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:14.835172 (+  4612us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:14.835205 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:14.870386 (+ 35181us) log.cc:844] Serialized 26 byte log entry
0709 13:19:14.982284 (+111898us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:14.982757 (+   473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.982820 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:15.830492 (+847672us) raft_consensus.cc:1685] finished
0709 13:19:15.830515 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:15.830676 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20902,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":8693,"thread_start_us":68465,"threads_started":1,"wal-append.queue_time_us":134988}
I20260709 13:19:15.840094 15674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 1771) took 3008 ms. Trace:
I20260709 13:19:15.840576 15674 rpcz_store.cc:276] 0709 13:19:12.831306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.167822 (+336516us) service_pool.cc:224] Handling call
0709 13:19:13.175349 (+  7527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:14.481025 (+1305676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:15.839801 (+1358776us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54001a7a54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:15.840017 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2393278720}
I20260709 13:19:14.603662 15596 rpcz_store.cc:276] 0709 13:19:12.802768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.802976 (+   208us) service_pool.cc:224] Handling call
0709 13:19:12.803341 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:12.803579 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:13.172855 (+369276us) log.cc:844] Serialized 14 byte log entry
0709 13:19:14.600899 (+1428044us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:14.600951 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:14.601014 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.601076 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.601161 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39954,"prepare.queue_time_us":690696,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":42093,"thread_start_us":1756676,"threads_started":2}
I20260709 13:19:15.845268 17988 raft_consensus.cc:3055] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:15.845769 17988 raft_consensus.cc:740] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:15.846393 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:15.846684 17988 consensus_queue.cc:260] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:15.847836 17988 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:15.848461 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:15.849197 16231 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:15.727334 17953 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:15.849675 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 530) took 2853 ms. Trace:
I20260709 13:19:15.852876 15526 rpcz_store.cc:276] 0709 13:19:12.996444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.996555 (+   111us) service_pool.cc:224] Handling call
0709 13:19:12.996895 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:13.990506 (+993611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:15.849386 (+1858880us) spinlock_profiling.cc:243] Waited 859 ms on lock 0x7b5400187754. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:15.849604 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1976718592}
I20260709 13:19:15.732191 18122 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:15.731510 18131 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:15.854528 17744 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:15.855000 18168 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:15.855551 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:15.855072 17953 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:15.855872 17942 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:15.737247 18008 consensus_queue.cc:1059] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.121s
I20260709 13:19:15.731190 17944 consensus_queue.cc:1059] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.122s
I20260709 13:19:15.868359 16238 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEARNER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 13:19:15.863502 15668 consensus_meta.cc:287] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.572s	user 0.009s	sys 0.000s
I20260709 13:19:15.858373 17978 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:15.813184 18139 consensus_queue.cc:1059] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:19:15.863739 18064 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:15.872466 17942 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:15.867723 18131 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:19:15.925487 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:15.963229 16236 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:15.885485 15668 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:15.898355 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:15.921823 16240 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
W20260709 13:19:15.941691 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.884404 18149 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:15.959076 15669 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:19:15.960511 17920 consensus_queue.cc:260] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:15.994683 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:15.995152 17953 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:19:15.995455 18108 log.cc:927] Time spent T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 2.144s	user 0.008s	sys 0.022s
I20260709 13:19:15.995774 16233 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:15.925613 17942 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:15.996829 15602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 430) took 3153 ms. Trace:
I20260709 13:19:15.917899 15642 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:15.994192 18149 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:15.899381 18063 raft_consensus.cc:3055] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:15.961725 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:15.995198 17920 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:15.964114 18064 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:15.998449 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 80633.4us
I20260709 13:19:15.997200 15602 rpcz_store.cc:276] 0709 13:19:12.843707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.491383 (+647676us) service_pool.cc:224] Handling call
0709 13:19:13.491740 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:13.492132 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:13.492160 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:13.504401 (+ 12241us) log.cc:844] Serialized 26 byte log entry
0709 13:19:13.782550 (+278149us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:13.782589 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:13.782657 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:15.968143 (+2185486us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:19:15.996522 (+ 28379us) raft_consensus.cc:1685] finished
0709 13:19:15.996547 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:15.996771 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2415,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"spinlock_wait_cycles":33340288,"thread_start_us":206284,"threads_started":1,"wal-append.queue_time_us":304478}
I20260709 13:19:16.017216 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:16.006567 18170 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:19:16.025784 15642 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.66:41593 timed out after 2.110s (SENT) [suppressed 50 similar messages]
I20260709 13:19:16.006067 18063 raft_consensus.cc:740] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:16.030468 15674 raft_consensus.cc:1240] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 13:19:16.029737 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 328695us
I20260709 13:19:16.020176 17729 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:16.029773 18170 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:16.019165 18064 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:16.107582 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 388278us
W20260709 13:19:16.109123 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 83374.2us
I20260709 13:19:16.112066 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:16.113037 15673 raft_consensus.cc:1240] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:16.114699 16240 raft_consensus.cc:1240] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:16.098023 17729 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:16.116972 16238 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:19:16.031723 18038 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
W20260709 13:19:16.107990 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 305793us
I20260709 13:19:16.118633 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:16.108888 18170 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:16.105759 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:16.119234 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:16.117707 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:16.120182 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:16.127635 15673 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:16.115700 17978 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.131592 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:19:16.136323 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 408116us
I20260709 13:19:16.117422 18162 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.119004 17729 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:16.143350 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:16.137571 15568 leader_election.cc:336] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627): Timed out: RequestConsensusVote RPC to 127.14.23.68:39627 timed out after 2.164s (SENT)
I20260709 13:19:16.151870 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:16.156801 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:16.158483 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:16.164649 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:16.164613 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:16.166239 15602 raft_consensus.cc:2376] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4. Re-sending same reply.
I20260709 13:19:16.167886 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:16.169407 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:16.140995 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:16.174147 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:16.147131 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:16.163200 18169 log.cc:927] Time spent T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 13:19:16.180058 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:16.149461 18024 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:16.136243 17978 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.161948 18141 raft_consensus.cc:3055] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:16.182682 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:16.184921 18051 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:16.188388 15594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 1656) took 3385 ms. Trace:
I20260709 13:19:16.198617 15594 rpcz_store.cc:276] 0709 13:19:12.803042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.817124 (+ 14082us) service_pool.cc:224] Handling call
0709 13:19:12.817499 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:13.161774 (+344275us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:19:13.593819 (+432045us) log.cc:844] Serialized 14 byte log entry
0709 13:19:14.625174 (+1031355us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:14.625237 (+    63us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:14.625292 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:16.188106 (+1562814us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b540025ab54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:16.188156 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:16.188309 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40560,"prepare.queue_time_us":328462,"prepare.run_cpu_time_us":23,"prepare.run_wall_time_us":24,"raft.queue_time_us":63631,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"spinlock_wait_cycles":760080128,"thread_start_us":1233734,"threads_started":2,"wal-append.queue_time_us":1044021,"wal-append.run_cpu_time_us":1372,"wal-append.run_wall_time_us":1072162}
I20260709 13:19:16.201845 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:16.203272 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:16.203887 15672 raft_consensus.cc:2376] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6. Re-sending same reply.
I20260709 13:19:16.204774 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:16.190656 15601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57344 (request call id 25) took 1301 ms. Trace:
I20260709 13:19:16.210007 15601 rpcz_store.cc:276] 0709 13:19:14.889412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.944854 (+ 55442us) service_pool.cc:224] Handling call
0709 13:19:15.826897 (+882043us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b5400033520. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4988520 00007f5ce4a05652 00007f5ce99d360a 00007f5ce2f597d4 00007f5ce2f59762 00007f5ce2f59691 00007f5ce2f5960c 00007f5ce2f588c2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:16.190409 (+363512us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b4c0041fb00. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce2cd7a20 00007f5ce2ce8a29 00007f5ce2cae644 00007f5ce2cd5f2d 00007f5ce2cd5b93 00007f5ce4a05c92 00007f5ce4a0584a 00007f5ce99d360a 00007f5ce2f597d4 00007f5ce2f59762 00007f5ce2f59691
0709 13:19:16.190557 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59171456}
I20260709 13:19:16.211604 17942 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:16.212069 18064 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:16.213415 15673 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
W20260709 13:19:16.216933 15641 leader_election.cc:336] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.643s (SENT)
I20260709 13:19:16.220222 15641 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:16.220484 18149 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:16.218122 18141 raft_consensus.cc:740] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:16.221817 18141 consensus_queue.cc:260] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.223052 18141 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:16.118310 18063 consensus_queue.cc:260] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.225713 18063 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:16.220240 17978 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:16.227013 16203 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:16.219132 16236 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:16.233317 15670 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.792s	user 0.016s	sys 0.060s
W20260709 13:19:16.234140 15670 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.016s	sys 0.060s
I20260709 13:19:16.235113 17978 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:16.193094 18176 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:16.234795 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:16.237191 18176 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:16.237321 17978 raft_consensus.cc:697] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:16.238123 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:16.146265 18162 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.238636 17978 consensus_queue.cc:237] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.239078 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:16.241087 18176 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:16.241343 15670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 1784) took 1177 ms. Trace:
I20260709 13:19:16.241760 15670 rpcz_store.cc:276] 0709 13:19:15.063920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.068262 (+  4342us) service_pool.cc:224] Handling call
0709 13:19:15.068580 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:15.068787 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:19:15.435257 (+366470us) log.cc:844] Serialized 14 byte log entry
0709 13:19:16.241020 (+805763us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:16.241067 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:16.241111 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:16.241173 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:16.241277 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15614,"prepare.queue_time_us":947299,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":6472,"raft.run_cpu_time_us":46,"raft.run_wall_time_us":47,"thread_start_us":1150569,"threads_started":2}
I20260709 13:19:16.243803 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:16.244720 15601 raft_consensus.cc:2410] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:19:16.246101 15641 leader_election.cc:336] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.673s (SENT)
I20260709 13:19:16.246591 15641 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:16.252674 17716 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:16.253458 15668 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:16.245815 17729 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.255522 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 535) took 3188 ms. Trace:
I20260709 13:19:16.256150 17729 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:16.256906 18162 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:16.255889 15522 rpcz_store.cc:276] 0709 13:19:13.067107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.536133 (+469026us) service_pool.cc:224] Handling call
0709 13:19:13.575714 (+ 39581us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:13.643259 (+ 67545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:16.255264 (+2612005us) spinlock_profiling.cc:243] Waited 66.9 ms on lock 0x7b540029c7d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:16.255461 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153898880}
I20260709 13:19:16.262481 15670 raft_consensus.cc:1240] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:16.263787 16236 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:16.269515 18176 consensus_queue.cc:1059] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:16.272832 17867 raft_consensus.cc:3055] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:16.273257 17867 raft_consensus.cc:740] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:16.269438 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:16.274159 17867 consensus_queue.cc:260] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.280421 18170 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.268342 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:16.283202 15670 raft_consensus.cc:2393] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 5: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:16.244043 18000 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:16.285672 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:16.287420 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:16.287834 15670 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:19:16.292403 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:16.292689 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:16.275189 17867 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:16.296362 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:16.309240 15670 raft_consensus.cc:1240] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:16.284993 17777 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.318719 16233 raft_consensus.cc:1240] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 5. Current term is 6. Ops: [4.6-5.7]
W20260709 13:19:16.319918 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.321282 17944 raft_consensus.cc:3055] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:16.289565 18163 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:19:16.321830 17944 raft_consensus.cc:740] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:16.322978 17944 consensus_queue.cc:260] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.324255 17944 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:16.324501 15598 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:19:16.325151 15598 raft_consensus.cc:606] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Received request to transfer leadership to TS dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:16.325489 15598 raft_consensus.cc:614] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:16.320626 17998 raft_consensus.cc:2955] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 2 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:16.327068 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:19:16.329079 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:16.329859 15674 raft_consensus.cc:2393] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:16.315521 15642 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:16.321223 17777 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.338084 15674 raft_consensus.cc:1240] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:16.339864 18170 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:16.340294 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:16.342772 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:16.343868 15597 raft_consensus.cc:2393] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:16.345710 15599 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:19:16.346095 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:16.346330 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:16.343920 16231 raft_consensus.cc:1240] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:16.346922 15598 raft_consensus.cc:2368] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5.
I20260709 13:19:16.342092 17729 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:16.347380 15596 raft_consensus.cc:2955] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:16.348392 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:16.346092 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:16.345621 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:16.352634 16238 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:16.354869 16232 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:16.357270 16235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53976 (request call id 483) took 1472 ms. Trace:
I20260709 13:19:16.360262 18168 raft_consensus.cc:3055] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:16.360690 18168 raft_consensus.cc:740] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:16.361445 18168 consensus_queue.cc:260] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.362403 18168 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:16.363008 17729 raft_consensus.cc:697] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:16.364104 17729 consensus_queue.cc:237] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:16.366679 15641 leader_election.cc:336] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.786s (SENT)
I20260709 13:19:16.367236 15641 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:16.368865 17744 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:16.357697 16235 rpcz_store.cc:276] 0709 13:19:14.885068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.889123 (+  4055us) service_pool.cc:224] Handling call
0709 13:19:14.957120 (+ 67997us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:14.969315 (+ 12195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:16.357063 (+1387748us) spinlock_profiling.cc:243] Waited 43.9 ms on lock 0x7b54005848d0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:16.357209 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102243456}
I20260709 13:19:16.383227 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:16.383931 15675 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 2.
I20260709 13:19:14.815547 15524 rpcz_store.cc:276] 0709 13:19:13.782085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.782255 (+   170us) service_pool.cc:224] Handling call
0709 13:19:13.786153 (+  3898us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:13.786355 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:13.957466 (+171111us) log.cc:844] Serialized 14 byte log entry
0709 13:19:14.804785 (+847319us) spinlock_profiling.cc:243] Waited 5.02 ms on lock 0x7b18004948c0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce45b34bc 00007f5ce190148b 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab23a
0709 13:19:14.804972 (+   187us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:14.805022 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:14.805068 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.805134 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.805261 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":771536,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":7564,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":3866,"spinlock_wait_cycles":11553024,"thread_start_us":770264,"threads_started":2,"wal-append.queue_time_us":767860}
I20260709 13:19:16.400306 15568 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:16.393695 16235 raft_consensus.cc:2955] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:16.404661 18131 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:16.412904 18131 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:16.413269 18131 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:16.422778 15568 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
W20260709 13:19:16.427902 15641 leader_election.cc:336] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.838s (SENT)
I20260709 13:19:16.428741 15641 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:16.430747 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:16.432533 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:16.434419 16239 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:16.436009 16239 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:16.436803 17867 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:16.437341 17867 raft_consensus.cc:697] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:16.437955 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:16.438232 17867 consensus_queue.cc:237] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.439270 17928 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:16.448626 15566 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:16.450857 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:16.452184 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:16.452744 15675 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:19:16.455147 15642 leader_election.cc:400] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5.
I20260709 13:19:16.455606 15642 leader_election.cc:403] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:19:16.458214 15566 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:16.459188 15525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 548) took 1480 ms. Trace:
I20260709 13:19:16.460191 15525 rpcz_store.cc:276] 0709 13:19:14.979186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.012356 (+ 33170us) service_pool.cc:224] Handling call
0709 13:19:15.020449 (+  8093us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:16.344994 (+1324545us) spinlock_profiling.cc:243] Waited 4.07 ms on lock 0x7b1800458b00. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce45b34bc 00007f5ce190148b 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191b455 00007f5ce4a03060 00007f5ce4a03bfb 00007f5ce4a0e64b
0709 13:19:16.345734 (+   740us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:16.345764 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:16.346361 (+   597us) log.cc:844] Serialized 52 byte log entry
0709 13:19:16.425387 (+ 79026us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:16.425428 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:16.425498 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:16.425534 (+    36us) raft_consensus.cc:1685] finished
0709 13:19:16.425545 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:16.459116 (+ 33571us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19356,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":6422,"spinlock_wait_cycles":9363840,"thread_start_us":107032,"threads_started":2,"wal-append.queue_time_us":61850}
I20260709 13:19:16.460559 15596 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:19:16.462356 15596 pending_rounds.cc:85] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting all ops after (but not including) 6
I20260709 13:19:16.462646 17984 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:16.463189 17984 raft_consensus.cc:697] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:16.464177 17984 consensus_queue.cc:237] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.464591 16204 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:16.462664 15596 pending_rounds.cc:107] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:19:16.465945 15596 raft_consensus.cc:2889] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:16.403216 17942 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:16.467849 15566 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:16.469122 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:16.470187 15670 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:16.469485 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:16.474601 15675 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:16.463092 15671 raft_consensus.cc:1240] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:19:16.479516 15566 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:16.462443 18064 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:16.466342 18170 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:16.466127 17777 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:16.509629 15567 leader_election.cc:336] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.862s (SENT)
I20260709 13:19:16.511137 15567 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:16.466153 17729 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:19:16.478500 18026 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5.
I20260709 13:19:16.481524 15672 raft_consensus.cc:1240] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:19:16.477427 17716 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.475318 18175 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:16.461947 17978 consensus_queue.cc:1059] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:19:16.487458 15642 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:16.459726 18163 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:19:16.465996 17920 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
W20260709 13:19:16.522866 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:16.523768 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.477989 18182 consensus_queue.cc:1059] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:19:16.494139 18061 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:16.495402 18176 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:16.484965 17355 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:19:16.505726 17942 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:16.513187 18149 consensus_queue.cc:1059] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:19:16.522737 15642 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:16.515143 16203 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:16.524724 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.518463 18150 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:19:16.517093 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 52566.8us
I20260709 13:19:16.531674 18166 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:16.534461 18026 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.508103 18177 consensus_queue.cc:1059] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:19:16.541157 18177 raft_consensus.cc:3055] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:16.538123 16204 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:16.538815 18046 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.542062 18046 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.542812 18051 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.543305 18051 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.544066 18046 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:16.540889 18026 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.544811 18166 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.545245 18051 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:16.537559 18176 raft_consensus.cc:697] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:16.526211 18170 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:16.545904 18026 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:16.541563 18177 raft_consensus.cc:740] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:16.546283 18170 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:16.546648 18170 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:16.546947 18166 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:16.519904 17716 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.546979 18177 consensus_queue.cc:260] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.549117 17716 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:16.551838 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:16.555912 17827 consensus_queue.cc:237] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.561061 18139 raft_consensus.cc:3055] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:16.561614 18139 raft_consensus.cc:740] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:19:16.566567 18172 log.cc:927] Time spent T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.599s	user 0.003s	sys 0.000s
I20260709 13:19:16.566594 18181 raft_consensus.cc:3055] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:16.537740 17942 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:16.573523 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 50867.5us
I20260709 13:19:16.574600 15642 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:16.575587 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:16.574760 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:16.576751 15597 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
W20260709 13:19:16.583333 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 68284.1us
I20260709 13:19:16.583968 15597 raft_consensus.cc:1275] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:16.585413 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:16.588294 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 50260.9us
I20260709 13:19:16.599195 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:16.576671 18181 raft_consensus.cc:740] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:16.627745 18139 consensus_queue.cc:260] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.629119 18139 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:16.581423 17729 raft_consensus.cc:2804] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:16.630728 17729 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:16.631125 17729 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:16.590693 17777 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:16.634930 17777 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:16.635296 17777 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:16.635458 15598 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:16.613867 18051 consensus_queue.cc:1059] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:19:16.638847 18051 raft_consensus.cc:3055] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:16.639207 18051 raft_consensus.cc:740] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:16.639603 17942 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:16.640007 18051 consensus_queue.cc:260] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.640556 18176 consensus_queue.cc:237] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:16.641346 18051 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:16.582350 18047 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:16.642380 18170 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.595263 18026 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:16.644654 18170 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:16.645629 18181 consensus_queue.cc:260] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.646250 18152 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:16.642942 18047 raft_consensus.cc:697] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:16.647070 18152 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:16.647624 18047 consensus_queue.cc:237] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.649045 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 542) took 1729 ms. Trace:
I20260709 13:19:16.625519 18046 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:16.650486 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 550) took 1616 ms. Trace:
I20260709 13:19:16.650874 15523 rpcz_store.cc:276] 0709 13:19:15.034471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.034624 (+   153us) service_pool.cc:224] Handling call
0709 13:19:15.035047 (+   423us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:15.039013 (+  3966us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:15.039049 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:15.039639 (+   590us) log.cc:844] Serialized 52 byte log entry
0709 13:19:15.249370 (+209731us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:15.267023 (+ 17653us) log.cc:844] Serialized 14 byte log entry
0709 13:19:16.650161 (+1383138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:16.650235 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:16.650275 (+    40us) raft_consensus.cc:1685] finished
0709 13:19:16.650288 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:16.650430 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1386449,"prepare.queue_time_us":13222,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":43696,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":7690,"thread_start_us":183256,"threads_started":1,"wal-append.queue_time_us":226597}
W20260709 13:19:16.656178 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.627444 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:16.656975 17990 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:16.657558 17990 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.658097 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:19:16.658342 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 868121us
I20260709 13:19:16.659181 15493 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:16.660349 15493 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:16.661002 18177 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:16.661304 18164 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:19:16.661907 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 542943us
I20260709 13:19:16.662139 18080 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:16.662843 18080 consensus_queue.cc:1059] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:16.663619 18080 raft_consensus.cc:3055] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:16.663908 18080 raft_consensus.cc:740] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:16.664613 18080 consensus_queue.cc:260] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:16.665297 15491 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:16.666071 17929 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:16.666507 17928 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:16.666625 17929 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:16.667007 17928 raft_consensus.cc:697] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:16.667922 17928 consensus_queue.cc:237] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.668772 17929 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:16.646353 18147 raft_consensus.cc:3055] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:16.670684 18147 raft_consensus.cc:740] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:16.658438 18160 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:16.672093 15595 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:16.626969 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:16.672436 15493 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:16.672832 15491 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:19:16.673496 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 84267.2us
I20260709 13:19:16.661742 18164 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:16.677783 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:16.685021 15493 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:16.673440 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 89473us
W20260709 13:19:16.696184 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.685832 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:16.696694 15491 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:16.649423 15520 rpcz_store.cc:276] 0709 13:19:14.919414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.926421 (+  7007us) service_pool.cc:224] Handling call
0709 13:19:14.926775 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:15.011742 (+ 84967us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:15.030420 (+ 18678us) log.cc:844] Serialized 14 byte log entry
0709 13:19:16.646864 (+1616444us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:16.646919 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:16.646957 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:16.648816 (+  1859us) spinlock_profiling.cc:243] Waited 84.8 ms on lock 0x7b54003cf5d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:16.648859 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:16.648986 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1421852,"prepare.queue_time_us":19474,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":1002,"raft.queue_time_us":1422755,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":73,"spinlock_wait_cycles":194928384,"thread_start_us":194170,"threads_started":1,"wal-append.queue_time_us":294641,"wal-append.run_cpu_time_us":1974,"wal-append.run_wall_time_us":1129189}
I20260709 13:19:16.665732 18080 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:16.595279 17978 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:16.695439 15676 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:19:16.659195 18181 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:16.666129 18104 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:16.704233 15493 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:16.707813 15493 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:16.709420 15493 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:16.718887 15493 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:16.719552 15595 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:16.721191 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:16.695302 18152 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:16.699079 18164 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:16.727591 18152 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:16.728065 18152 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:16.684356 18064 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:16.703397 18079 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:16.730592 15599 raft_consensus.cc:1240] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:16.698201 18151 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:16.683506 17355 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:19:16.733891 18079 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:16.754832 18079 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:16.757546 15493 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:16.759989 15493 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:16.747157 18152 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:16.751996 17990 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:16.739454 17920 consensus_queue.cc:1059] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:19:16.758883 18048 raft_consensus.cc:1027] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: attempt to promote peer dd1a14c3173d49dc9b55bb116e3783c9: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 13:19:16.779733 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:16.781153 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:16.756029 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:16.782188 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:16.739212 18096 consensus_queue.cc:1059] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:19:16.783051 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:16.783902 15524 raft_consensus.cc:2410] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:19:16.784649 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:16.785159 15524 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:16.754166 15641 leader_election.cc:336] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.682s (SENT)
I20260709 13:19:16.723245 18128 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:16.786077 18160 raft_consensus.cc:2804] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:16.722560 18062 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:16.786495 18160 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:16.787433 17952 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:16.711427 18122 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:16.769465 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:16.789479 15522 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:16.790380 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:16.769948 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:16.769761 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:16.702144 18170 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:19:16.710567 18131 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.786793 18062 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.792620 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:16.704790 18147 consensus_queue.cc:260] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.731282 18154 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:16.735248 18145 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:16.736243 18151 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:16.786900 18160 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:16.812088 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 552) took 1369 ms. Trace:
I20260709 13:19:16.814280 15522 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:16.814944 15602 raft_consensus.cc:1240] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:16.723888 18104 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.814221 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:16.819002 15524 raft_consensus.cc:2410] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:16.812557 15519 rpcz_store.cc:276] 0709 13:19:15.442433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.442680 (+   247us) service_pool.cc:224] Handling call
0709 13:19:15.456817 (+ 14137us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:16.654982 (+1198165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:16.655009 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:16.655457 (+   448us) log.cc:844] Serialized 26 byte log entry
0709 13:19:16.705889 (+ 50432us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:16.806852 (+100963us) log.cc:844] Serialized 14 byte log entry
0709 13:19:16.811853 (+  5001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:16.811913 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:16.811942 (+    29us) raft_consensus.cc:1685] finished
0709 13:19:16.811948 (+     6us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:16.812041 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1148640,"raft.queue_time_us":1145787,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"thread_start_us":149479,"threads_started":2,"wal-append.queue_time_us":78459}
I20260709 13:19:16.837577 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:16.806967 18151 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:16.813668 18147 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:16.846597 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 276227us
I20260709 13:19:16.820084 17958 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:16.852458 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:16.854884 15520 raft_consensus.cc:2393] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:16.837357 17929 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:16.855763 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:16.857239 15524 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:16.858546 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:16.862423 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:16.863368 15519 raft_consensus.cc:2410] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:19:16.877420 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.882128 18062 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:16.882640 18205 log.cc:927] Time spent T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:19:16.883452 15521 raft_consensus.cc:2410] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:19:16.883167 17777 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:16.885303 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:16.886233 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:16.886615 15567 leader_election.cc:336] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.131s (SENT)
I20260709 13:19:16.887715 15567 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:16.886950 15520 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
W20260709 13:19:16.895455 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:16.896098 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 103784us
I20260709 13:19:16.908969 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:16.936586 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.081355 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:17.104001 15526 raft_consensus.cc:2410] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:19:17.071714 15669 raft_consensus.cc:1240] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: []
W20260709 13:19:16.915410 15567 leader_election.cc:336] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.020s (SENT)
I20260709 13:19:16.985503 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:17.030462 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:17.035357 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.105166 15567 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:16.904919 16205 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:17.078415 16203 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:17.124642 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:17.136720 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:17.139525 15522 raft_consensus.cc:2410] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:17.140776 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:16.889194 18176 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:17.071828 18145 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:17.141480 15520 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:17.142077 15642 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:17.144017 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:17.146966 16238 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
W20260709 13:19:17.147058 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 242223us
I20260709 13:19:17.147513 15642 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:17.142269 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 63947.4us
I20260709 13:19:17.148039 16205 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:17.118192 15491 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:17.142171 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:17.151988 16205 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
W20260709 13:19:16.910610 18211 log.cc:927] Time spent T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:19:16.898389 18026 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.157474 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.027519 18177 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:17.160668 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 245295us
I20260709 13:19:17.161550 15519 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:16.930891 18220 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:17.162237 15567 leader_election.cc:336] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.807s (SENT)
I20260709 13:19:16.970624 18131 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:17.157262 18181 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:17.143013 18051 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:17.123581 18204 raft_consensus.cc:3055] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:17.163259 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.120436 18074 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:19:17.181608 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:17.182536 18225 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:17.183032 18225 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.183348 18225 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:17.150302 18024 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.158798 18056 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:17.170200 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.157505 17998 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:17.140941 18170 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:19:17.110982 18152 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:17.167815 16238 raft_consensus.cc:2955] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:17.076962 18151 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.166587 18079 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:17.179642 16205 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:17.120668 15598 raft_consensus.cc:1275] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:19:17.181274 15519 raft_consensus.cc:2393] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 5: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:19:17.270558 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 91014.6us
I20260709 13:19:17.271342 16205 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:17.164069 18177 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.188808 15524 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:17.159761 18062 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:17.157568 17744 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:17.071537 18160 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.171581 15567 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:16.971329 17729 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:17.215451 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.173060 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.276463 15520 raft_consensus.cc:2393] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 5: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:17.165261 18025 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:17.160455 18224 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:17.165692 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:17.257582 16235 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:17.186079 18204 raft_consensus.cc:740] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:17.259403 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:17.186308 17777 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:17.172049 16234 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:17.194526 18056 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:17.174786 18026 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:17.177800 18220 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.208918 18024 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:17.278101 15525 raft_consensus.cc:2368] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 3. Current term is 4.
W20260709 13:19:17.285488 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 123281us
I20260709 13:19:17.270066 18000 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:17.288756 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.290565 15522 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:19:17.290208 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 139711us
I20260709 13:19:17.296118 16205 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:17.274173 17950 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:17.292929 15568 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:17.306664 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.274431 18182 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:17.276014 18062 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:17.289673 18164 consensus_meta.cc:287] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.562s	user 0.010s	sys 0.000s
I20260709 13:19:17.290617 17942 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:17.287225 18219 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:17.314781 15642 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:17.315757 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:17.303145 18056 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:17.305759 16231 raft_consensus.cc:2410] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:17.345896 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.324604 15677 raft_consensus.cc:1240] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:17.348369 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:17.323333 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.319383 18168 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:17.320287 18181 raft_consensus.cc:2764] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:19:17.359493 15519 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:17.320874 18062 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:17.362320 16205 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:17.375564 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.376767 15520 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
W20260709 13:19:17.371632 18221 log.cc:927] Time spent T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:19:17.378937 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.382757 15522 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:17.359273 18170 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:17.387712 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.388368 15528 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
W20260709 13:19:17.389029 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.389240 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.390069 15528 raft_consensus.cc:2393] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 5: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:17.390259 18079 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:17.390833 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:17.360498 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:17.391089 18220 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:17.391592 18026 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:17.382757 16238 raft_consensus.cc:1275] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEARNER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:19:17.391849 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:17.384635 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.323081 18204 consensus_queue.cc:260] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.398324 18204 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:17.399634 17958 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:17.400164 17958 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:17.401034 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:17.401491 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.373859 18227 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:17.403381 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:17.404067 16236 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 3.
I20260709 13:19:17.386103 15598 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:17.386663 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.387584 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.391755 15672 raft_consensus.cc:2393] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 6: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:17.396651 15519 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:17.347517 18164 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:17.373682 18168 raft_consensus.cc:697] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:17.371212 18182 consensus_queue.cc:1048] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:19:17.432356 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.422953 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.356839 18219 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:17.445449 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 134151us
W20260709 13:19:17.449824 15567 leader_election.cc:336] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.147s (SENT)
I20260709 13:19:17.450347 15567 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:17.412446 18079 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:17.452090 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:17.474650 15567 leader_election.cc:336] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.166s (SENT)
I20260709 13:19:17.475242 15567 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:17.480876 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.481675 15522 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:17.418462 18170 raft_consensus.cc:697] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:17.456555 15520 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:17.420675 18220 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:17.486382 15528 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:17.504925 18220 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.506554 18056 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.420928 18026 raft_consensus.cc:2955] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 3 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:19:17.495610 18147 consensus_meta.cc:287] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.645s	user 0.009s	sys 0.000s
I20260709 13:19:17.507377 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:19:17.490005 18231 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:19:17.507769 18160 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:17.446502 18152 consensus_queue.cc:1059] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:19:17.509004 18024 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:17.509157 18152 raft_consensus.cc:3055] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:17.509593 18152 raft_consensus.cc:740] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:17.486194 18079 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.510473 18152 consensus_queue.cc:260] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.511500 18079 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:17.511701 18152 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:17.497728 15493 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:57310 dropped due to backpressure: service queue is full with 50 items
I20260709 13:19:17.485961 18025 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:17.500334 18170 consensus_queue.cc:237] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.521169 18056 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:17.525643 17729 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:17.527679 18164 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:17.487133 18177 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:17.502529 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.530726 18225 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.502288 18168 consensus_queue.cc:237] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.533172 18225 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:17.534060 16231 raft_consensus.cc:1240] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:19:17.504359 18219 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:17.531396 15520 raft_consensus.cc:2393] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 5: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:17.507058 18220 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:17.539032 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 164404us
I20260709 13:19:17.541299 15528 raft_consensus.cc:2410] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:19:17.498564 15567 leader_election.cc:336] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.218s (SENT)
I20260709 13:19:17.542209 15567 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:17.485762 18163 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:17.487293 18151 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:17.544528 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 46014.9us
W20260709 13:19:17.545203 15567 leader_election.cc:336] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.231s (SENT)
I20260709 13:19:17.545498 15601 raft_consensus.cc:1240] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:17.545560 15567 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:17.518257 15493 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:57310 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1637)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1662)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1633)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1621)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56840 (request call id 565)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1625)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1630)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1622)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1626)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1623)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1624)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1627)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1655)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1620)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1618)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1660)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1677)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1635)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1612)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1628)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1629)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1644)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1651)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1661)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1666)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1667)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1668)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1670)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1672)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1684)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 555)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 556)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 557)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 558)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 559)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1617)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 560)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 561)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 562)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 563)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 564)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 566)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 567)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 568)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 569)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 570)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 571)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1632)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1634)
W20260709 13:19:17.547683 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 164374us
I20260709 13:19:17.548681 15642 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:17.555190 15642 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:17.560300 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.560055 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:19:17.566794 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.571313 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:17.572140 15672 raft_consensus.cc:2393] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:17.558552 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:17.574370 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.575208 15528 raft_consensus.cc:2393] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 7: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:17.550262 18177 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:17.550402 18163 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:17.578505 15673 raft_consensus.cc:1275] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:19:17.579334 15669 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:19:17.578697 15677 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:17.587972 18244 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:17.588546 18057 raft_consensus.cc:3055] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:17.588990 18057 raft_consensus.cc:740] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:17.589797 18057 consensus_queue.cc:260] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.590790 18057 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:17.592237 17744 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:17.595158 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:17.597988 18243 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:17.568595 18241 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:17.599682 18241 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.600035 18241 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:17.601876 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:17.602550 15528 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:17.601099 15642 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:17.634938 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:17.635653 16236 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:17.579566 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:17.646978 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:17.649343 15601 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:17.656172 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:17.657008 16236 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:17.657148 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:17.658859 15673 raft_consensus.cc:2393] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 4: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:17.656441 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:17.661067 15642 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:17.667111 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:17.676136 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 257171us
W20260709 13:19:17.677349 15641 leader_election.cc:336] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.106s (SENT)
I20260709 13:19:17.585700 18237 consensus_queue.cc:1059] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.088s
I20260709 13:19:17.640105 18243 consensus_queue.cc:1048] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
W20260709 13:19:17.680001 15641 leader_election.cc:336] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.653s (SENT)
I20260709 13:19:17.683501 15641 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:17.684513 15641 leader_election.cc:336] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.660s (SENT)
I20260709 13:19:17.684945 15641 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:17.662902 18239 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:19:17.686367 15641 leader_election.cc:336] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.795s (SENT)
W20260709 13:19:17.700013 15641 leader_election.cc:336] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.938s (SENT)
I20260709 13:19:17.700420 15641 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:17.690477 16236 raft_consensus.cc:2955] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 3 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:17.694574 18063 raft_consensus.cc:3055] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:17.695432 17729 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:17.695933 17998 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:17.669374 17744 raft_consensus.cc:2804] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:17.711181 16205 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:19:17.712435 15641 leader_election.cc:336] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.179s (SENT)
I20260709 13:19:17.726001 15641 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:17.721743 15493 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:56840 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1642)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1638)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1641)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1625)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1630)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1622)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1643)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1626)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1623)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1624)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1627)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1655)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1620)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1618)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1660)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1677)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1635)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1612)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1628)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1629)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1640)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1644)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1651)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1661)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1666)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1667)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1668)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1670)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1672)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1684)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 555)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 556)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 557)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 558)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 559)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1617)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 560)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 562)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 563)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 564)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 566)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 567)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 568)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 569)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 570)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 571)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1632)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1634)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 572)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1639)
W20260709 13:19:17.727051 15641 leader_election.cc:336] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.673s (SENT)
W20260709 13:19:17.685499 18245 log.cc:927] Time spent T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:19:17.679569 15601 raft_consensus.cc:2955] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 3 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:17.675298 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:17.735539 15493 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:57310 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1642)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1638)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1641)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1625)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1630)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1622)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1643)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1626)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1623)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1624)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1627)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1655)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1620)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1618)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1660)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1677)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1635)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1612)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1628)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1629)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1640)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1644)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1651)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1661)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1666)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1667)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1668)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1670)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1672)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1684)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 555)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 556)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 557)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 558)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 559)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1617)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 560)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 562)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 563)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 564)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 566)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 567)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 568)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 569)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 570)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 571)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1632)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1634)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 572)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1639)
I20260709 13:19:17.727473 15641 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:17.673051 18131 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:19:17.702597 18239 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:17.733487 15567 leader_election.cc:336] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.846s (SENT)
I20260709 13:19:17.669580 17942 consensus_queue.cc:1048] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:19:17.739095 17744 raft_consensus.cc:697] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
W20260709 13:19:17.765112 18246 log.cc:927] Time spent T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:19:17.724107 17944 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.729323 18048 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:17.714967 18000 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:17.720932 18237 raft_consensus.cc:3055] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:17.678392 17958 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.675359 18031 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:19:17.740067 18008 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:19:17.778836 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.780769 15567 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:17.815945 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 82496.2us
I20260709 13:19:17.782078 18151 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.816926 18151 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:17.779718 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 57551us
I20260709 13:19:17.784204 15524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1640) took 1013 ms. Trace:
I20260709 13:19:17.781803 15493 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:57310 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1642)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1638)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1641)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1625)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1630)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1622)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1643)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1626)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1623)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1624)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1627)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1655)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1620)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1618)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1660)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1677)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1635)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1612)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1628)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1629)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1640)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1644)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1651)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1661)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1666)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1667)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1668)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1670)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1672)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1684)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 555)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 556)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 557)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 558)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 559)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1617)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 560)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 562)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 563)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 564)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 566)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 567)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 568)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 569)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 570)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 571)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1632)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1634)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 572)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1639)
I20260709 13:19:17.711753 18063 raft_consensus.cc:740] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:17.797546 18026 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:17.792613 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 553) took 1128 ms. Trace:
I20260709 13:19:17.826426 15523 rpcz_store.cc:276] 0709 13:19:16.664521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.681007 (+ 16486us) service_pool.cc:224] Handling call
0709 13:19:16.681341 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:16.848967 (+167626us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:16.848997 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:16.849039 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:16.849069 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:17.792449 (+943380us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54003ccdd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:17.792479 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:17.792569 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":385015168}
I20260709 13:19:17.794458 15599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 452) took 1221 ms. Trace:
I20260709 13:19:17.827968 15599 rpcz_store.cc:276] 0709 13:19:16.572689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.575163 (+  2474us) service_pool.cc:224] Handling call
0709 13:19:16.575416 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:16.731064 (+155648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:17.794239 (+1063175us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b540016ab54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:17.794399 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":356387072}
I20260709 13:19:17.795447 15602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 448) took 1223 ms. Trace:
W20260709 13:19:17.795958 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 68936.2us
W20260709 13:19:17.829689 15641 leader_election.cc:336] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.875s (SENT)
W20260709 13:19:17.831017 15641 leader_election.cc:336] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.984s (SENT)
I20260709 13:19:17.831514 15641 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:17.760531 17729 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:17.798900 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.778555 17998 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:17.804728 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.819038 18151 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:17.819465 15524 rpcz_store.cc:276] 0709 13:19:16.770447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.865479 (+ 95032us) service_pool.cc:224] Handling call
0709 13:19:17.784034 (+918555us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b54003cebd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:17.784141 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":659605376}
I20260709 13:19:17.819588 18177 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:17.821694 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:17.822532 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:17.786805 18104 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:17.829375 15602 rpcz_store.cc:276] 0709 13:19:16.571602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.573992 (+  2390us) service_pool.cc:224] Handling call
0709 13:19:16.574280 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:16.815377 (+241097us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:17.795276 (+979899us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b540025ba54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:17.795402 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":553002752}
I20260709 13:19:17.833470 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:17.849871 15670 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:17.824373 15493 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:57300 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1642)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1638)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1641)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1646)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1625)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1630)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1622)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1643)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1626)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1623)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1624)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1627)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1655)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1620)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1618)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1660)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1635)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1612)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1628)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1629)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1640)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1644)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1651)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1661)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1666)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1667)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1668)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1670)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1672)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1684)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 555)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 556)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 557)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 558)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 559)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1617)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 560)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 562)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 563)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 564)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 566)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 567)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 568)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 569)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 570)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 571)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1632)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1634)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 572)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1639)
I20260709 13:19:17.855289 18063 consensus_queue.cc:260] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.856418 18063 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:17.838451 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.860540 16233 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:17.843158 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.782768 18239 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:17.864457 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.865115 15523 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:17.865947 15670 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:17.810495 18237 raft_consensus.cc:740] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:17.810220 18062 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.876155 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.848857 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:17.855491 15493 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:57310 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1641)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1646)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1625)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1630)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1622)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1643)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1626)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1623)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1624)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1627)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1648)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1655)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1620)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1618)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1660)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1635)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1612)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1628)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1629)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1640)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1644)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1651)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1661)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1666)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1667)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1668)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1670)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1672)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1684)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 555)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 556)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 557)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 558)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 559)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1617)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 560)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 562)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 563)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 564)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 566)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 567)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 568)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 569)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 570)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 571)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1632)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1634)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 572)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1639)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1647)
I20260709 13:19:17.887274 15523 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:17.803804 17944 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.878818 18131 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.880146 18062 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:17.880856 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.878093 16203 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:17.836547 18251 log.cc:927] Time spent T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:19:17.889307 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.907603 15522 raft_consensus.cc:2393] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:17.888095 15528 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:17.890271 15519 raft_consensus.cc:2393] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:17.848443 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:17.846488 18048 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:17.909092 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:17.914551 15641 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:17.918987 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:17.919543 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.919790 15524 raft_consensus.cc:2393] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:17.922634 18239 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.925555 18239 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:17.927356 18241 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:17.930335 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 55969.4us
I20260709 13:19:17.931403 15642 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:17.931636 15641 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:17.935081 18237 consensus_queue.cc:260] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.936254 18237 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:17.938820 15567 leader_election.cc:336] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:57310 dropped due to backpressure: service queue is full with 50 items
I20260709 13:19:17.939317 15567 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:17.940644 17942 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:17.909735 18008 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:17.845290 17729 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.847733 17998 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.951176 15567 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:17.953401 15567 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:17.955295 16235 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:17.877744 17827 consensus_queue.cc:1048] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 13:19:17.938356 18000 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:17.966166 15641 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:17.950119 18167 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:17.910049 18131 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.848789 17744 consensus_queue.cc:237] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.959167 18141 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:17.960196 18008 raft_consensus.cc:697] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:17.966799 18139 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:17.975147 18167 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:18.002226 17944 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:18.030802 18008 consensus_queue.cc:237] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:17.982635 18168 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.007278 18241 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:17.990141 18255 log.cc:927] Time spent T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 13:19:17.950310 18260 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:19:17.986619 18141 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:17.993162 18139 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:18.033663 18167 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:17.949693 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1636) took 1178 ms. Trace:
W20260709 13:19:18.180059 15493 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.180533 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:18.180706 15521 rpcz_store.cc:276] 0709 13:19:16.769334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.769508 (+   174us) service_pool.cc:224] Handling call
0709 13:19:17.941825 (+1172317us) spinlock_profiling.cc:243] Waited 79.9 ms on lock 0x7b540041f854. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:17.947386 (+  5561us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183842176}
I20260709 13:19:18.184056 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:18.189356 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:18.192318 15567 leader_election.cc:336] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 3.142s (SENT)
W20260709 13:19:18.196239 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:18.196797 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.029716 17827 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:18.200713 17729 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:18.201125 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:18.202780 15521 raft_consensus.cc:2393] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 3: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:18.204208 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:18.205402 15521 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:19:18.180969 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 214801us
W20260709 13:19:18.181201 15493 connection.cc:256] RPC call timeout handler was delayed by -1.7771140310001101s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15080000000000002s for a response
W20260709 13:19:18.206560 15493 connection.cc:256] RPC call timeout handler was delayed by -1.7087140309999995s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1584s for a response
W20260709 13:19:18.206945 15493 connection.cc:256] RPC call timeout handler was delayed by -1.4855140310000934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18320000000000003s for a response
W20260709 13:19:18.207288 15493 connection.cc:256] RPC call timeout handler was delayed by -1.4342140310000104s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1889s for a response
W20260709 13:19:18.207587 15493 connection.cc:256] RPC call timeout handler was delayed by -1.4171140309999828s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1908s for a response
W20260709 13:19:18.207917 15493 connection.cc:256] RPC call timeout handler was delayed by -1.147114031000001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22080000000000002s for a response
W20260709 13:19:18.208221 15493 connection.cc:256] RPC call timeout handler was delayed by -1.1264140309999675s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2231s for a response
I20260709 13:19:18.209161 15493 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:18.028999 17998 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:18.008106 17958 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:18.214581 17958 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.215308 17958 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:18.221061 18024 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.221961 18024 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:18.223681 18104 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:18.206717 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:18.225229 15523 raft_consensus.cc:2393] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 6: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:18.224701 18104 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:18.226051 18062 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.226608 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:18.226884 18062 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:18.227624 15523 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:18.226536 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:18.228607 18104 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:18.097072 18141 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:18.230747 18131 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:18.232283 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:18.233285 15600 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:18.131881 18139 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:18.241544 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 49451.4us
I20260709 13:19:18.055480 18168 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:18.224570 18024 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:18.247372 18168 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:18.250666 15567 leader_election.cc:336] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 3.187s (SENT)
I20260709 13:19:18.251365 15567 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:18.228036 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1655) took 1317 ms. Trace:
I20260709 13:19:18.313257 15523 rpcz_store.cc:276] 0709 13:19:16.911022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.226134 (+1315112us) service_pool.cc:224] Handling call
0709 13:19:18.227978 (+  1844us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:18.341171 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:18.349318 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 98779.4us
I20260709 13:19:18.336903 18024 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:18.340250 15521 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
W20260709 13:19:18.362502 18063 consensus_meta.cc:287] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.505s	user 0.008s	sys 0.004s
I20260709 13:19:18.365227 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:18.348398 17958 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:18.354045 18104 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:18.379837 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:18.380767 15671 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
W20260709 13:19:18.388345 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.388666 18145 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:18.389598 18145 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:18.390561 15677 raft_consensus.cc:1275] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:18.376649 15527 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:18.369614 18062 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:18.390246 18145 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:18.389370 18152 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:18.393735 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:18.398782 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:18.401546 15600 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:18.403962 15566 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:18.411644 18149 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:18.407030 18152 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:18.418834 17827 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:18.420837 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:18.421660 15521 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:18.421763 18152 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:18.422998 15527 raft_consensus.cc:2410] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:19:18.393410 18104 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:18.427589 15523 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:18.427723 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:18.429304 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:18.412196 18149 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:18.429821 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1660) took 1459 ms. Trace:
I20260709 13:19:18.438683 15527 rpcz_store.cc:276] 0709 13:19:16.970863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.392941 (+1422078us) service_pool.cc:224] Handling call
0709 13:19:18.429761 (+ 36820us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:18.427973 18104 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:18.422346 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1618) took 1042 ms. Trace:
I20260709 13:19:18.440081 18149 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:18.456182 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:18.461862 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:18.462196 18145 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:18.465430 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:18.466456 15527 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:18.451360 18151 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:18.449714 15521 rpcz_store.cc:276] 0709 13:19:17.380361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.394102 (+1013741us) service_pool.cc:224] Handling call
0709 13:19:18.422245 (+ 28143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:18.473824 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:18.476611 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:18.477278 15527 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:18.481235 15669 raft_consensus.cc:2393] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 4: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:18.481985 15523 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:19:18.483994 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1612) took 1305 ms. Trace:
I20260709 13:19:18.484347 15523 rpcz_store.cc:276] 0709 13:19:17.178491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.471451 (+1292960us) service_pool.cc:224] Handling call
0709 13:19:18.483929 (+ 12478us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:18.485993 18000 consensus_meta.cc:287] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.504s	user 0.005s	sys 0.006s
I20260709 13:19:18.486536 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:18.491164 15527 raft_consensus.cc:2410] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:19:18.471863 18151 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:18.487108 15642 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:18.481611 18270 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:18.497635 18145 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.513967 17827 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:18.514758 17827 raft_consensus.cc:697] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:18.500450 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:18.519801 18274 log.cc:927] Time spent T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:19:18.520164 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:18.497188 18167 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:18.523345 18145 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:18.524987 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:18.497449 18270 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:18.528527 18167 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:18.528913 18270 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:18.530824 17827 consensus_queue.cc:237] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:18.522259 18139 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:18.504290 18151 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:18.543205 18139 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:18.546244 18275 log.cc:927] Time spent T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:19:18.548100 15600 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:18.551443 15528 raft_consensus.cc:1240] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:18.552295 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1651) took 1745 ms. Trace:
I20260709 13:19:18.552639 15528 rpcz_store.cc:276] 0709 13:19:16.806563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.550635 (+1744072us) service_pool.cc:224] Handling call
0709 13:19:18.551103 (+   468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:18.551975 (+   872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.552193 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:18.565764 18249 log.cc:927] Time spent T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.854s	user 0.008s	sys 0.011s
W20260709 13:19:18.566980 18268 log.cc:927] Time spent T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.420s	user 0.003s	sys 0.001s
W20260709 13:19:18.579237 18278 log.cc:927] Time spent T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:19:18.589959 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:18.590605 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1661) took 1614 ms. Trace:
I20260709 13:19:18.590960 15520 raft_consensus.cc:2410] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:19:18.591282 15521 raft_consensus.cc:2410] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:18.591466 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1662) took 1615 ms. Trace:
I20260709 13:19:18.591777 15520 rpcz_store.cc:276] 0709 13:19:16.976473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.589484 (+1613011us) service_pool.cc:224] Handling call
0709 13:19:18.591412 (+  1928us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:18.593376 15520 raft_consensus.cc:1240] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:19:18.536958 15523 raft_consensus.cc:2410] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:18.595098 15523 raft_consensus.cc:1240] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:19:18.595671 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1668) took 1617 ms. Trace:
I20260709 13:19:18.597671 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:18.598239 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1628) took 1103 ms. Trace:
I20260709 13:19:18.572628 15423 catalog_manager.cc:5697] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 1 to 2, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:18.539038 18145 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:18.542609 15527 raft_consensus.cc:1275] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:19:18.542719 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 576544us
I20260709 13:19:18.543200 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:18.499218 18000 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:18.598703 15521 rpcz_store.cc:276] 0709 13:19:17.494714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.474431 (+979717us) service_pool.cc:224] Handling call
0709 13:19:18.598168 (+123737us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:18.567071 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:18.596012 15523 rpcz_store.cc:276] 0709 13:19:16.978068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.594415 (+1616347us) service_pool.cc:224] Handling call
0709 13:19:18.594826 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:18.595445 (+   619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.595620 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:18.617507 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:18.603147 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1644) took 1831 ms. Trace:
W20260709 13:19:18.620569 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 93809us
I20260709 13:19:18.634387 15527 rpcz_store.cc:276] 0709 13:19:16.771346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.499293 (+1727947us) service_pool.cc:224] Handling call
0709 13:19:18.542262 (+ 42969us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:18.602864 (+ 60602us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.603073 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:18.610320 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1683) took 1628 ms. Trace:
I20260709 13:19:18.618059 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:18.616664 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:18.636080 15519 rpcz_store.cc:276] 0709 13:19:17.004246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.422686 (+418440us) service_pool.cc:224] Handling call
0709 13:19:18.609798 (+1187112us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b5400070dd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:18.610091 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1022861056}
I20260709 13:19:18.636873 16239 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:18.638609 16232 raft_consensus.cc:1240] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 5. Current term is 6. Ops: [3.6-5.7]
W20260709 13:19:18.602782 15641 leader_election.cc:336] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:57300 dropped due to backpressure: service queue is full with 50 items
I20260709 13:19:18.639665 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1667) took 1646 ms. Trace:
I20260709 13:19:18.640117 15520 rpcz_store.cc:276] 0709 13:19:16.977769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.592416 (+1614647us) service_pool.cc:224] Handling call
0709 13:19:18.592987 (+   571us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:18.593719 (+   732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.623768 (+ 30049us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:18.591001 15528 rpcz_store.cc:276] 0709 13:19:16.976114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.553502 (+1577388us) service_pool.cc:224] Handling call
0709 13:19:18.589997 (+ 36495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:18.590246 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:18.590270 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:18.590326 (+    56us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:18.590358 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.590411 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.590526 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:18.618536 16233 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:19:18.643502 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:18.645023 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:18.608709 15525 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1633) took 1840 ms (client timeout 1673 ms). Trace:
I20260709 13:19:18.612316 15521 raft_consensus.cc:1275] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:18.607205 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:18.623070 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:18.667675 15524 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:18.671382 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:18.673336 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:18.601950 15423 catalog_manager.cc:5697] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 3 to 4, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 4 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:18.675631 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:18.676784 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.678170 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:18.611608 18201 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.674656 15670 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
W20260709 13:19:18.654392 15525 rpcz_store.cc:269] 0709 13:19:16.768308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.768419 (+   111us) service_pool.cc:224] Handling call
0709 13:19:18.608228 (+1839809us) spinlock_profiling.cc:243] Waited 495 ms on lock 0x7b54003da4d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:18.608488 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1138728832}
I20260709 13:19:18.683082 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:18.642846 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:18.674823 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:18.643895 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:18.607095 18131 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:19:18.693326 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:18.690230 16240 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:18.697125 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:18.698745 16238 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:18.603785 18279 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
W20260709 13:19:18.691609 15567 leader_election.cc:336] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.499s (SENT)
I20260709 13:19:18.709052 15527 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:19:18.715628 15567 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:18.715888 15600 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:18.709455 15527 pending_rounds.cc:85] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 6
W20260709 13:19:18.717620 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.714442 15528 pending_rounds.cc:85] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 5
I20260709 13:19:18.691834 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1670) took 1713 ms. Trace:
I20260709 13:19:18.708508 16231 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:18.719182 15528 pending_rounds.cc:107] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 3.6
W20260709 13:19:18.727380 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:18.725121 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 516216us
I20260709 13:19:18.716482 15527 pending_rounds.cc:107] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:19:18.730367 18239 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:18.730580 18131 raft_consensus.cc:3055] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:18.731035 18131 raft_consensus.cc:740] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:18.731153 18239 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:18.732203 18131 consensus_queue.cc:260] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:18.733425 17729 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.734030 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:18.734195 17729 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:18.738108 18168 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:18.738502 18074 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:18.738175 18046 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:19:18.739804 18260 consensus_queue.cc:1059] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:18.740790 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:18.742429 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:18.743265 15675 raft_consensus.cc:2393] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 8: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:19:18.744479 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 102759us
I20260709 13:19:18.730378 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:18.747133 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:18.747903 16204 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:18.748052 16233 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:18.748273 17729 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:18.747231 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:18.751338 16236 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:18.723032 15521 rpcz_store.cc:276] 0709 13:19:16.978591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.611432 (+1632841us) service_pool.cc:224] Handling call
0709 13:19:18.611849 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:18.676628 (+ 64779us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.688781 (+ 12153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:18.701380 18149 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:18.756953 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:18.757972 18149 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:18.758200 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:18.758802 16236 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:18.759574 18149 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:18.738312 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:18.763302 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:18.728324 15527 raft_consensus.cc:2889] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:18.749145 18239 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:18.726732 15528 raft_consensus.cc:2889] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:18.772495 15523 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:19:18.748237 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:18.787529 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1672) took 1800 ms. Trace:
I20260709 13:19:18.787990 15523 rpcz_store.cc:276] 0709 13:19:16.979120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.662803 (+1683683us) service_pool.cc:224] Handling call
0709 13:19:18.718074 (+ 55271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:18.778874 (+ 60800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.779068 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:18.790949 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 51707.2us
I20260709 13:19:18.749202 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:18.772349 18131 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:18.804263 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 56519.5us
I20260709 13:19:18.809890 15526 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:19:18.809067 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 145901us
I20260709 13:19:18.817836 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1637) took 1316 ms. Trace:
I20260709 13:19:18.820024 15523 raft_consensus.cc:1240] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:18.821342 18051 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:18.822077 18051 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:18.824277 18051 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:18.825865 16237 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:19:18.829207 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1684) took 1722 ms. Trace:
I20260709 13:19:18.829627 15527 rpcz_store.cc:276] 0709 13:19:17.106469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.664407 (+1557938us) service_pool.cc:224] Handling call
0709 13:19:18.696655 (+ 32248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:18.828953 (+132298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.829142 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:18.831346 15527 raft_consensus.cc:1240] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:19:18.832005 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 562) took 1412 ms. Trace:
I20260709 13:19:18.832345 15527 rpcz_store.cc:276] 0709 13:19:17.419325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.830629 (+1411304us) service_pool.cc:224] Handling call
0709 13:19:18.831069 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:18.831756 (+   687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.831952 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:18.839505 15493 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:18.723724 18201 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:18.760288 18149 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:18.818280 15526 rpcz_store.cc:276] 0709 13:19:17.497077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.870357 (+373280us) service_pool.cc:224] Handling call
0709 13:19:18.747963 (+877606us) spinlock_profiling.cc:243] Waited 6.22 ms on lock 0x7b5800003698. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99d10a0 00007f5ce99d0984 00007f5ce2f4b6d4 00007f5ce2f4b662
0709 13:19:18.813308 (+ 65345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14296320}
I20260709 13:19:18.856411 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 560) took 1437 ms. Trace:
I20260709 13:19:18.857142 15523 rpcz_store.cc:276] 0709 13:19:17.418782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.788784 (+1370002us) service_pool.cc:224] Handling call
0709 13:19:18.813101 (+ 24317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:18.856098 (+ 42997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.856348 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:18.866371 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 185335us
W20260709 13:19:18.883345 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.895859 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:18.896529 15674 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:18.897014 17984 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.897624 17984 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:18.903302 18269 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:18.905393 17984 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:18.831231 18074 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:18.916026 15642 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:18.817138 18141 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:18.909983 18074 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:18.992887 15600 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:19.022665 16239 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:19:19.023119 16239 pending_rounds.cc:85] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9: Aborting all ops after (but not including) 7
I20260709 13:19:19.028048 16239 pending_rounds.cc:107] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:19:19.028306 16239 raft_consensus.cc:2889] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:19.028843 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 112927us
I20260709 13:19:19.030474 15642 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:19.030555 18179 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.132s
I20260709 13:19:19.031867 15568 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:19.032619 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 216904us
I20260709 13:19:19.034336 18057 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:19.034565 18282 log.cc:927] Time spent T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:19:19.034806 17777 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:19.035251 17777 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.035581 17777 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:19.036738 18285 log.cc:927] Time spent T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.218s	user 0.001s	sys 0.006s
I20260709 13:19:19.038015 16234 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:19.039376 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 566) took 1615 ms. Trace:
I20260709 13:19:19.039822 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1666) took 2062 ms. Trace:
I20260709 13:19:19.040150 16234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 917) took 1115 ms. Trace:
I20260709 13:19:19.040607 16234 rpcz_store.cc:276] 0709 13:19:17.925106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.927616 (+  2510us) service_pool.cc:224] Handling call
0709 13:19:18.832957 (+905341us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b580009ff98. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99e4fc0 00007f5ce99cf929 00007f5ce99cf3d2 00007f5ce2f46bd4
0709 13:19:18.833262 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.038390 (+205128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.039900 (+  1510us) spinlock_profiling.cc:243] Waited 69 ms on lock 0x7b540068f454. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:19.040059 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182489600}
I20260709 13:19:19.023542 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:19.027534 15526 raft_consensus.cc:1240] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:19.043385 15568 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:19.039764 15523 rpcz_store.cc:276] 0709 13:19:17.423477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.038444 (+1614967us) service_pool.cc:224] Handling call
0709 13:19:19.038823 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.039042 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:19.039064 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:19.039112 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:19.039143 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.039195 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.039304 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:19.050232 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 564) took 1627 ms. Trace:
I20260709 13:19:19.050657 15526 rpcz_store.cc:276] 0709 13:19:17.422954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.906537 (+1483583us) service_pool.cc:224] Handling call
0709 13:19:19.013268 (+106731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.049977 (+ 36709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.050176 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:19.052260 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:19.054678 17744 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:19.055205 17744 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.049552 18131 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:19.056059 18131 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.056449 18131 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:19.058055 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:19.058519 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.040225 15522 rpcz_store.cc:276] 0709 13:19:16.977513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.555922 (+1578409us) service_pool.cc:224] Handling call
0709 13:19:18.556398 (+   476us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:18.556979 (+   581us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:19:18.557315 (+   336us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.765201 (+207886us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:18.774716 (+  9515us) log.cc:844] Serialized 26 byte log entry
0709 13:19:18.774844 (+   128us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:18.775147 (+   303us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.775348 (+   201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.775414 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:19.039505 (+264091us) raft_consensus.cc:1685] finished
0709 13:19:19.039522 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.039634 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7680,"prepare.queue_time_us":1319,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":74,"raft.queue_time_us":17100,"raft.run_cpu_time_us":401,"raft.run_wall_time_us":13679,"spinlock_wait_cycles":21084544,"thread_start_us":205753,"threads_started":1,"wal-append.queue_time_us":298391}
I20260709 13:19:19.059512 16203 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:19.059675 18201 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:19.060094 17984 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:19.060508 17984 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.060853 17984 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:19.055576 17744 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:19.062357 18057 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.064728 18057 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:19.069078 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 557) took 1960 ms. Trace:
I20260709 13:19:19.053771 18260 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:19.060087 15676 raft_consensus.cc:1240] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:19.072549 15600 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:19:19.070114 17744 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.069911 15674 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:19.082391 17744 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:19.076151 18267 log.cc:927] Time spent T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.708s	user 0.003s	sys 0.001s
I20260709 13:19:19.071894 18260 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:19.084408 18260 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:19.085573 16236 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:19.085731 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:19.085525 18131 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.087752 18149 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:19.089203 18131 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:19.069522 15528 rpcz_store.cc:276] 0709 13:19:17.108917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.641971 (+1533054us) service_pool.cc:224] Handling call
0709 13:19:18.690017 (+ 48046us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:18.827861 (+137844us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:18.827890 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:18.828358 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 13:19:18.987579 (+159221us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:18.987620 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.987702 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:19.068844 (+ 81142us) raft_consensus.cc:1685] finished
0709 13:19:19.068862 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.069011 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":158746,"threads_started":1,"wal-append.queue_time_us":225330}
I20260709 13:19:19.124346 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:19.126845 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 115355us
I20260709 13:19:19.130000 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:19.129024 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 230477us
I20260709 13:19:19.137249 15642 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:19.127817 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1617) took 1768 ms. Trace:
I20260709 13:19:19.144342 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:19.145360 15602 raft_consensus.cc:2393] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 7: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:19.144048 15521 rpcz_store.cc:276] 0709 13:19:17.359550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.752403 (+1392853us) service_pool.cc:224] Handling call
0709 13:19:18.752973 (+   570us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:18.753438 (+   465us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:18.753472 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:18.754181 (+   709us) log.cc:844] Serialized 26 byte log entry
0709 13:19:18.906313 (+152132us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:18.914616 (+  8303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.919035 (+  4419us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:19.092042 (+173007us) raft_consensus.cc:1685] finished
0709 13:19:19.098219 (+  6177us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.125648 (+ 27429us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2223,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"thread_start_us":127403,"threads_started":1,"wal-append.queue_time_us":271250}
W20260709 13:19:19.146206 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.134300 18201 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:19.147035 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 569) took 1708 ms. Trace:
I20260709 13:19:19.147650 15597 raft_consensus.cc:1240] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:19:19.148906 18141 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:19.150380 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 91044.2us
I20260709 13:19:19.151458 16203 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:19.144055 18048 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:19.155077 18048 raft_consensus.cc:697] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:19.156072 18048 consensus_queue.cc:237] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:19.147716 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 555) took 2039 ms. Trace:
I20260709 13:19:19.146070 18289 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:19.143311 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 568) took 1719 ms. Trace:
I20260709 13:19:19.161448 15522 rpcz_store.cc:276] 0709 13:19:17.423984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.101569 (+1677585us) service_pool.cc:224] Handling call
0709 13:19:19.139442 (+ 37873us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.142960 (+  3518us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:19.142992 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:19.143036 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:19.143070 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.143128 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.143236 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:19.162513 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 567) took 1738 ms. Trace:
I20260709 13:19:19.163529 15523 rpcz_store.cc:276] 0709 13:19:17.423754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.096610 (+1672856us) service_pool.cc:224] Handling call
0709 13:19:19.140402 (+ 43792us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.157524 (+ 17122us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:19.157555 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:19.162233 (+  4678us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:19.162283 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.162348 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.162457 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:19.164805 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:19.165275 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 570) took 1721 ms. Trace:
I20260709 13:19:19.170970 18145 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.171542 18145 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:19.173528 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 385740us
I20260709 13:19:19.174494 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:19.142637 18260 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.177013 18260 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:19.177667 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.178319 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:19.179209 15600 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:19.179553 17958 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:19.180305 17958 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:19.165659 15523 rpcz_store.cc:276] 0709 13:19:17.444101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.164421 (+1720320us) service_pool.cc:224] Handling call
0709 13:19:19.164779 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.164987 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:19.165011 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:19.165049 (+    38us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:19.165077 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.165125 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.165222 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:19.178941 15525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 559) took 2061 ms. Trace:
I20260709 13:19:19.185734 16237 raft_consensus.cc:1275] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:19.157662 15519 rpcz_store.cc:276] 0709 13:19:17.108432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.637137 (+1528705us) service_pool.cc:224] Handling call
0709 13:19:18.637540 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:18.638274 (+   734us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:18.638304 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:18.638760 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 13:19:18.993079 (+354319us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:18.993180 (+   101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.993272 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:19.147482 (+154210us) raft_consensus.cc:1685] finished
0709 13:19:19.147508 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.147652 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":122,"thread_start_us":286014,"threads_started":1,"wal-append.queue_time_us":496304}
I20260709 13:19:19.158509 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:19.195242 18056 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:19:19.196333 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 45118.9us
I20260709 13:19:19.196480 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1639) took 1554 ms. Trace:
I20260709 13:19:19.196872 16231 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:19:19.197388 16238 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:19.198314 18145 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:19.204291 15528 rpcz_store.cc:276] 0709 13:19:17.642372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.195512 (+1553140us) service_pool.cc:224] Handling call
0709 13:19:19.195876 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.196105 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:19.196138 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:19.196205 (+    67us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:19.196256 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.196317 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.196421 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:19.205411 15526 raft_consensus.cc:1240] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 LEADER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:19.208312 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:19.208967 16236 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:19.210217 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 571) took 1762 ms. Trace:
I20260709 13:19:19.210661 15526 rpcz_store.cc:276] 0709 13:19:17.447911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.184206 (+1736295us) service_pool.cc:224] Handling call
0709 13:19:19.205223 (+ 21017us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.209838 (+  4615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.210104 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:19.211763 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 563) took 1792 ms. Trace:
I20260709 13:19:19.186117 15525 rpcz_store.cc:276] 0709 13:19:17.117853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.694613 (+1576760us) service_pool.cc:224] Handling call
0709 13:19:18.760860 (+ 66247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:18.761181 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:18.761531 (+   350us) log.cc:844] Serialized 14 byte log entry
0709 13:19:19.178407 (+416876us) log.cc:844] Serialized 14 byte log entry
0709 13:19:19.178578 (+   171us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:19.178624 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:19.178678 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.178744 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.178874 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":295276,"prepare.queue_time_us":3163,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":68,"raft.queue_time_us":300803,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":132,"thread_start_us":120202,"threads_started":1,"wal-append.queue_time_us":379391}
I20260709 13:19:19.231904 15602 raft_consensus.cc:1275] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:19:19.206112 16233 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:19:19.241811 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1634) took 1745 ms. Trace:
I20260709 13:19:19.242030 18241 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:19.242449 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:19.243836 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:19.244169 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:19.245352 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:19.246510 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:19.247823 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:19.248126 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:19.249436 18182 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.249817 15525 raft_consensus.cc:1240] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:19.242383 15523 rpcz_store.cc:276] 0709 13:19:17.496327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.202135 (+1705808us) service_pool.cc:224] Handling call
0709 13:19:19.241300 (+ 39165us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.241500 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:19.241524 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:19.241567 (+    43us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:19.241596 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.241651 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.241749 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:19.243860 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:19.252954 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:19:19.250042 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 47935.6us
I20260709 13:19:19.237035 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:19.253572 16236 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:19:19.256371 15599 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:19.258232 15493 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:19.202557 17777 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.259034 15519 raft_consensus.cc:1240] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:19.258705 15595 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:19:19.266954 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 572) took 1668 ms. Trace:
I20260709 13:19:19.267375 15519 rpcz_store.cc:276] 0709 13:19:17.598284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.204057 (+1605773us) service_pool.cc:224] Handling call
0709 13:19:19.257354 (+ 53297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.266727 (+  9373us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.266899 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:19.268622 15528 raft_consensus.cc:1275] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:19:19.269398 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1647) took 1415 ms. Trace:
I20260709 13:19:19.269824 15528 rpcz_store.cc:276] 0709 13:19:17.854466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.207155 (+1352689us) service_pool.cc:224] Handling call
0709 13:19:19.258243 (+ 51088us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.269145 (+ 10902us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.269345 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:19.273303 18170 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:19:19.275260 16231 raft_consensus.cc:1275] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:19.275933 15596 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:19.276888 18024 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:19.277697 17777 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:19.278501 18024 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:19:19.278947 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 61373.9us
I20260709 13:19:19.279904 15493 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:19.280275 18145 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:19:19.280519 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 288107us
I20260709 13:19:19.281354 15493 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:19.284834 18080 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:19.285158 15524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 558) took 2167 ms. Trace:
I20260709 13:19:19.285524 15524 rpcz_store.cc:276] 0709 13:19:17.117550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.668488 (+1550938us) service_pool.cc:224] Handling call
0709 13:19:18.668942 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:18.669369 (+   427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:18.669394 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:18.670004 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 13:19:18.720321 (+ 50317us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:18.720354 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.720413 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:19.284905 (+564492us) raft_consensus.cc:1685] finished
0709 13:19:19.284924 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.285062 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1591,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":130,"thread_start_us":49899,"threads_started":1,"wal-append.queue_time_us":547663}
I20260709 13:19:19.286372 15566 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:19:19.288290 18296 log.cc:927] Time spent T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:19:19.288524 18024 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:19.289116 18024 raft_consensus.cc:697] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:19.290122 18024 consensus_queue.cc:237] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.292255 16237 raft_consensus.cc:1240] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:19:19.250305 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:19.267745 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:19.161005 15521 rpcz_store.cc:276] 0709 13:19:17.438990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.146031 (+1707041us) service_pool.cc:224] Handling call
0709 13:19:19.146426 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.146621 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:19.146653 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:19.146728 (+    75us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:19.146769 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.146852 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.146979 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:19.273828 15568 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:19.218837 18289 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.288332 16234 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:19.304730 15598 raft_consensus.cc:2410] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:19:19.313793 17978 consensus_queue.cc:1059] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:19:19.314532 18170 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:19:19.319406 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:19.320886 15493 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:19.322145 18152 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:19.323053 18152 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:19.323827 18152 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:19.326542 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:19.330312 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 56460.1us
I20260709 13:19:19.339995 16236 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:19.301178 16232 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:19.341078 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.249990 18182 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.356446 15568 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:19.362468 15642 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:19.373594 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:19.378039 15669 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:19.379771 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 62577.3us
I20260709 13:19:19.393311 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:19.336339 18241 consensus_queue.cc:1048] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:19:19.380084 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:19.396363 17716 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:19:19.400732 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 86187.9us
I20260709 13:19:19.368343 18141 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:19.378768 18048 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:19.213773 15527 rpcz_store.cc:276] 0709 13:19:17.419554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.907473 (+1487919us) service_pool.cc:224] Handling call
0709 13:19:19.079051 (+171578us) spinlock_profiling.cc:243] Waited 8.46 ms on lock 0x7b5800003698. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99e4fc0 00007f5ce99cf929 00007f5ce99cf3d2 00007f5ce2f46bd4
0709 13:19:19.079259 (+   208us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.079521 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:19.079817 (+   296us) log.cc:844] Serialized 14 byte log entry
0709 13:19:19.211427 (+131610us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:19.211481 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:19.211527 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.211589 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.211709 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59028,"prepare.queue_time_us":2404,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":73632,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":6899,"spinlock_wait_cycles":19455744,"thread_start_us":50899,"threads_started":1,"wal-append.queue_time_us":117062}
I20260709 13:19:19.415132 18141 raft_consensus.cc:697] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:19.428905 15675 raft_consensus.cc:1275] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:19:19.429888 17716 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.430332 18201 raft_consensus.cc:3055] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:19.430881 18201 raft_consensus.cc:740] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:19.431905 18201 consensus_queue.cc:260] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.430991 18074 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:19.435673 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.367564 18179 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:19.411844 18048 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.450448 15675 raft_consensus.cc:1240] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
W20260709 13:19:19.443056 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 50016us
I20260709 13:19:19.442436 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:19.432806 17716 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:19.476068 15493 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:19.432932 18201 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:19.474537 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 73857.8us
I20260709 13:19:19.476538 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:19.477962 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:19.434051 18141 consensus_queue.cc:237] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.480572 18244 consensus_queue.cc:1059] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.114s
I20260709 13:19:19.439515 18167 raft_consensus.cc:3055] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:19.408463 17744 consensus_queue.cc:1048] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 13:19:19.473424 18048 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:19.425582 17729 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:19:19.493587 15596 raft_consensus.cc:1240] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:19.497150 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:19.495944 15493 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:19.429230 18289 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:19.432058 18182 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:19.505856 16238 raft_consensus.cc:2368] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 4. Current term is 5.
I20260709 13:19:19.487879 18167 raft_consensus.cc:740] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:19.489241 18145 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:19.517900 18141 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:19.529800 15641 leader_election.cc:336] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.610s (SENT)
I20260709 13:19:19.530382 15641 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:19.531054 18309 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.531638 18309 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.534024 17942 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:19.487131 18162 consensus_queue.cc:1059] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.179s
I20260709 13:19:19.511497 18080 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:19.548832 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:19.554529 15601 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:19:19.557080 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:19.533869 18309 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:19.557767 15598 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:19:19.562080 18290 raft_consensus.cc:3055] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:19.562609 18290 raft_consensus.cc:740] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:19.563560 18290 consensus_queue.cc:260] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.565183 18290 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:19.566268 18322 consensus_queue.cc:1059] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:19.567353 15598 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:19.567804 18309 raft_consensus.cc:3055] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:19.569353 18309 raft_consensus.cc:740] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:19.570384 18309 consensus_queue.cc:260] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.571656 18309 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:19.566941 18096 consensus_queue.cc:1059] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:19:19.574824 18096 raft_consensus.cc:3055] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:19.488582 18179 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.575217 18096 raft_consensus.cc:740] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:19:19.577824 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 81968.6us
I20260709 13:19:19.578624 18096 consensus_queue.cc:260] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.579173 15493 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:19.588718 15568 leader_election.cc:400] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 4. Current term is 5.
I20260709 13:19:19.589833 15568 leader_election.cc:403] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:19:19.567035 18292 log.cc:927] Time spent T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.394s	user 0.002s	sys 0.000s
W20260709 13:19:19.595623 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 58498.6us
I20260709 13:19:19.597318 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:19.598243 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:19.548576 18269 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:19.620867 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:19.621578 15674 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:19.622201 18167 consensus_queue.cc:260] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.623642 18167 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:19.619068 15671 raft_consensus.cc:1240] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 5. Current term is 6. Ops: []
W20260709 13:19:19.626573 17984 consensus_meta.cc:287] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.565s	user 0.012s	sys 0.000s
I20260709 13:19:19.627184 17984 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.629894 15603 raft_consensus.cc:1240] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 3. Current term is 5. Ops: [3.6-3.6]
I20260709 13:19:19.632274 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:19.633023 15521 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:19.598367 18289 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 4. Current term is 5.
I20260709 13:19:19.534085 18141 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.635234 18074 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:19.637212 16232 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
W20260709 13:19:19.617774 18312 log.cc:927] Time spent T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:19:19.639277 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.594357 17958 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.640794 17958 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:19.641716 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1632) took 2146 ms. Trace:
I20260709 13:19:19.642633 15519 raft_consensus.cc:1275] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:19:19.643168 17958 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:19.644021 15600 raft_consensus.cc:1240] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:19:19.612605 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:19.597738 18152 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:19:19.615468 15567 leader_election.cc:336] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.730s (SENT)
I20260709 13:19:19.619021 15493 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:19.651782 15567 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:19.648934 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:19.653545 16234 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:19.629614 17984 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:19.650538 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 57500.8us
I20260709 13:19:19.644040 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:19.658219 15528 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:19.658792 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 122776us
W20260709 13:19:19.658141 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 51894.8us
I20260709 13:19:19.642158 15522 rpcz_store.cc:276] 0709 13:19:17.495762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.190553 (+1694791us) service_pool.cc:224] Handling call
0709 13:19:19.233539 (+ 42986us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:19.339634 (+106095us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:19.339668 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:19.365781 (+ 26113us) log.cc:844] Serialized 52 byte log entry
0709 13:19:19.507940 (+142159us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:19.513543 (+  5603us) log.cc:844] Serialized 14 byte log entry
0709 13:19:19.527003 (+ 13460us) log.cc:844] Serialized 14 byte log entry
0709 13:19:19.529119 (+  2116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.529206 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:19.623575 (+ 94369us) spinlock_profiling.cc:243] Waited 3.71 ms on lock 0x7b180047e600. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce191e3f3 00007f5ce19207c1 00007f5ce1920779 00007f5ce1920709 00007f5ce19206d1 00007f5ce191f9cd 000000000038f354 000000000038f199 00007f5ce190183a
0709 13:19:19.641478 (+ 17903us) raft_consensus.cc:1685] finished
0709 13:19:19.641501 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.641652 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33498,"prepare.queue_time_us":23175,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":281,"raft.queue_time_us":34558,"raft.run_cpu_time_us":366,"raft.run_wall_time_us":17168,"spinlock_wait_cycles":8533760,"thread_start_us":99517,"threads_started":1,"wal-append.queue_time_us":242206}
I20260709 13:19:19.663414 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 556) took 2554 ms. Trace:
I20260709 13:19:19.659372 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:19.606896 18096 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:19.675071 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:19.675434 15601 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:19.630178 18179 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:19.672925 17978 consensus_queue.cc:1048] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:19:19.663858 15520 rpcz_store.cc:276] 0709 13:19:17.108680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.666221 (+1557541us) service_pool.cc:224] Handling call
0709 13:19:18.714074 (+ 47853us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:18.742370 (+ 28296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:18.742404 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:18.742954 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 13:19:18.862621 (+119667us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:18.910529 (+ 47908us) log.cc:844] Serialized 14 byte log entry
0709 13:19:19.181767 (+271238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.181849 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:19.566397 (+384548us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:19:19.652281 (+ 85884us) raft_consensus.cc:1685] finished
0709 13:19:19.663176 (+ 10895us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.663351 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":270982,"prepare.queue_time_us":51240,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":292015,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":163,"spinlock_wait_cycles":41929472,"thread_start_us":103735,"threads_started":1,"wal-append.queue_time_us":417602}
I20260709 13:19:19.685670 16203 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:19.688831 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:19.700809 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.701471 15669 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:19.703265 15528 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:19.714093 18177 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:19.715538 16205 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:19.716773 16205 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:19.718186 18057 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:19.718715 18057 raft_consensus.cc:697] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:19.719002 18141 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:19.719673 18057 consensus_queue.cc:237] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.719673 16204 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:19.721321 15669 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:19.721995 18328 consensus_queue.cc:1059] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:19:19.722810 18318 consensus_queue.cc:1048] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:19.722277 18319 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:19.723233 18328 raft_consensus.cc:3055] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:19.723520 18319 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.723920 18319 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:19.733566 18319 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:19.738795 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 123376us
I20260709 13:19:19.739518 18317 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:19:19.740342 15567 leader_election.cc:336] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.086s (SENT)
W20260709 13:19:19.750524 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 131579us
I20260709 13:19:19.686733 17958 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:19.753839 16234 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:19.694845 17942 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:19.710443 17978 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:19.748011 18328 raft_consensus.cc:740] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:19.783226 15493 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:19.784111 18152 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:19.785544 18152 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:19.749889 18056 consensus_queue.cc:1059] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:19:19.786096 18152 raft_consensus.cc:697] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:19.787070 18152 consensus_queue.cc:237] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.791937 15528 raft_consensus.cc:1275] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:19:19.791841 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:19.760933 17942 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:19.805013 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:19.805742 15520 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:19.807114 18328 consensus_queue.cc:260] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.808668 18328 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:19.816946 18179 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:19.766507 17978 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.688642 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:19:19.829288 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.831049 15568 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:19.835836 15568 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:19.841004 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:19.841952 16236 raft_consensus.cc:2368] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7.
I20260709 13:19:19.850378 15520 raft_consensus.cc:1275] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:19:19.829958 17978 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:19.854070 16233 consensus_meta.cc:287] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.586s	user 0.009s	sys 0.001s
I20260709 13:19:19.838692 18294 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:19.845038 18289 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:19.874408 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:19.888564 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:19.890173 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:19.877331 16233 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:19.885330 18239 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:19.902068 18239 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.875296 18048 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:19.886915 18289 raft_consensus.cc:697] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:19.908983 18289 consensus_queue.cc:237] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.909507 18048 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:19.912827 15675 raft_consensus.cc:1240] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:19.913864 15522 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:19.919203 15528 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:19.924463 18317 raft_consensus.cc:3055] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:19.926549 18317 raft_consensus.cc:740] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:19.922281 15598 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:19:19.927937 18322 consensus_queue.cc:1059] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:19.929877 15568 leader_election.cc:400] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7.
I20260709 13:19:19.930279 15568 leader_election.cc:403] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:19.930837 17984 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:19:19.925743 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:19.937160 15522 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:19.946871 18317 consensus_queue.cc:260] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.956429 15642 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:19.951565 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:19.960956 15601 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:19.964792 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:19.965149 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:19.966213 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:19.966811 15524 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:19.973902 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:19.974798 15520 raft_consensus.cc:2410] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:19:19.975344 17942 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:19.976449 15603 raft_consensus.cc:1240] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8. Ops: []
W20260709 13:19:19.978305 15567 leader_election.cc:336] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.380s (SENT)
I20260709 13:19:19.978823 15567 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:19.979619 18131 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:19.980425 17942 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:19.986693 15671 raft_consensus.cc:1275] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:19.969534 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:19.988515 15519 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:19.994638 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:19.996270 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:19.998586 18232 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:19.999241 18232 raft_consensus.cc:697] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:19.998837 15642 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:20.001967 15524 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:20.003196 15601 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:20.007403 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:20.008322 15522 raft_consensus.cc:2393] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:20.011415 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:20.020183 16204 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:20.028577 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.021389 18074 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:20.044386 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:20.018162 18177 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:19:20.048594 18074 raft_consensus.cc:697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:20.049772 18074 consensus_queue.cc:237] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:20.032016 17777 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:20.058085 18152 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:20.058560 17729 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:20.058702 18152 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:20.060925 18152 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:20.062364 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:20.048707 17984 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7.
I20260709 13:19:20.004015 17978 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:20.075986 18346 log.cc:927] Time spent T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7: Append to log took a long time: real 0.125s	user 0.000s	sys 0.002s
I20260709 13:19:20.064503 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:20.073021 15568 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:20.079674 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 51102.6us
I20260709 13:19:20.064411 15642 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:20.081715 15671 raft_consensus.cc:2410] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:19:20.094449 16237 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:20.095997 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:20.096601 15522 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:20.099676 16240 raft_consensus.cc:1275] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:20.090737 18131 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:20.092752 18167 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:20.124097 18167 raft_consensus.cc:697] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:20.125041 18167 consensus_queue.cc:237] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:20.126968 18239 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:20.115599 18319 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:20.121644 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:19:20.122143 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.123386 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:20.108294 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:20.131086 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:20.139072 15599 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:20.130511 15642 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:20.134640 15524 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:20.040396 18232 consensus_queue.cc:237] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:20.109185 18317 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:20.145622 18241 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:20.149760 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:20.150599 16237 raft_consensus.cc:2393] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 8: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:20.146641 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:20.154788 15524 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:20.168569 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:20.156293 16204 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:20.170270 18244 consensus_queue.cc:1059] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:19:20.174933 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:20.175642 15524 raft_consensus.cc:2376] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8. Re-sending same reply.
I20260709 13:19:20.191847 15521 raft_consensus.cc:2393] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:20.183393 17777 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:20.192549 17777 raft_consensus.cc:697] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:19:20.192849 15421 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49132 (request call id 337) took 18577 ms (client timeout 15000 ms). Trace:
W20260709 13:19:20.193274 15421 rpcz_store.cc:269] 0709 13:19:01.502391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.502626 (+   235us) service_pool.cc:224] Handling call
0709 13:19:20.074160 (+18571534us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:12.001076 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:12.001792 (+   716us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:12.001845 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:19:12.001866 (+    21us) tablet.cc:662] Decoding operations
0709 13:19:12.002970 (+  1104us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:12.003077 (+   107us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:12.003108 (+    31us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:19:12.011873 (+  8765us) tablet.cc:701] Row locks acquired
0709 13:19:12.034359 (+ 22486us) write_op.cc:260] PREPARE: finished
0709 13:19:12.084342 (+ 49983us) write_op.cc:270] Start()
0709 13:19:12.118822 (+ 34480us) write_op.cc:276] Timestamp: P: 1783603152069354 usec, L: 0
0709 13:19:12.118852 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:19:12.414472 (+295620us) log.cc:844] Serialized 3580 byte log entry
0709 13:19:14.803081 (+2388609us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce19010ac 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b
0709 13:19:15.828317 (+1025236us) spinlock_profiling.cc:243] Waited 1.04 ms on lock 0x7b180020b0c0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce45b34bc 00007f5ce190148b 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab0d1
0709 13:19:16.121717 (+293400us) op_driver.cc:464] REPLICATION: finished
0709 13:19:16.270429 (+148712us) write_op.cc:301] APPLY: starting
0709 13:19:16.270539 (+   110us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:16.270801 (+   262us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:16.270856 (+    55us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:16.272141 (+  1285us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:16.272509 (+   368us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 13:19:16.272567 (+    58us) write_op.cc:312] APPLY: finished
0709 13:19:16.272959 (+   392us) log.cc:844] Serialized 71 byte log entry
0709 13:19:16.273173 (+   214us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:16.273327 (+   154us) write_op.cc:454] Released schema lock
0709 13:19:16.273398 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:16.276710 (+  3312us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:16.276740 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2785083,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":135725,"apply.run_cpu_time_us":3452,"apply.run_wall_time_us":6286,"num_ops":9,"prepare.queue_time_us":118744,"prepare.run_cpu_time_us":66146,"prepare.run_wall_time_us":3827635,"raft.queue_time_us":638102,"raft.run_cpu_time_us":18112,"raft.run_wall_time_us":209590,"replication_time_us":3978481,"spinlock_wait_cycles":490370304,"thread_start_us":3132043,"threads_started":4,"wal-append.queue_time_us":2503362,"wal-append.run_cpu_time_us":102576,"wal-append.run_wall_time_us":2295909}]]}
I20260709 13:19:20.183652 17744 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:20.198463 17744 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:20.198861 17744 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:20.190529 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:20.199698 15524 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:20.187665 18179 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
W20260709 13:19:20.203446 18349 log.cc:927] Time spent T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
W20260709 13:19:20.213225 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.224844 15522 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:20.189395 18048 raft_consensus.cc:3055] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:20.189155 18141 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:19:20.234316 18048 raft_consensus.cc:740] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:20.249080 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:20.252059 15671 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:20.240310 17978 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:20.265192 15596 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:20.267717 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:20.272324 15519 raft_consensus.cc:1275] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:20.277488 15522 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 13:19:20.294093 18340 log.cc:927] Time spent T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.274s	user 0.000s	sys 0.001s
I20260709 13:19:20.299863 17777 consensus_queue.cc:237] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:20.303931 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:20.304713 15522 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:20.315209 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:20.315765 18170 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.315872 15519 raft_consensus.cc:2376] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6. Re-sending same reply.
I20260709 13:19:20.316334 18170 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:20.318527 18170 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:20.319125 15567 leader_election.cc:336] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.356s (SENT)
I20260709 13:19:20.322870 15567 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:20.324102 18289 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:20.324760 18289 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:20.325245 18289 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:20.315439 15598 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:19:20.335763 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:20.335412 18048 consensus_queue.cc:260] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:20.336449 15677 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:20.336843 18048 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:20.339648 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:20.340497 15519 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:20.342871 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:20.344826 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:20.347488 18289 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:20.347692 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:20.355422 15528 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:19:20.357391 18096 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:20.358448 15566 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:20.360339 18294 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:20.360734 18294 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:20.361083 18294 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:20.366765 18201 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:20.370843 16237 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:19:20.371480 17744 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:20.413295 17744 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:20.377825 18294 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:20.416569 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:20.419287 18294 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:20.428224 15670 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:20.400095 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:20.400638 17978 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:20.433255 17978 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:20.401067 15519 raft_consensus.cc:2410] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:19:20.401819 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:20.402042 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:20.402525 15522 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:20.403326 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:20.437489 16204 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:20.404500 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:20.438181 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:20.438709 18322 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:20.439280 18322 raft_consensus.cc:697] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:20.440184 18322 consensus_queue.cc:237] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:20.405544 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:20.385462 15671 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:20.435550 17978 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:20.430529 15641 leader_election.cc:336] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.365s (SENT)
I20260709 13:19:20.576539 15600 raft_consensus.cc:1240] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8. Ops: []
W20260709 13:19:20.578256 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:20.589481 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 159018us
W20260709 13:19:20.590293 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.592760 18364 consensus_queue.cc:1059] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:19:20.594040 18364 consensus_queue.cc:1048] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:20.595842 18074 consensus_queue.cc:1059] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:19:20.597311 18074 consensus_queue.cc:1048] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:20.597862 18232 raft_consensus.cc:3055] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:20.598392 18232 raft_consensus.cc:740] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:20.599156 18232 consensus_queue.cc:260] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:20.600826 18232 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:20.578131 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 130780us
I20260709 13:19:20.596177 18170 raft_consensus.cc:3055] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:20.597776 18290 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:20.601567 18170 raft_consensus.cc:740] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:20.601209 16234 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:20.602550 18170 consensus_queue.cc:260] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:20.604007 18170 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:20.574378 16237 raft_consensus.cc:2393] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 5: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:20.607249 18152 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.607798 18152 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:20.609031 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:20.609788 18152 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:20.614037 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:20.614874 16237 raft_consensus.cc:2393] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
W20260709 13:19:20.615571 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 158718us
I20260709 13:19:20.585570 15670 raft_consensus.cc:1275] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:20.621013 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:20.623576 15566 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:20.620971 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:20.632642 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:20.655308 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:20.658079 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:20.638069 18289 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:20.679940 15596 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:20.680004 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:20.675935 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:20.684149 16237 raft_consensus.cc:2393] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:20.650388 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:20.686627 15603 raft_consensus.cc:2393] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 5: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:20.687500 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:20.692469 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:20.662920 18167 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:19:20.714097 18048 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:20.704478 18294 consensus_queue.cc:1048] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:19:20.714958 15641 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:20.716753 15641 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:20.718806 18239 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:20.716938 18244 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:20.716418 17744 consensus_queue.cc:1048] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:19:20.727178 18152 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:20.727821 18152 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:20.730062 15567 leader_election.cc:336] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.807s (SENT)
I20260709 13:19:20.695406 18289 raft_consensus.cc:697] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:20.730525 15567 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:20.731654 15567 leader_election.cc:336] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.822s (SENT)
I20260709 13:19:20.745473 18289 consensus_queue.cc:237] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:20.746603 16205 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:20.747687 17958 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:20.748260 17958 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:20.714741 18048 raft_consensus.cc:697] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:20.757256 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:20.757855 15603 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:20.758026 16234 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:20.749049 18319 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:20.729779 18152 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:20.755115 16237 raft_consensus.cc:1275] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:19:20.750789 17942 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:20.764567 18048 consensus_queue.cc:237] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:20.768792 18331 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:20.773870 18239 consensus_queue.cc:1048] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:19:20.775632 15493 connection.cc:256] RPC call timeout handler was delayed by -1.0280836270001146s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15760000000000002s for a response
I20260709 13:19:20.787045 15642 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:20.790629 18331 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:20.792601 17958 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:20.793161 17958 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.793439 18331 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:20.793910 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:20.796000 16232 raft_consensus.cc:2393] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 8: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:20.796815 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:20.798171 15493 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:20.799433 15493 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:20.786249 15567 leader_election.cc:336] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.914s (SENT)
I20260709 13:19:20.785197 17744 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:19:20.802774 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:20.803524 15670 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:20.810719 17958 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:20.812973 17958 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:20.813565 17958 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:19:20.820068 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 44578.5us
I20260709 13:19:20.823101 17958 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:20.832535 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:20.834427 15598 raft_consensus.cc:2393] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 5: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:20.793354 18056 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:19:20.783054 18152 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:20.835351 17958 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:20.835280 18152 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:20.837167 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 108689us
I20260709 13:19:20.848529 15603 raft_consensus.cc:1275] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:19:20.822649 18317 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.845999 18241 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:20.886492 18322 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:20.887737 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 100777us
I20260709 13:19:20.849766 18377 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:20.901181 15642 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:20.904623 18239 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:20.905102 18239 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:20.915257 18239 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:20.918141 15423 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49144 (request call id 349) took 22986 ms (client timeout 15000 ms). Trace:
I20260709 13:19:20.900077 18177 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
W20260709 13:19:20.918623 15423 rpcz_store.cc:269] 0709 13:18:57.932362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.932798 (+   436us) service_pool.cc:224] Handling call
0709 13:18:58.146804 (+214006us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b5400002490. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e70b2 00007f5cea860a89 00007f5cea7f8da6 00007f5cea7ca088 00007f5cea8c837e 00007f5ce8d892a4 00007f5ce8d89232 00007f5ce8d89161 00007f5ce8d890dc
0709 13:19:20.918026 (+22771222us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:19.810113 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:19.810759 (+   646us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:19.810811 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:19:19.810846 (+    35us) tablet.cc:662] Decoding operations
0709 13:19:19.811333 (+   487us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:19.811416 (+    83us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:19.811443 (+    27us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:19:19.811883 (+   440us) tablet.cc:701] Row locks acquired
0709 13:19:19.811932 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:19:19.812085 (+   153us) write_op.cc:270] Start()
0709 13:19:19.812232 (+   147us) write_op.cc:276] Timestamp: P: 1783603159812026 usec, L: 0
0709 13:19:19.812256 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:19:19.813007 (+   751us) log.cc:844] Serialized 909 byte log entry
0709 13:19:20.416620 (+603613us) op_driver.cc:464] REPLICATION: finished
0709 13:19:20.699481 (+282861us) write_op.cc:301] APPLY: starting
0709 13:19:20.699606 (+   125us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:20.699742 (+   136us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:20.699877 (+   135us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:20.710257 (+ 10380us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:20.710538 (+   281us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:19:20.710580 (+    42us) write_op.cc:312] APPLY: finished
0709 13:19:20.710851 (+   271us) log.cc:844] Serialized 29 byte log entry
0709 13:19:20.711033 (+   182us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:20.711118 (+    85us) write_op.cc:454] Released schema lock
0709 13:19:20.711175 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:20.711609 (+   434us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:20.711638 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3319835,"reported_tablets":13,"spinlock_wait_cycles":42887040,"child_traces":[["op",{"apply.queue_time_us":249930,"apply.run_cpu_time_us":5201,"apply.run_wall_time_us":22831,"num_ops":2,"prepare.queue_time_us":151889,"prepare.run_cpu_time_us":9940,"prepare.run_wall_time_us":71264,"raft.queue_time_us":339164,"raft.run_cpu_time_us":12106,"raft.run_wall_time_us":233818,"replication_time_us":604194,"thread_start_us":552859,"threads_started":4,"wal-append.queue_time_us":86419}]]}
I20260709 13:19:20.885681 18241 raft_consensus.cc:697] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:20.946017 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:20.947845 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:20.948529 15525 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:20.859058 18317 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:20.954692 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 95158.5us
I20260709 13:19:20.955179 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:20.967465 15525 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:20.968952 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:20.948254 18377 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:20.972934 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:20.905400 18322 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:20.980108 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 180742us
I20260709 13:19:20.984615 18378 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:20.985337 18317 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:20.981850 18152 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:20.988515 16204 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:20.986166 18378 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.990561 18378 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:20.991542 17950 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:20.991950 17950 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:20.992312 17950 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:20.992677 18378 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:20.995653 16233 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:19:20.989777 18239 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:20.998487 18241 consensus_queue.cc:237] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.002130 15426 catalog_manager.cc:5697] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 2 to 3, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 3 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:21.004215 16240 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:21.005414 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:21.007915 18239 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:21.016268 15426 catalog_manager.cc:5697] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 1 to 2, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 2 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:21.017079 18162 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:21.018546 15426 catalog_manager.cc:5697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 2 to 3, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 3 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:19:21.020720 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.020394 15426 catalog_manager.cc:5697] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 3 to 4, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 4 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:21.020955 18170 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:21.022540 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:21.023257 16238 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:21.022526 18317 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.975589 18048 consensus_queue.cc:1048] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:19:21.025130 18317 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.028699 18322 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:21.020617 15670 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:21.031742 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:21.035154 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:21.035341 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 51745.8us
I20260709 13:19:21.035842 15595 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:21.036934 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:21.037621 15594 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:21.038789 18322 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:21.042200 15493 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:21.043828 16205 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:21.044430 15426 catalog_manager.cc:5697] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 1 to 3, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 3 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:21.047379 16238 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:21.047396 16205 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:21.048595 15642 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:21.057190 15426 catalog_manager.cc:5697] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 2 to 3, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 3 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:21.059180 15426 catalog_manager.cc:5697] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 2 to 3, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 3 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:21.060734 15426 catalog_manager.cc:5697] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 3 to 4, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 4 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:21.062012 18378 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:21.062744 17978 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:21.063155 17978 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.063473 17978 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:21.064056 17958 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.065223 15528 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:21.069114 18377 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:21.044131 18317 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:21.071969 15493 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:21.075831 18294 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:21.076419 18294 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.077402 15493 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:19:21.078081 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 83275.1us
W20260709 13:19:21.088268 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 117666us
I20260709 13:19:21.093859 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:21.080904 18331 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:21.099653 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:21.102154 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:21.102900 15522 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:21.105007 15426 catalog_manager.cc:5697] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 1 to 3, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 3 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:21.107803 15493 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:21.084493 18377 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:21.118588 17958 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:21.090847 18048 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:21.130316 18048 raft_consensus.cc:697] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:21.131253 18048 consensus_queue.cc:237] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.108968 18331 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.140587 18331 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:21.146667 15493 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:21.147576 17958 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:21.147675 18377 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:21.148232 18377 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:21.148615 18377 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:21.112991 15525 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:21.102531 17950 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.089214 18378 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:21.155396 15426 catalog_manager.cc:5697] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 2 to 4, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 4 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:21.156333 17950 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:21.116416 17978 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.161794 17978 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:21.163879 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:21.165040 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:21.165071 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:21.165714 15596 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:21.107177 18317 consensus_queue.cc:1059] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:19:21.165387 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:21.170559 18317 raft_consensus.cc:3055] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:21.107417 18322 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:19:21.170930 16238 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:21.172451 18177 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:21.147889 15493 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:21.170902 15600 raft_consensus.cc:2393] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 5: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:21.174762 15493 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:21.175536 17958 raft_consensus.cc:2804] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:21.175701 18378 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:21.175956 17958 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:21.176463 17958 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:21.157313 15426 catalog_manager.cc:5697] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 2 to 3, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 3 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:21.170969 18317 raft_consensus.cc:740] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:21.175984 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:21.180729 18317 consensus_queue.cc:260] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.182140 18317 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:21.182646 15675 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:21.169150 18377 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.185322 18377 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:21.187976 15493 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:19:21.251194 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 63342.4us
I20260709 13:19:21.252239 15493 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:21.188342 17958 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.256659 17958 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:21.258025 15528 raft_consensus.cc:1240] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:21.258344 15493 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:21.259879 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:21.260562 15519 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:21.188779 15674 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:21.173187 18294 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:21.205984 15426 catalog_manager.cc:5697] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 1 to 3, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 3 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:21.180752 16238 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:19:21.210933 15596 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:19:21.212736 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:21.239218 18331 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.254668 18378 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:21.254156 18389 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:21.259917 15493 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:21.274544 16205 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:21.274351 15525 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 3.
I20260709 13:19:21.286672 15426 catalog_manager.cc:5697] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 1 to 2, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 2 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:21.287653 15519 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:21.289348 15426 catalog_manager.cc:5697] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 1 to 2, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 2 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:21.291589 15493 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:21.295818 15426 catalog_manager.cc:5697] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:21.277117 17958 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:21.301534 18331 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:21.302348 18182 consensus_queue.cc:1059] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:21.305824 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:21.312114 15596 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:21.307684 16204 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:21.314584 15493 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:21.292528 18377 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:21.321576 15567 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.202s (SENT) [suppressed 33 similar messages]
I20260709 13:19:21.328986 15493 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:21.304390 18201 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:21.338922 16204 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:21.285117 18057 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:19:21.340658 15596 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:21.343791 16205 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:21.326895 18378 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:21.347968 15568 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:21.333820 17958 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:21.333513 18383 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:21.349085 15493 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:21.341996 15521 raft_consensus.cc:1240] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:21.342381 18056 raft_consensus.cc:2804] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:21.347759 18316 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:21.346668 18201 raft_consensus.cc:697] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:21.372388 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:21.374220 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:21.380867 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:21.381733 15522 raft_consensus.cc:2393] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:21.383765 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:21.384678 15520 raft_consensus.cc:2410] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:21.385855 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:21.356021 18294 raft_consensus.cc:2804] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:21.355960 18331 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:21.390055 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:21.368927 18056 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.400431 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:21.401362 15522 raft_consensus.cc:2393] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:21.402264 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:21.375342 17984 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:21.424750 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:21.380144 18378 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:21.427100 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:21.431248 15526 raft_consensus.cc:2393] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:21.432319 17984 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.392315 18294 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:21.435009 18294 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:21.434693 18201 consensus_queue.cc:237] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.422523 18331 raft_consensus.cc:697] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:21.438997 18331 consensus_queue.cc:237] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.442505 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:21.446329 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:21.447192 15522 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:21.448154 15567 leader_election.cc:336] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.905s (SENT)
I20260709 13:19:21.448621 15567 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:21.426224 15528 raft_consensus.cc:2955] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:21.455186 17942 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:21.458482 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:21.460914 15520 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:21.466864 15523 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:21.388928 18316 raft_consensus.cc:697] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:21.488858 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:21.403406 18056 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:21.495406 15522 raft_consensus.cc:2410] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:21.503122 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:21.511682 15523 raft_consensus.cc:1275] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:19:21.485352 18149 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:19:21.512204 15523 pending_rounds.cc:85] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 5
I20260709 13:19:21.520357 15523 pending_rounds.cc:107] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:19:21.525419 15523 raft_consensus.cc:2889] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:21.526933 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:19:21.530165 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 59367.7us
I20260709 13:19:21.527603 15525 raft_consensus.cc:2376] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7. Re-sending same reply.
I20260709 13:19:21.540755 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:21.541512 15523 raft_consensus.cc:2393] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:21.527531 17777 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:21.544454 18316 consensus_queue.cc:237] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.552127 18366 raft_consensus.cc:3055] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:21.552608 18366 raft_consensus.cc:740] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:21.553458 18366 consensus_queue.cc:260] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.555418 18074 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.555997 18074 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.557937 18074 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:21.558578 18393 log.cc:927] Time spent T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
W20260709 13:19:21.560950 18396 log.cc:927] Time spent T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:19:21.566056 17777 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.569078 17777 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:21.570742 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:21.575834 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:21.576263 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:21.576982 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:21.580470 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:21.581984 15526 raft_consensus.cc:2393] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 6: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:21.583374 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:21.583590 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:21.584101 15523 raft_consensus.cc:2393] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:21.585619 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:21.582233 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:21.591781 15567 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:21.592218 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:21.593400 18149 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:21.593832 18149 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:21.594305 18149 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:21.594588 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:21.593624 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:21.588869 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:21.601686 15523 raft_consensus.cc:2410] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:19:21.601444 15567 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:21.594192 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:21.602523 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:21.603749 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:21.603875 18179 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:21.605283 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:21.587517 18400 log.cc:927] Time spent T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:19:21.605993 15519 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:21.603983 18149 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:21.593027 15521 raft_consensus.cc:2393] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 3: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:21.605229 15567 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:21.611081 15525 raft_consensus.cc:1275] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:19:21.604413 15526 raft_consensus.cc:2376] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8. Re-sending same reply.
I20260709 13:19:21.612325 15525 pending_rounds.cc:85] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 2
I20260709 13:19:21.613617 15525 pending_rounds.cc:107] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:19:21.613870 15525 raft_consensus.cc:2889] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:21.617628 15526 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:21.618124 18149 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:21.627791 15528 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:19:21.630173 18131 raft_consensus.cc:2764] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 13:19:21.634213 15567 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:21.626572 18378 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:21.636605 15493 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:21.633590 18149 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:19:21.649259 18179 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:21.614704 18294 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.655900 15493 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:21.638427 18378 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.650292 17958 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:21.650120 18179 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:21.672662 16238 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:21.670608 15493 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:21.674885 15493 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:21.675673 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:21.676242 15493 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:21.677317 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:21.682655 18167 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:21.683598 18167 consensus_queue.cc:1243] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer dd1a14c3173d49dc9b55bb116e3783c9 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:19:21.684943 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:21.685398 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:21.687215 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:21.688970 15677 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:21.667342 18383 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:21.705451 15594 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:21.707607 18056 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.714421 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:21.725733 15597 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:21.741647 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:21.744129 18056 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:21.684453 18152 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:21.686489 18377 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:21.685015 17958 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:21.684226 18364 raft_consensus.cc:3055] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:21.758315 18378 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:21.721055 18057 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:21.760196 16237 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
W20260709 13:19:21.764106 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 50166.3us
W20260709 13:19:21.768981 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:21.773767 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 97591.6us
I20260709 13:19:21.776723 15493 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:21.777257 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:21.699062 18179 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.774178 18056 consensus_queue.cc:1059] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:19:21.781216 18179 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:21.781356 18056 raft_consensus.cc:3055] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:21.786691 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:21.757365 18364 raft_consensus.cc:740] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:21.787302 15567 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:21.792758 18364 consensus_queue.cc:260] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.794013 18179 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:21.796201 15641 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:21.796671 18294 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:21.797122 18074 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:21.771605 18057 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.800760 18366 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:21.802699 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:21.804129 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:21.804840 15526 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:21.777609 15523 raft_consensus.cc:2955] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 3 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:21.777976 15595 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:21.810721 15493 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:21.822742 15493 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:19:21.826836 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 124371us
I20260709 13:19:21.827472 15493 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:21.834115 18416 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:21.834887 18416 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:21.834898 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:21.836987 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:21.837168 17984 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:21.838827 18057 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:21.840684 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:21.838070 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:21.781803 18056 raft_consensus.cc:740] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:21.841642 15522 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:21.851346 15524 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:21.844432 16204 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:21.794034 18364 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:21.833621 16233 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:21.858654 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:21.825888 18152 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:21.859856 16236 raft_consensus.cc:2393] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 6: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:21.867340 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:21.872112 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:21.872910 16237 raft_consensus.cc:2368] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 3. Current term is 4.
I20260709 13:19:21.874003 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:21.874717 16237 raft_consensus.cc:2393] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 6: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:21.857462 18331 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:21.855329 18383 raft_consensus.cc:697] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:21.845822 18074 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:19:21.896908 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:21.845636 18290 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:19:21.898283 15522 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:21.906071 15674 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:21.900770 15568 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:21.895426 16232 raft_consensus.cc:1275] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:21.906867 15595 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:21.913132 15568 leader_election.cc:400] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 3. Current term is 4.
I20260709 13:19:21.918377 15568 leader_election.cc:403] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:19:21.926113 15527 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:21.925897 17984 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:21.929955 18294 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 3. Current term is 4.
I20260709 13:19:21.917315 18056 consensus_queue.cc:260] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:21.974555 15526 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:21.978423 16204 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:22.002553 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:19:22.025234 18420 log.cc:927] Time spent T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:19:22.051605 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.009207 17777 raft_consensus.cc:2804] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:22.081067 15594 raft_consensus.cc:1240] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: []
W20260709 13:19:22.156299 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 328890us
I20260709 13:19:21.978268 18383 consensus_queue.cc:237] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:22.198359 17984 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:22.335800 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:22.344292 15493 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:22.337057 15642 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:22.334450 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:22.351436 15673 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
W20260709 13:19:22.353502 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 273901us
W20260709 13:19:22.350997 18420 log.cc:927] Time spent T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.325s	user 0.002s	sys 0.012s
I20260709 13:19:22.350781 18331 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:22.353040 17777 raft_consensus.cc:697] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:22.359719 18239 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:22.367233 16240 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:22.369954 17777 consensus_queue.cc:237] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:22.370234 16232 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:19:22.371045 18433 log.cc:927] Time spent T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:19:22.366843 18432 consensus_queue.cc:1059] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:19:22.375718 18430 raft_consensus.cc:3055] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:22.376049 18430 raft_consensus.cc:740] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:22.361022 18057 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:22.377117 16236 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:22.365372 17978 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:22.368661 18239 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.398797 18239 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:22.379418 18430 consensus_queue.cc:260] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:22.379211 18057 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:22.378675 18201 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.398175 17978 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:22.389670 18364 consensus_meta.cc:287] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.532s	user 0.006s	sys 0.004s
W20260709 13:19:22.391793 18366 consensus_meta.cc:287] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.590s	user 0.013s	sys 0.000s
I20260709 13:19:22.418736 15674 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:19:22.419128 15674 pending_rounds.cc:85] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting all ops after (but not including) 7
I20260709 13:19:22.419373 15674 pending_rounds.cc:107] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:19:22.419595 15674 raft_consensus.cc:2889] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:22.420614 18289 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:22.421229 18289 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:22.421813 18057 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:22.422415 18201 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:22.423374 18289 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:22.424580 18201 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:22.425351 18430 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:22.425879 16203 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:22.426863 18366 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:22.427264 18366 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:22.427626 18366 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:22.440485 15674 raft_consensus.cc:1275] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:19:22.436654 18057 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.442432 18057 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:22.444581 18057 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:22.420807 17978 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:22.448700 16238 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:22.457719 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:22.449609 18241 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.466122 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:22.601758 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 235636us
W20260709 13:19:22.577050 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.466403 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:22.641512 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:22.563535 18289 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:22.495075 18057 consensus_queue.cc:1048] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.153s
I20260709 13:19:22.652410 18289 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:22.466894 18239 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:22.493103 17978 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.154s
I20260709 13:19:22.656682 18289 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:22.656757 18239 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:22.504870 18366 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:22.636094 18241 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:22.659603 18415 log.cc:927] Time spent T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.824s	user 0.004s	sys 0.020s
I20260709 13:19:22.664291 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1717) took 1079 ms. Trace:
I20260709 13:19:22.665622 15519 rpcz_store.cc:276] 0709 13:19:21.583878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.606625 (+ 22747us) service_pool.cc:224] Handling call
0709 13:19:21.606938 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:21.607285 (+   347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:21.607315 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:21.608149 (+   834us) log.cc:844] Serialized 26 byte log entry
0709 13:19:21.664233 (+ 56084us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:21.823892 (+159659us) log.cc:844] Serialized 14 byte log entry
0709 13:19:21.824102 (+   210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:21.824196 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:22.451676 (+627480us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b1800361e40. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce191e3f3 00007f5ce19207c1 00007f5ce1920779 00007f5ce1920709 00007f5ce19206d1 00007f5ce191f9cd 000000000038f354 000000000038f199 00007f5ce190183a
0709 13:19:22.660611 (+208935us) raft_consensus.cc:1685] finished
0709 13:19:22.662670 (+  2059us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:22.662779 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":66477,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":10354,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":202,"spinlock_wait_cycles":6835456,"thread_start_us":187299,"threads_started":2,"wal-append.queue_time_us":147477}
I20260709 13:19:22.673319 18241 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:22.675125 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:22.675793 15603 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:19:22.676071 15595 raft_consensus.cc:2393] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 8: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:22.677232 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:22.677897 15596 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:22.678844 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:22.679433 15599 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
W20260709 13:19:22.691880 18442 log.cc:927] Time spent T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:19:22.692767 18366 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:22.693452 15595 raft_consensus.cc:1275] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:19:22.659695 18239 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:19:22.696213 16205 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:22.696913 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:22.697119 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:19:22.698701 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 49952.4us
I20260709 13:19:22.699643 15595 raft_consensus.cc:2393] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 8: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:22.700358 15673 raft_consensus.cc:2368] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 8. Current term is 9.
I20260709 13:19:22.701574 18201 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:22.702067 18201 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:22.702414 18201 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:22.701670 18439 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:19:22.709180 18201 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:22.710356 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1724) took 1081 ms. Trace:
I20260709 13:19:22.711792 16205 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:22.712988 18440 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:22.713881 18319 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.714485 18378 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:22.710835 15528 rpcz_store.cc:276] 0709 13:19:21.628708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.629625 (+   917us) service_pool.cc:224] Handling call
0709 13:19:21.629986 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:21.630314 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:21.630342 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:21.630784 (+   442us) log.cc:844] Serialized 26 byte log entry
0709 13:19:21.919083 (+288299us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:21.946217 (+ 27134us) log.cc:844] Serialized 14 byte log entry
0709 13:19:22.709721 (+763504us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b180039eb60. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1898c0a 00007f5ceafd8219 00007f5ce1920a40 00007f5ce191b4e4 00007f5ce191b455 00007f5ce4a03060 00007f5ce4a15716 00007f5ce4a26287 00007f5ce4a26208
0709 13:19:22.709906 (+   185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:22.710012 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:22.710055 (+    43us) raft_consensus.cc:1685] finished
0709 13:19:22.710130 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:22.710289 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15966,"prepare.queue_time_us":28166,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":1061,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":1831,"spinlock_wait_cycles":24733440,"thread_start_us":287943,"threads_started":1,"wal-append.queue_time_us":422340}
I20260709 13:19:22.715647 15525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1725) took 1084 ms. Trace:
I20260709 13:19:22.718273 16203 leader_election.cc:400] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 8. Current term is 9.
I20260709 13:19:22.718562 16203 leader_election.cc:403] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:19:22.716039 15525 rpcz_store.cc:276] 0709 13:19:21.631713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.853104 (+221391us) service_pool.cc:224] Handling call
0709 13:19:21.853472 (+   368us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:21.853841 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:21.853868 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:21.854441 (+   573us) log.cc:844] Serialized 52 byte log entry
0709 13:19:21.932689 (+ 78248us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:21.999725 (+ 67036us) log.cc:844] Serialized 14 byte log entry
0709 13:19:22.670389 (+670664us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b180004c840. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce191e3f3 00007f5ce19207c1 00007f5ce1920779 00007f5ce1920709 00007f5ce19206d1 00007f5ce191f9cd 000000000038f354 000000000038f199 00007f5ce190183a
0709 13:19:22.715105 (+ 44716us) log.cc:844] Serialized 14 byte log entry
0709 13:19:22.715285 (+   180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:22.715355 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:22.715461 (+   106us) raft_consensus.cc:1685] finished
0709 13:19:22.715474 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:22.715594 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":739837,"prepare.queue_time_us":79908,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":1546,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":4767,"spinlock_wait_cycles":49841024,"thread_start_us":39786,"threads_started":1,"wal-append.queue_time_us":155670}
I20260709 13:19:22.697665 17777 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
W20260709 13:19:22.721199 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 655294us
I20260709 13:19:22.732091 15525 raft_consensus.cc:1275] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:19:22.733399 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:22.738930 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1719) took 1154 ms. Trace:
I20260709 13:19:22.739631 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:22.739426 15523 rpcz_store.cc:276] 0709 13:19:21.584311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.613665 (+ 29354us) service_pool.cc:224] Handling call
0709 13:19:21.614211 (+   546us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:21.620968 (+  6757us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:21.620994 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:21.651387 (+ 30393us) log.cc:844] Serialized 556 byte log entry
0709 13:19:21.777492 (+126105us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:22.077091 (+299599us) log.cc:844] Serialized 14 byte log entry
0709 13:19:22.651421 (+574330us) spinlock_profiling.cc:243] Waited 5.43 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:19:22.721737 (+ 70316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:22.721808 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:22.721844 (+    36us) raft_consensus.cc:1685] finished
0709 13:19:22.738671 (+ 16827us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:22.738859 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":644383,"spinlock_wait_cycles":12489088,"thread_start_us":116451,"threads_started":1,"wal-append.queue_time_us":177943}
I20260709 13:19:22.713405 18440 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.750520 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:22.751185 15595 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:22.752223 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:22.752990 15600 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:22.750520 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:22.757071 15524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 633) took 1066 ms. Trace:
I20260709 13:19:22.757148 15525 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:22.757858 15524 rpcz_store.cc:276] 0709 13:19:21.690112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.878088 (+187976us) service_pool.cc:224] Handling call
0709 13:19:21.878453 (+   365us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:21.897230 (+ 18777us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:21.897259 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:21.906884 (+  9625us) log.cc:844] Serialized 52 byte log entry
0709 13:19:21.943252 (+ 36368us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:22.346600 (+403348us) log.cc:844] Serialized 14 byte log entry
0709 13:19:22.756756 (+410156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:22.756837 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:22.756881 (+    44us) raft_consensus.cc:1685] finished
0709 13:19:22.756894 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:22.757007 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":435027,"prepare.queue_time_us":78994,"prepare.run_cpu_time_us":581,"prepare.run_wall_time_us":3825,"raft.queue_time_us":18745,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":7832,"thread_start_us":378592,"threads_started":3,"wal-append.queue_time_us":271510}
I20260709 13:19:22.730120 17978 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:22.714434 18319 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:22.771595 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.748814 18377 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.779393 15595 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:19:22.756412 18440 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:22.778658 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:19:22.892444 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 548231us
W20260709 13:19:22.895383 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 84428.4us
W20260709 13:19:22.868005 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 99712.1us
I20260709 13:19:22.894698 15493 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:22.804631 17958 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.803853 18383 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:22.812630 18201 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:22.805369 18377 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:22.897773 15642 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:22.932675 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 643) took 1111 ms. Trace:
I20260709 13:19:22.934278 15642 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:19:22.934713 15527 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1658) took 1560 ms (client timeout 1822 ms). Trace:
W20260709 13:19:22.935539 15527 rpcz_store.cc:269] 0709 13:19:21.373850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.373972 (+   122us) service_pool.cc:224] Handling call
0709 13:19:22.933957 (+1559985us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b54003ce1d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:22.934644 (+   687us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1262930688}
I20260709 13:19:22.933251 15520 rpcz_store.cc:276] 0709 13:19:21.820996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.003086 (+182090us) service_pool.cc:224] Handling call
0709 13:19:22.011054 (+  7968us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:22.011266 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:22.026143 (+ 14877us) log.cc:844] Serialized 14 byte log entry
0709 13:19:22.913676 (+887533us) log.cc:844] Serialized 14 byte log entry
0709 13:19:22.932315 (+ 18639us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:22.932371 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:22.932421 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:22.932493 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:22.932612 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":750326,"prepare.queue_time_us":61472,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":67,"raft.queue_time_us":702743,"spinlock_wait_cycles":10607872,"thread_start_us":169578,"threads_started":1}
I20260709 13:19:22.933375 15493 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:19:22.939208 18446 log.cc:927] Time spent T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:19:22.939438 16240 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57244 (request call id 971) took 1072 ms. Trace:
I20260709 13:19:22.940474 16240 rpcz_store.cc:276] 0709 13:19:21.866964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.867119 (+   155us) service_pool.cc:224] Handling call
0709 13:19:22.939233 (+1072114us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b54008507d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:22.939351 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1148026624}
I20260709 13:19:22.912779 17958 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:22.974664 15527 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:22.949210 18365 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:22.948899 18170 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:22.973079 15520 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:22.948382 18201 consensus_queue.cc:1048] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.034s
W20260709 13:19:22.984026 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 87465.6us
I20260709 13:19:22.955935 18451 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:22.985740 16204 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:22.984062 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:22.988536 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:22.989194 15520 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:22.980521 18385 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:22.981441 18365 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.012182 18385 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:23.014437 18385 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:23.015127 17978 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 8. Current term is 9.
I20260709 13:19:23.015862 17958 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:23.016006 17978 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:23.020165 16204 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:23.036049 18366 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:23.036489 18366 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:23.036863 18366 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:23.037204 18377 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:23.022373 18365 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:23.034792 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 44941.5us
I20260709 13:19:23.045174 16233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 585) took 1332 ms. Trace:
I20260709 13:19:23.045653 16233 rpcz_store.cc:276] 0709 13:19:21.712769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.734410 (+ 21641us) service_pool.cc:224] Handling call
0709 13:19:23.044988 (+1310578us) spinlock_profiling.cc:243] Waited 91.2 ms on lock 0x7b540041b4d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:23.045112 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":209645312}
I20260709 13:19:23.061121 18440 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:23.066184 18239 consensus_queue.cc:1048] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:19:23.068120 16238 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57258 (request call id 656) took 1391 ms (client timeout 1718 ms). Trace:
I20260709 13:19:23.068710 18239 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:23.069303 18239 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:23.068329 18048 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:19:23.075354 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1681) took 1266 ms. Trace:
I20260709 13:19:23.075762 15522 rpcz_store.cc:276] 0709 13:19:21.808895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.001250 (+192355us) service_pool.cc:224] Handling call
0709 13:19:22.003227 (+  1977us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:22.717130 (+713903us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:19:22.740164 (+ 23034us) log.cc:844] Serialized 14 byte log entry
0709 13:19:22.938616 (+198452us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:22.939340 (+   724us) log.cc:844] Serialized 26 byte log entry
0709 13:19:22.939545 (+   205us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:22.939900 (+   355us) log.cc:844] Serialized 14 byte log entry
0709 13:19:23.025883 (+ 85983us) spinlock_profiling.cc:243] Waited 3.87 ms on lock 0x7b1800292fc0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce18ec2a6 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce9934aa1 00007f5ce992bb37 00007f5ce992b9ca 00007f5ce9a2326f
0709 13:19:23.026375 (+   492us) spinlock_profiling.cc:243] Waited 3.94 ms on lock 0x7b4800002958. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce9934aa1 00007f5ce992bb37 00007f5ce992b9ca 00007f5ce9a2326f 00007f5ce9a22f37 00007f5ce9a3755c 00007f5ce9a37508 00007f5ce9a37488 00007f5ce9a37443 00007f5ce9a366ec 00007f5ce8960a96
0709 13:19:23.074893 (+ 48518us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b1800292fc0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1898c0a 00007f5ceafd8219 00007f5ce1920a40 00007f5ce191b4e4 00007f5ce191b455 00007f5ce4a03060 00007f5ce4a15716 00007f5ce4a26287 00007f5ce4a26208
0709 13:19:23.075048 (+   155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.075116 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:23.075157 (+    41us) raft_consensus.cc:1685] finished
0709 13:19:23.075170 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:23.075295 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":730961,"prepare.queue_time_us":29451,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":1403,"raft.queue_time_us":733823,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":291306,"spinlock_wait_cycles":31456128,"thread_start_us":158358,"threads_started":1,"wal-append.queue_time_us":260648}
I20260709 13:19:23.078302 18383 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:23.081282 18239 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:23.081661 18447 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:23.082465 18056 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:23.082257 18447 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:23.084440 18447 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:23.081393 18378 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:23.086333 18378 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:23.088171 18378 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:23.088840 18447 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:23.089323 18447 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:23.092103 18447 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:23.093228 15493 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:23.094036 17958 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:23.041342 18170 raft_consensus.cc:697] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:23.098013 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:23.105940 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:23.107472 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:23.110764 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:23.118886 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 343720us
I20260709 13:19:23.111486 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:23.068593 16238 rpcz_store.cc:269] 0709 13:19:21.676984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.677103 (+   119us) service_pool.cc:224] Handling call
0709 13:19:23.067954 (+1390851us) spinlock_profiling.cc:243] Waited 739 ms on lock 0x7b54004eedd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:23.068063 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1699777920}
I20260709 13:19:23.113976 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:23.113283 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:23.122766 15522 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:23.115208 15527 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:23.112063 18451 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.094528 17958 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:23.113514 18378 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:23.128585 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:23.178220 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:23.165119 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:23.196709 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:23.094381 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:19:23.175920 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 64123.6us
I20260709 13:19:23.199004 16238 raft_consensus.cc:1240] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:23.088222 18365 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:23.203751 15676 raft_consensus.cc:2393] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 6: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:23.206483 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:23.198791 15566 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:23.170814 18322 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.128660 18319 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:23.217636 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 123319us
I20260709 13:19:23.217764 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:23.133833 18131 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:23.163733 18447 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:23.223495 15493 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:23.176262 18468 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:23.124915 18383 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:23.238909 15603 raft_consensus.cc:1275] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:23.212277 18389 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:23.134764 17978 consensus_queue.cc:1048] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 13:19:23.129753 18378 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:23.115449 18366 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:23.262187 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 60431.9us
I20260709 13:19:23.223307 18449 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:23.253111 16240 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:23.224248 18289 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:19:23.265743 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 52792us
I20260709 13:19:23.125491 18451 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:23.264791 18383 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:23.285454 15566 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:23.205192 18170 consensus_queue.cc:237] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:23.253273 18448 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:23.247951 18468 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:23.297096 18451 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:23.244230 18447 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:23.307040 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 69970.6us
I20260709 13:19:23.309827 15493 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:19:23.289531 18465 log.cc:927] Time spent T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:19:23.280500 18389 raft_consensus.cc:697] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:23.238615 18131 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:23.288889 18419 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:23.235199 18322 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:23.314577 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:23.317355 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:23.294832 18289 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:23.326910 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:23.357115 15520 raft_consensus.cc:2393] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 6: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:23.327859 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:23.357728 15522 raft_consensus.cc:2393] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:23.338904 18451 raft_consensus.cc:2804] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:23.305267 18365 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:23.339274 18389 consensus_queue.cc:237] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:23.340543 18383 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:23.348249 18468 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:23.350207 18378 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:23.350561 18131 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:23.298254 18440 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:23.352767 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 641) took 1582 ms. Trace:
I20260709 13:19:23.354118 15603 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:19:23.279817 18416 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:19:23.307343 16232 raft_consensus.cc:1240] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:23.360467 15600 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:23.363991 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:23.364250 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:23.367892 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:23.369424 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:23.371780 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:23.378438 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:23.370900 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:23.379488 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:23.384032 15523 raft_consensus.cc:1275] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:23.383798 15527 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:23.384716 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:23.386729 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:23.366101 18322 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:23.387486 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:23.392735 16236 raft_consensus.cc:1240] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 6. Current term is 7. Ops: []
W20260709 13:19:23.393111 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 129955us
I20260709 13:19:23.356591 18289 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:23.393584 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:23.380353 18451 raft_consensus.cc:697] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:23.395848 15526 rpcz_store.cc:276] 0709 13:19:21.770829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.975991 (+205162us) service_pool.cc:224] Handling call
0709 13:19:21.976310 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:22.669550 (+693240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:22.669579 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:22.669618 (+    39us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:22.669648 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.352523 (+682875us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b5400023d54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:23.352572 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:23.352702 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1593917312}
W20260709 13:19:23.402287 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 100656us
I20260709 13:19:23.396174 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:23.395308 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:23.507035 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:23.507982 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:23.508739 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:23.509160 15523 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:23.510929 15674 raft_consensus.cc:2410] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:19:23.517486 15523 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:23.406256 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:23.430265 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:23.512691 16204 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:23.392596 18383 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:23.388111 18056 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:23.529358 18447 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:23.512044 18074 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:19:23.535986 18366 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:23.536517 17978 consensus_queue.cc:1048] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:23.401259 18421 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:23.539870 18421 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:23.357262 18419 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:23.552021 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:23.554055 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:23.554636 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:23.555223 15526 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:23.557542 15527 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:23.559800 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:23.396229 18476 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:23.562328 15603 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:23.484225 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:23.510588 18289 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:23.575827 15526 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:23.593587 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:23.631860 15675 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
W20260709 13:19:23.650609 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 55466us
I20260709 13:19:23.529270 18477 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:23.651561 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:23.654697 16204 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:23.655330 18481 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:23.659013 18450 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:23.541419 18451 consensus_queue.cc:237] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:23.661644 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:23.673707 16236 raft_consensus.cc:2393] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:23.677264 15642 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:23.660696 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:23.681062 16239 raft_consensus.cc:2393] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:23.678506 15491 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:23.559712 18383 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:23.559881 18056 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:23.672953 18450 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:23.684912 18239 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:19:23.831517 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 127704us
I20260709 13:19:23.845847 16238 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:19:23.689636 18432 consensus_queue.cc:1059] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.160s
I20260709 13:19:23.831107 18439 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:23.840036 18162 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:23.848064 18416 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:19:23.913574 18486 log.cc:927] Time spent T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:19:23.645044 18476 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:23.673533 18477 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:23.936038 18477 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:23.663514 18481 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:23.945035 18290 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:23.957782 18290 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:23.969831 18491 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:23.970520 18491 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:23.937033 18416 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:23.975241 18056 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:23.929579 18239 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:23.977494 18481 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:23.978413 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:23.902115 18162 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:23.949764 18421 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:23.986814 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:23.901762 18439 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:24.000805 18383 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:23.996622 18419 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:24.004781 18491 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:24.014897 18162 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:23.981401 18239 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:24.018425 18290 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:24.074023 18239 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:24.066154 18428 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.075209 18428 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:24.093783 18466 log.cc:927] Time spent T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.745s	user 0.000s	sys 0.011s
I20260709 13:19:24.104504 18439 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:24.110905 18289 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:24.124190 17958 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:24.185822 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 501551us
W20260709 13:19:24.197124 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 503479us
W20260709 13:19:24.217504 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 56389.6us
I20260709 13:19:24.219357 15567 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:24.219781 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:24.220121 15602 raft_consensus.cc:2410] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:19:24.221397 18290 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.221441 18445 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:24.222784 18443 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:24.137460 18464 log.cc:927] Time spent T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.822s	user 0.006s	sys 0.000s
I20260709 13:19:24.148499 18377 consensus_queue.cc:1059] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:24.221962 18290 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:24.149478 15493 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:24.224005 16240 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57244 (request call id 982) took 1116 ms. Trace:
W20260709 13:19:24.224582 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 75192us
I20260709 13:19:24.225347 15493 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:24.226073 18449 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:24.226905 16205 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:24.226331 18443 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:24.228076 18383 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:24.228648 18383 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:24.226599 18290 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:24.230149 18331 consensus_queue.cc:1048] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:24.230803 18383 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:24.231333 18383 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:24.231794 18383 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:24.223973 18377 raft_consensus.cc:3055] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:24.228749 18440 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:23.688957 18057 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:23.701604 18074 consensus_queue.cc:1059] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.462s
I20260709 13:19:23.708076 18048 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:19:24.155675 15524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.853s	user 0.006s	sys 0.005s
W20260709 13:19:24.269363 15524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.970s	user 0.006s	sys 0.005s
I20260709 13:19:24.158241 17744 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:24.159824 18432 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:24.161264 18450 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:24.163679 18416 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:24.171123 18490 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:24.222652 18445 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:24.285115 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 44811.8us
I20260709 13:19:24.222395 18170 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:24.234006 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:24.235949 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 365807us
I20260709 13:19:24.237671 18458 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:19:24.237049 18377 raft_consensus.cc:740] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:24.236609 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:24.294745 16238 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:19:24.240013 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 51256.4us
I20260709 13:19:24.296667 16204 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:24.247238 15523 raft_consensus.cc:1240] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:19:24.250176 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:24.250818 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:24.256014 16240 rpcz_store.cc:276] 0709 13:19:23.107099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.107227 (+   128us) service_pool.cc:224] Handling call
0709 13:19:24.182519 (+1075292us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54005703d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:24.223903 (+ 41384us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":309802368}
I20260709 13:19:24.252138 15493 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:24.059397 18389 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:24.259634 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:24.222456 18289 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:24.290813 18490 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:24.272202 18449 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:24.233891 18383 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:24.271934 15524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1733) took 1313 ms. Trace:
I20260709 13:19:24.288435 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:24.307117 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:24.306568 18385 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.278789 17744 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:24.259172 18443 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:24.311905 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:24.269506 18421 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:19:24.327196 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:24.327769 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:24.330547 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:24.354508 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:24.334360 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:24.331164 15524 rpcz_store.cc:276] 0709 13:19:22.958661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.958781 (+   120us) service_pool.cc:224] Handling call
0709 13:19:22.971926 (+ 13145us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:23.078785 (+106859us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:23.078814 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:23.079395 (+   581us) log.cc:844] Serialized 52 byte log entry
0709 13:19:23.195697 (+116302us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:23.241576 (+ 45879us) log.cc:844] Serialized 14 byte log entry
0709 13:19:23.298822 (+ 57246us) log.cc:844] Serialized 14 byte log entry
0709 13:19:24.136896 (+838074us) spinlock_profiling.cc:243] Waited 9.8 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:19:24.269762 (+132866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:24.269860 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:24.271700 (+  1840us) raft_consensus.cc:1685] finished
0709 13:19:24.271719 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:24.271872 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119806,"prepare.queue_time_us":49463,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":17076,"raft.queue_time_us":201847,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":13685,"spinlock_wait_cycles":22542080,"thread_start_us":1143344,"threads_started":3,"wal-append.queue_time_us":201528}
I20260709 13:19:24.340504 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:24.355695 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:24.358628 16240 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:24.340406 16236 raft_consensus.cc:1240] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: []
W20260709 13:19:24.342861 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 123380us
W20260709 13:19:24.343034 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 123765us
I20260709 13:19:24.360872 15567 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:24.319609 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 67560.1us
I20260709 13:19:24.333081 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:24.285183 18432 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:24.311609 18416 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:24.310478 18389 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:24.327775 18378 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:24.328536 17958 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:24.338513 18484 raft_consensus.cc:3055] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:24.365155 15598 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 5: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:24.347086 18179 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:24.311556 18445 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:24.367542 15594 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:24.386677 17958 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:24.370745 18490 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:24.361778 15493 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:24.329815 18364 raft_consensus.cc:3055] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:24.375504 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:24.308768 18468 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.383428 18432 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:24.384927 17744 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:24.312572 18170 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:24.361824 18479 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.375607 18389 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:24.364717 18457 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:24.362286 18458 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.379812 18484 raft_consensus.cc:740] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:24.347533 18483 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:24.373790 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:24.402079 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:24.411566 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:24.423028 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 68231.6us
I20260709 13:19:24.441396 15568 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:24.355643 18048 raft_consensus.cc:697] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:24.356688 18385 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:24.394897 17958 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:24.441234 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 79538.5us
I20260709 13:19:24.443045 15568 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:24.444213 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:24.451035 15595 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:24.405674 18364 raft_consensus.cc:740] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:24.452170 15568 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:24.453660 15493 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:24.412907 18477 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:24.453519 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:24.386284 18378 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:24.457644 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:24.458020 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:24.461777 15493 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:24.460850 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:24.455607 18457 raft_consensus.cc:697] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:24.462841 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:24.436347 18490 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:24.459663 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:24.464511 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:24.438464 18131 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:24.469476 15598 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:24.471164 15595 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:24.459187 18471 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:24.426905 18244 consensus_queue.cc:1059] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:19:24.474009 16205 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:24.439579 18479 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:24.422717 18468 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } }
I20260709 13:19:24.489558 18468 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:24.482682 18317 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:19:24.483332 18500 log.cc:927] Time spent T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:19:24.492017 15594 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:24.465267 17958 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:24.478147 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:24.474602 18474 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:24.474395 18508 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:24.441939 18458 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:24.532313 18458 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:24.479276 18131 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:24.453433 18483 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:24.534819 18131 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:24.535181 18483 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:24.535802 18457 consensus_queue.cc:237] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:24.538096 18504 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:24.539230 18479 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:24.539805 18385 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:24.493113 18364 consensus_queue.cc:260] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:24.481053 18048 consensus_queue.cc:237] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:24.512151 18490 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:25.190474 18498 log.cc:927] Time spent T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.791s	user 0.010s	sys 0.006s
I20260709 13:19:25.245159 18440 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.248848 18476 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:25.256078 18428 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:25.272631 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:25.878335 18440 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:25.880528 18440 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:25.881301 15525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 661) took 2498 ms. Trace:
I20260709 13:19:25.881716 15525 rpcz_store.cc:276] 0709 13:19:23.382439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.384172 (+  1733us) service_pool.cc:224] Handling call
0709 13:19:23.384676 (+   504us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:23.672409 (+287733us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:23.672439 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:24.133452 (+461013us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b2c00000658. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce3304a11 00007f5ce3314372 00007f5ce4a46a01 00007f5ce4a0b43a 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2
0709 13:19:24.134359 (+   907us) log.cc:844] Serialized 582 byte log entry
0709 13:19:24.321481 (+187122us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:24.321519 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:24.321618 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:25.157283 (+835665us) spinlock_profiling.cc:243] Waited 5.16 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:19:25.881042 (+723759us) raft_consensus.cc:1685] finished
0709 13:19:25.881062 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:25.881212 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4895,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"spinlock_wait_cycles":65000960,"thread_start_us":466091,"threads_started":2,"wal-append.queue_time_us":264226}
I20260709 13:19:25.300556 18317 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:25.883581 18317 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:25.885736 18317 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:25.887987 18364 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:25.305326 18416 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.890903 18416 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:25.892863 18057 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:25.893548 18057 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:25.315927 18518 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:25.895156 18518 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:25.895704 18057 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:25.897678 18458 raft_consensus.cc:2955] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Committing config change with OpId 4.5: config changed from index 2 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:25.898994 16203 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:25.901216 18057 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:25.901767 18057 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:25.902150 18319 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:25.902750 18319 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:25.903950 18057 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:25.904551 18057 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:25.905046 18057 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:25.400790 18389 consensus_meta.cc:287] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.958s	user 0.011s	sys 0.002s
I20260709 13:19:25.908229 18389 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:25.552757 15457 debug-util.cc:398] Leaking SignalData structure 0x7b080003b220 after lost signal to thread 14429
I20260709 13:19:25.606655 18519 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:25.909343 18519 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:25.910434 18389 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:25.910876 18518 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:25.911502 18519 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:25.670354 18520 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:25.912218 18520 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:25.914376 18520 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:25.730430 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:25.762058 18457 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.791934 18523 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:25.807301 15597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 548) took 1370 ms. Trace:
W20260709 13:19:25.807528 18316 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.028s	sys 0.024s
I20260709 13:19:25.808034 18522 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:25.826676 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:25.851716 15594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 1694) took 1497 ms. Trace:
I20260709 13:19:25.862037 18318 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:25.915469 18057 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:25.919800 18523 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:25.920871 18057 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:25.921375 18057 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:25.922676 18523 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:25.921811 15597 rpcz_store.cc:276] 0709 13:19:24.436733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.436881 (+   148us) service_pool.cc:224] Handling call
0709 13:19:24.437431 (+   550us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:24.494290 (+ 56859us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:24.494324 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:25.311629 (+817305us) log.cc:844] Serialized 582 byte log entry
0709 13:19:25.804957 (+493328us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:25.804996 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:25.806843 (+  1847us) spinlock_profiling.cc:243] Waited 55.1 ms on lock 0x7b540025ab54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:25.806893 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:25.806942 (+    49us) raft_consensus.cc:1685] finished
0709 13:19:25.806954 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:25.807207 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27676,"raft.queue_time_us":1535,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":7844,"spinlock_wait_cycles":126748544,"thread_start_us":492914,"threads_started":1,"wal-append.queue_time_us":491214}
I20260709 13:19:25.923437 18457 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:25.923952 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:25.924574 18522 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:25.925310 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:25.926249 15594 rpcz_store.cc:276] 0709 13:19:24.355063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.361986 (+  6923us) service_pool.cc:224] Handling call
0709 13:19:24.367410 (+  5424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:24.510094 (+142684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:25.851639 (+1341545us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:25.928148 18057 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:25.928460 18522 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:25.929061 18316 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.029s	sys 0.025s
W20260709 13:19:25.909152 15457 debug-util.cc:398] Leaking SignalData structure 0x7b0800139100 after lost signal to thread 14432
I20260709 13:19:25.931612 18457 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:19:25.932070 15457 debug-util.cc:398] Leaking SignalData structure 0x7b080032fc40 after lost signal to thread 15460
W20260709 13:19:25.864835 15602 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53324 (request call id 1689) took 2478 ms (client timeout 2200 ms). Trace:
W20260709 13:19:25.932529 15602 rpcz_store.cc:269] 0709 13:19:23.386692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.391296 (+  4604us) service_pool.cc:224] Handling call
0709 13:19:25.864574 (+2473278us) spinlock_profiling.cc:243] Waited 698 ms on lock 0x7b540030e654. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:25.864743 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1605747968}
I20260709 13:19:25.878796 15595 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:24.512938 18474 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:25.933744 18474 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:25.930124 18318 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:25.937484 18318 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:25.940491 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:25.940388 18471 consensus_meta.cc:287] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 1.459s	user 0.017s	sys 0.002s
I20260709 13:19:25.943171 18474 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:25.943840 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:25.944213 18526 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:25.944764 18526 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:25.892988 18416 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:25.915867 15599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 1847) took 1415 ms. Trace:
I20260709 13:19:25.947566 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:25.949368 15525 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:25.942047 18471 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:25.905403 18319 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:25.321094 18516 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:25.907284 18056 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:25.951035 18516 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:25.952502 16231 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:25.953383 18516 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:25.953569 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 53114.3us
I20260709 13:19:25.959214 16204 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:25.959906 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:25.960178 18056 raft_consensus.cc:2804] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:25.960671 18056 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:25.961059 18056 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:25.961359 15526 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:25.962509 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:25.963188 15525 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:25.963994 17744 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:25.964505 17744 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:25.964984 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:25.965584 15670 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:25.966732 18492 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:25.967393 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:25.967243 18492 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:25.969413 18492 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:25.969223 18056 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:25.970427 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:25.960264 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:25.971112 15523 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:25.971958 18056 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:25.964502 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:25.975451 17744 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:19:25.977612 18499 log.cc:927] Time spent T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 1.582s	user 0.000s	sys 0.015s
I20260709 13:19:25.978144 15675 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
W20260709 13:19:25.978760 15675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49100 (request call id 1916) took 1662 ms (client timeout 2086 ms). Trace:
I20260709 13:19:25.948695 15599 rpcz_store.cc:276] 0709 13:19:24.500821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.501719 (+   898us) service_pool.cc:224] Handling call
0709 13:19:24.502107 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:25.913222 (+1411115us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:25.913253 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:25.913294 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:25.913322 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:25.915628 (+  2306us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400389ad4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:25.915670 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:25.915812 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3244897152}
W20260709 13:19:25.979106 15675 rpcz_store.cc:269] 0709 13:19:24.315981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.339180 (+ 23199us) service_pool.cc:224] Handling call
0709 13:19:25.960104 (+1620924us) spinlock_profiling.cc:243] Waited 8.91 ms on lock 0x7b5800012398. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99d10a0 00007f5ce99d0984 00007f5ce2f4b6d4 00007f5ce2f4b662
0709 13:19:25.978654 (+ 18550us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20502784}
I20260709 13:19:25.979924 18492 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:25.980429 18492 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:25.981408 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.981537 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:25.982198 15669 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:25.982601 18492 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:25.983099 16204 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:25.983814 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:25.984035 16204 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:25.985149 18319 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:25.984373 15669 raft_consensus.cc:2955] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:25.985546 18319 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:25.985936 18319 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:25.947647 18526 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:25.984378 15523 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:25.987524 18526 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:25.987914 18526 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:25.988268 18526 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:25.992300 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:25.995268 15523 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:25.996523 16204 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:25.997324 18057 raft_consensus.cc:2804] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:25.997872 18057 raft_consensus.cc:697] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:25.998799 18057 consensus_queue.cc:237] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.000618 18378 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:25.995160 18377 consensus_queue.cc:260] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.003937 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.006413 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.006597 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.008535 15675 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:26.009977 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.010527 16203 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:26.010020 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:26.011584 18331 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:26.012007 18331 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:26.012413 18331 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:26.012748 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.013417 16232 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:26.012990 15519 raft_consensus.cc:2955] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:26.015455 18439 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.015988 18439 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.018666 18439 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:26.022182 15603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 553) took 1567 ms. Trace:
I20260709 13:19:26.023365 16203 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:26.022591 15603 rpcz_store.cc:276] 0709 13:19:24.454547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.454751 (+   204us) service_pool.cc:224] Handling call
0709 13:19:24.455063 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:25.857871 (+1402808us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:25.857903 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:25.858453 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 13:19:25.923700 (+ 65247us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:25.923739 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.021776 (+ 98037us) spinlock_profiling.cc:243] Waited 55.6 ms on lock 0x7b540043c954. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:26.021823 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:26.021877 (+    54us) raft_consensus.cc:1685] finished
0709 13:19:26.021889 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:26.022119 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1346694,"raft.queue_time_us":1497039,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"spinlock_wait_cycles":127981952,"thread_start_us":64882,"threads_started":1,"wal-append.queue_time_us":64648}
I20260709 13:19:26.028273 16203 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:26.023520 18474 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:26.023077 18319 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.030139 17950 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:26.032887 17950 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:26.033303 17950 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:26.038631 15493 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.570s	user 0.035s	sys 0.011s
W20260709 13:19:26.039179 15493 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.571s	user 0.035s	sys 0.011s
W20260709 13:19:26.039513 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 1.57501e+06us
I20260709 13:19:26.040261 15493 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:26.048255 18457 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:26.048993 18457 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.031468 18440 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:26.051048 18440 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:26.051493 18440 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:26.055959 18378 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:26.057497 15493 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:26.058530 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 680) took 1655 ms. Trace:
I20260709 13:19:26.058952 15528 rpcz_store.cc:276] 0709 13:19:24.403057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.403226 (+   169us) service_pool.cc:224] Handling call
0709 13:19:24.403524 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:24.403850 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:24.403874 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:24.421440 (+ 17566us) log.cc:844] Serialized 26 byte log entry
0709 13:19:24.474067 (+ 52627us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:24.474103 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:24.474167 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:26.058333 (+1584166us) raft_consensus.cc:1685] finished
0709 13:19:26.058351 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:26.058475 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":535,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":11378,"thread_start_us":51220,"threads_started":1,"wal-append.queue_time_us":97033,"wal-append.run_cpu_time_us":1581,"wal-append.run_wall_time_us":1001616}
W20260709 13:19:26.060912 16233 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.592s	user 0.007s	sys 0.035s
W20260709 13:19:26.061378 16233 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.593s	user 0.007s	sys 0.035s
I20260709 13:19:26.062541 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.062666 18526 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:26.064359 16205 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.582s	user 0.010s	sys 0.009s
W20260709 13:19:26.064795 16205 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.582s	user 0.010s	sys 0.009s
W20260709 13:19:26.065137 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 1.58272e+06us
I20260709 13:19:26.066403 16205 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:26.067978 18473 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:26.065085 15493 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:26.068113 17950 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.058578 18378 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:26.072405 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:26.071749 18457 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:26.073179 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.079721 18440 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.080142 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.079535 15523 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:26.082054 18516 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:26.082219 18519 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:26.082630 18516 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:26.082778 18519 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:26.084216 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.084682 18516 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:26.084811 18519 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:26.085309 18364 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:26.085805 18364 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.085170 18416 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:26.086906 18416 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.087736 18364 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:26.073812 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:26.075387 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:26.083184 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.084936 15673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 1921) took 1751 ms. Trace:
I20260709 13:19:26.091406 15673 rpcz_store.cc:276] 0709 13:19:24.333130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.333627 (+   497us) service_pool.cc:224] Handling call
0709 13:19:24.333903 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:24.374537 (+ 40634us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:24.374565 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:24.374608 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:24.374653 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.084653 (+1710000us) spinlock_profiling.cc:243] Waited 40.4 ms on lock 0x7b540046ee54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:26.084702 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:26.084826 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92976512}
I20260709 13:19:26.094658 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.094821 16204 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:26.095938 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.095839 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:26.096674 15672 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:19:26.097602 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:26.098104 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.102608 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.103657 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.105139 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.094223 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:26.106649 15493 connection.cc:256] RPC call timeout handler was delayed by -1.3101870119999148s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1582s for a response
I20260709 13:19:26.107246 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.108363 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.095517 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:26.110821 15493 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:26.114776 18457 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:26.110936 15642 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:26.117660 15642 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:26.120165 18457 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:26.120837 18457 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:26.125129 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.127465 16234 raft_consensus.cc:1240] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:26.129098 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.130281 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:26.073221 18170 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:26.094455 18057 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.146235 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:26.087749 18473 raft_consensus.cc:697] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:26.077155 18167 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.131110 15523 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:26.076341 18439 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:26.102186 18519 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:26.078236 17744 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:26.134310 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.075469 17950 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:26.148124 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:26.151110 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:26.106292 18378 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:26.154816 15521 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:26.151400 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:26.129127 18074 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:26.151700 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:26.157152 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.157315 15520 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:26.152562 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.131994 18520 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:26.164707 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.164407 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.148336 18431 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:26.165848 15673 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:19:26.166589 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.155009 18416 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:26.157415 18167 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:26.162993 18378 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.153299 18057 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.151785 17744 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:26.191354 18527 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:26.192461 18526 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:26.175067 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.192670 18378 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:26.193202 16231 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:26.194214 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.151031 18439 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.195385 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.195605 18057 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:26.196511 18331 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.197489 18167 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:26.197749 16240 raft_consensus.cc:2393] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 6: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:26.174758 18473 consensus_queue.cc:237] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.176167 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.199024 16232 raft_consensus.cc:2393] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 7: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:26.200151 16232 raft_consensus.cc:1240] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:26.176851 18484 consensus_queue.cc:260] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.177783 18319 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:26.131059 18523 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:19:26.173357 18457 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.193598 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.204988 15670 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:26.192772 17744 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:26.209789 18331 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:26.210458 16234 raft_consensus.cc:1240] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:26.210983 18484 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:26.209880 16231 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:19:26.214318 16231 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:26.209645 16232 raft_consensus.cc:1240] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:26.219434 18457 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:26.212564 18439 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:26.239377 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:26.242697 17744 consensus_queue.cc:1059] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:19:26.245860 18365 consensus_queue.cc:1059] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:26.246585 17744 consensus_queue.cc:1243] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer dd1a14c3173d49dc9b55bb116e3783c9 log is divergent from this leader: its last log entry 7.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:19:26.247108 17744 raft_consensus.cc:3055] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:26.247431 17744 raft_consensus.cc:740] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:26.247862 18167 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:26.248245 17744 consensus_queue.cc:260] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.249485 17744 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:26.250761 16231 raft_consensus.cc:1240] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:26.248268 18167 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.252296 18167 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:26.255051 15668 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:19:26.255144 18365 raft_consensus.cc:3055] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:26.255656 18365 raft_consensus.cc:740] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:26.255685 18492 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.256604 18365 consensus_queue.cc:260] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.257313 18492 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.267735 16233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57258 (request call id 664) took 1975 ms. Trace:
I20260709 13:19:26.268230 16233 rpcz_store.cc:276] 0709 13:19:24.291915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.306397 (+ 14482us) service_pool.cc:224] Handling call
0709 13:19:24.312234 (+  5837us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:24.389972 (+ 77738us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:24.390006 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:24.390538 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 13:19:24.461592 (+ 71054us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:26.061775 (+1600183us) log.cc:844] Serialized 14 byte log entry
0709 13:19:26.159491 (+ 97716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.265816 (+106325us) spinlock_profiling.cc:243] Waited 32.6 ms on lock 0x7b5400240ed4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:26.265875 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:26.267385 (+  1510us) raft_consensus.cc:1685] finished
0709 13:19:26.267485 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:26.267667 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16355,"prepare.queue_time_us":111292,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":20753,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":176,"spinlock_wait_cycles":75049856,"thread_start_us":1740083,"threads_started":3,"wal-append.queue_time_us":202725,"wal-append.run_cpu_time_us":1600,"wal-append.run_wall_time_us":1001624}
W20260709 13:19:26.270965 18450 consensus_meta.cc:287] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 1.982s	user 0.020s	sys 0.005s
I20260709 13:19:26.271438 18450 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.272523 16238 raft_consensus.cc:1240] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:26.273576 18450 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:26.273963 16238 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53976 (request call id 624) took 1874 ms. Trace:
I20260709 13:19:26.274391 16238 rpcz_store.cc:276] 0709 13:19:24.399563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.402344 (+  2781us) service_pool.cc:224] Handling call
0709 13:19:24.402619 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:26.272894 (+1870275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.273717 (+   823us) spinlock_profiling.cc:243] Waited 1.87 s on lock 0x7b54007b02d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:26.273882 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4300079872}
I20260709 13:19:26.276235 18541 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:26.276539 18318 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:26.276803 18541 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.277307 18538 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:26.277680 18450 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:26.278239 18450 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.279080 18541 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:26.279600 18541 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:26.280045 18541 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.281111 18318 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.280344 18450 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:26.282315 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.282503 15598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 547) took 1854 ms. Trace:
I20260709 13:19:26.282994 16232 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:26.283244 18318 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:26.277820 18538 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.285861 18538 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:26.282895 15598 rpcz_store.cc:276] 0709 13:19:24.427707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.427842 (+   135us) service_pool.cc:224] Handling call
0709 13:19:24.428147 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:24.469891 (+ 41744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.282267 (+1812376us) spinlock_profiling.cc:243] Waited 41 ms on lock 0x7b5400439254. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:26.282442 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":94361984}
I20260709 13:19:26.286110 16234 raft_consensus.cc:1240] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:19:26.288055 18365 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:26.290081 16238 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:19:26.290856 16238 raft_consensus.cc:606] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:26.291172 16238 raft_consensus.cc:614] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:19:26.294795 15596 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.816s	user 0.016s	sys 0.026s
W20260709 13:19:26.295297 15596 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.816s	user 0.016s	sys 0.026s
I20260709 13:19:26.297158 18383 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.297719 18383 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.299826 18383 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:26.307042 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:26.307567 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:26.307720 15519 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:26.308151 15523 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:26.309079 18541 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:26.313266 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:26.314081 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.314476 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:26.315045 15520 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:26.318840 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 58002.5us
I20260709 13:19:26.321836 18048 consensus_queue.cc:1059] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:26.320874 16233 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:26.324375 15491 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:26.325815 16239 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:26.326191 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:26.325793 18383 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:26.327150 18383 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:26.327783 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:26.325822 15491 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:26.328168 18048 consensus_queue.cc:1059] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:26.329062 18290 raft_consensus.cc:3055] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:26.329246 18377 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:26.329479 18290 raft_consensus.cc:740] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:26.329526 15491 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:26.330556 18290 consensus_queue.cc:260] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.331668 18048 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:26.322748 18162 raft_consensus.cc:3055] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:26.332396 18162 raft_consensus.cc:740] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:26.333292 18162 consensus_queue.cc:260] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.334496 18162 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:26.327833 18383 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:26.331804 18290 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:26.339216 18484 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:26.339708 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.339890 18432 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:26.340445 18432 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.342550 18432 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:26.318517 18167 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.332041 18048 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:26.345176 15519 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:26.330971 15491 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:26.345692 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:26.352371 15642 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:26.354744 15491 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:26.355610 16204 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:26.357124 15520 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:19:26.357633 16235 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:26.370666 15523 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
W20260709 13:19:26.374235 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:26.376650 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.380666 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.379935 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.356621 18048 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:26.362437 18449 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:26.356783 17958 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:26.356519 18439 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:26.390924 15668 raft_consensus.cc:2410] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:19:26.361653 18508 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:26.378430 18318 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:26.339761 18484 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.393625 16204 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:26.403009 16234 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
W20260709 13:19:26.403421 15642 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.66:41593 timed out after 2.007s (SENT) [suppressed 5 similar messages]
I20260709 13:19:26.391633 18449 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:26.389415 18443 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:26.403630 16234 raft_consensus.cc:606] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:26.404314 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 94201.4us
I20260709 13:19:26.394686 18519 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:26.410311 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:26.412305 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:26.410019 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:26.414232 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:26.412281 16204 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:26.416550 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:26.421694 16204 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:26.422227 18316 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:26.422794 18316 raft_consensus.cc:697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:26.423637 18316 consensus_queue.cc:237] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.424922 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.409821 18449 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:26.426219 18441 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:26.426795 18441 raft_consensus.cc:697] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:26.427740 18441 consensus_queue.cc:237] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.432063 18162 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:26.443904 18440 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:26.445070 15493 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:26.445369 18385 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:26.446422 18470 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:26.446708 18179 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.447229 18179 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:26.454892 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:26.455628 15668 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:26.456233 15493 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:26.459283 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:19:26.460309 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 64243.3us
I20260709 13:19:26.409647 18443 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:26.409157 18508 raft_consensus.cc:697] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:26.463852 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:26.399462 18439 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.461694 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:26.462461 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:26.446774 18474 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:26.416189 18538 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:26.487825 15668 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:26.417215 18492 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:26.461472 15641 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:26.494436 15641 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:26.490630 15523 raft_consensus.cc:1275] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:19:26.482633 15670 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:26.498651 18162 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:26.499073 18167 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:26.500051 18474 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.491652 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.498728 18179 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:26.501382 15668 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:26.454258 18289 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:26.521581 18538 raft_consensus.cc:697] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:26.522549 18504 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:26.523201 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:26.523116 18504 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:26.523756 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.509236 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:26.524370 18508 consensus_queue.cc:237] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:26.525602 18289 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.526265 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.526870 16237 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:26.527696 18289 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:26.461601 17958 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:26.464768 18443 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:26.531915 15677 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:19:26.484834 18516 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.519094 18474 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:26.519344 18439 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:26.520393 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.521217 18492 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:26.524924 16234 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:26.461649 18421 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:26.509653 15641 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:26.462777 18523 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 13:19:26.523509 18471 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:26.560392 18546 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:19:26.540867 15675 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:26.541164 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.508095 15670 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:26.594018 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:26.543365 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.521288 18541 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:19:26.459365 18470 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:26.463460 18385 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:26.525146 18504 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:26.522598 18538 consensus_queue.cc:237] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.534461 15493 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:26.565199 15669 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
W20260709 13:19:26.571326 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 61765.1us
I20260709 13:19:26.606954 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.610153 15641 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:26.611264 15641 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:26.611830 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.511646 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.461727 18378 consensus_queue.cc:1059] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:19:26.575505 16203 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:26.537323 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.592412 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.592909 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.595408 15677 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
W20260709 13:19:26.614472 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 80109us
I20260709 13:19:26.615536 15493 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:26.571446 18421 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:26.603924 16234 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:26.577386 18492 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:26.598150 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.567059 18522 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.607023 15603 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:26.607643 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.638675 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.639346 15675 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:26.641079 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.596837 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.642326 15598 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:19:26.643146 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.617053 15596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 556) took 2142 ms. Trace:
I20260709 13:19:26.617511 15641 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:26.617688 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.613536 16203 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:26.621423 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:26.597375 18474 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:26.643906 15596 rpcz_store.cc:276] 0709 13:19:24.466028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.467162 (+  1134us) service_pool.cc:224] Handling call
0709 13:19:24.467393 (+   231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:24.467562 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:26.295651 (+1828089us) log.cc:844] Serialized 14 byte log entry
0709 13:19:26.607752 (+312101us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:26.607810 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:26.607869 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.607936 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:26.608063 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74999,"prepare.queue_time_us":113626,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":739,"raft.queue_time_us":147933,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"thread_start_us":2053301,"threads_started":3,"wal-append.queue_time_us":158192}
I20260709 13:19:26.647291 15641 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:26.649034 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.649803 15595 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:26.650858 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:26.652833 15568 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:26.653630 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.656922 15675 raft_consensus.cc:1275] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:19:26.607211 18479 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:26.597657 18471 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.663484 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.623024 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.620756 18453 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:26.624342 18377 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:26.649731 16203 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:26.571862 18516 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:26.667491 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.668150 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.623446 18421 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:26.680748 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:26.667268 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.620056 18365 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:26.686414 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:26.660892 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.624076 18504 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:26.661760 18484 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:26.747188 15596 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
W20260709 13:19:26.749518 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 68611us
I20260709 13:19:26.574442 18048 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:26.753086 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 103419us
I20260709 13:19:26.765039 16203 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:26.753767 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.754756 15601 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:26.766551 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.751266 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.767158 15597 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:19:26.754987 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.768086 15595 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:19:26.768524 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.768968 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.748337 15598 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:19:26.762105 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.658480 18522 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.769672 15598 raft_consensus.cc:606] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:26.622915 18167 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:26.770504 15598 raft_consensus.cc:614] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:26.771416 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:26.769120 15603 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:19:26.682137 15642 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:26.777750 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.733783 18549 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.736474 18377 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:19:26.776955 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 94908.6us
I20260709 13:19:26.781147 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.659783 18290 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:26.782636 15597 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:19:26.759996 18541 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:19:26.783434 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 99317.2us
I20260709 13:19:26.784133 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:26.665493 18481 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:26.785822 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.662997 18364 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:26.659948 18444 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:26.779978 18484 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:26.761054 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:26.857669 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.770053 18504 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.757169 18553 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:26.787873 18551 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:26.665340 18449 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:26.768277 18523 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:26.682111 18479 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.786465 18377 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:26.775540 18167 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.783677 18453 raft_consensus.cc:697] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:26.805896 18549 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.917578 18551 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:26.943666 18551 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:26.918190 18541 raft_consensus.cc:697] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:26.946791 18541 consensus_queue.cc:237] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.947562 18516 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:26.881424 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.882200 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.953485 18449 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:26.956118 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:26.957268 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.667979 18474 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.886723 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:26.889310 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 124829us
I20260709 13:19:26.961143 15566 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:26.876129 18550 log.cc:927] Time spent T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
W20260709 13:19:26.897363 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 256366us
I20260709 13:19:26.963382 15493 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:26.751880 15594 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:26.900614 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.966884 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.967509 15596 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:26.967948 15493 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:26.967649 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.968349 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.968999 15596 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:26.969229 15493 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:26.969417 18449 raft_consensus.cc:2804] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:26.969803 18449 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:26.970268 18449 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:26.902674 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 137700us
I20260709 13:19:26.970384 15493 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:26.970980 16203 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:26.971644 15603 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:26.662880 18473 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.111s
I20260709 13:19:26.968016 18289 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:19:26.906615 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 254857us
I20260709 13:19:26.973317 18289 raft_consensus.cc:697] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:26.911939 18162 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:26.907083 18458 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:26.910353 18290 raft_consensus.cc:697] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:26.919775 18553 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:26.912573 18444 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:26.930248 15600 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:19:26.933086 18560 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:26.926712 18048 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:26.938469 18453 consensus_queue.cc:237] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:26.949030 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.911641 18481 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:26.922578 18167 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:26.784485 15670 raft_consensus.cc:2393] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 8: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:19:26.908650 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 124598us
W20260709 13:19:26.907788 18383 consensus_meta.cc:287] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.000s
I20260709 13:19:26.969782 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.975383 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:26.969251 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:26.975641 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:26.972954 18549 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:26.985208 18559 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:26.971486 18474 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:27.211333 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:27.006027 18481 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:26.974974 18516 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:26.974268 18289 consensus_queue.cc:237] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.005259 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:26.976752 18378 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:27.011487 15595 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:27.241065 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 235880us
W20260709 13:19:27.234614 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 260085us
I20260709 13:19:27.012089 15598 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:27.241855 15642 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:27.242750 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:27.013585 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:27.205785 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 219104us
I20260709 13:19:27.210510 18560 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:26.976734 18162 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:27.252017 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:19:26.958899 18555 log.cc:927] Time spent T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:19:27.000279 18444 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:27.237085 18553 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:26.967906 18443 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.238164 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:27.235103 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 270413us
I20260709 13:19:27.249192 16205 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:27.233456 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:27.211894 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:27.206480 18559 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:27.271345 15601 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:27.250984 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:27.273626 15598 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:19:27.204674 18383 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:27.244415 18484 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:27.250290 18560 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:27.279630 15597 raft_consensus.cc:1240] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:19:27.249579 18365 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:27.278853 18559 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:27.255512 18162 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:27.252274 18516 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:27.253037 18440 consensus_queue.cc:1048] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:19:27.254907 18378 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:27.277786 15595 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:27.280911 15602 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:19:27.297216 15642 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:27.295374 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 319783us
I20260709 13:19:27.287861 16239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 992) took 3593 ms. Trace:
I20260709 13:19:27.311797 15602 raft_consensus.cc:606] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:27.308744 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 59628.3us
I20260709 13:19:27.312404 15493 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:27.312085 16239 rpcz_store.cc:276] 0709 13:19:23.694581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.694759 (+   178us) service_pool.cc:224] Handling call
0709 13:19:26.197969 (+2503210us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b580009ff98. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99e4fc0 00007f5ce99cf929 00007f5ce99cf3d2 00007f5ce2f46bd4
0709 13:19:26.325639 (+127670us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:26.326334 (+   695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.287634 (+961300us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b540068f450. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:27.287768 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":298360448}
I20260709 13:19:27.313851 15642 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:27.292076 18560 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:27.315037 16205 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:27.316802 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:27.254415 18290 consensus_queue.cc:237] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.318588 16205 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:27.323235 15493 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:27.325644 15642 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:27.326318 16205 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:27.310289 18473 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:27.335955 18365 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:27.319696 18492 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:27.322448 18432 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:27.322080 18316 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:27.347347 18535 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:27.347952 18535 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:27.348346 18535 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:27.347273 18562 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.143s
I20260709 13:19:27.350008 18432 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:27.350528 18432 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:27.353048 18383 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:27.343850 15424 catalog_manager.cc:5697] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 2 to 4, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 4 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:19:27.353735 18383 raft_consensus.cc:2764] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:19:27.333094 18458 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:27.356356 18458 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:27.356791 18458 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:27.359031 18535 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.361308 18535 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:27.363742 18377 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.329309 18484 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:27.309894 18516 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:27.325278 18571 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:27.381289 18449 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.397132 18377 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:27.399358 18516 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:27.388118 18421 consensus_meta.cc:287] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.628s	user 0.009s	sys 0.000s
I20260709 13:19:27.402926 18421 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.405097 18421 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:27.385313 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 80011.1us
I20260709 13:19:27.406159 15566 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:27.317819 18378 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.407874 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:27.342581 18443 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:27.332072 18563 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:27.409580 18563 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:27.410033 18563 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:27.412295 18579 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:27.412811 18579 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:27.413143 18579 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:27.385699 18484 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:27.408834 18378 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:27.438499 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 115329us
I20260709 13:19:27.439831 15493 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:27.440935 15493 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:27.346323 18551 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.447932 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:27.455474 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.456050 15493 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:27.456212 18481 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.430230 18484 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:27.458261 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:27.447512 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:27.462602 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:27.447429 18421 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:27.462805 16236 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:27.463675 16232 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:27.464700 16237 raft_consensus.cc:1240] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:27.465523 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:27.466157 15520 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:27.469193 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:27.470976 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:27.475143 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:27.476104 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:27.456885 18378 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:27.480233 15520 raft_consensus.cc:1275] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:19:27.481721 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:27.483247 15526 raft_consensus.cc:1240] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:27.469679 18582 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:27.497534 16236 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:27.469211 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:27.509631 16232 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:19:27.523680 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:27.525094 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:27.525949 15522 raft_consensus.cc:2368] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7.
I20260709 13:19:27.527854 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:27.530220 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:27.537189 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 68067us
I20260709 13:19:27.540452 18570 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:27.540977 18570 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:27.541333 18570 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:27.542291 18162 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.543373 15677 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:27.545496 18162 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:27.551038 18481 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:27.551348 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:27.555546 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:27.558614 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:27.559995 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:27.563027 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:27.563627 15668 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:27.565135 18560 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.586321 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:27.596719 16205 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:27.598737 15527 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:27.599669 16205 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:27.600034 15668 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:27.600461 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:27.601146 15527 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:27.604427 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:27.598143 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:27.605988 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:27.536412 18527 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:27.602157 16203 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:27.607411 16205 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:27.606463 18383 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.611579 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:27.611770 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:27.552610 18290 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:27.568331 15594 raft_consensus.cc:1275] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:19:27.549351 18563 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.616731 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:27.610064 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:27.606786 18450 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.606812 18481 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:27.620672 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:27.622313 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:27.623087 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:27.625109 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:27.626757 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:27.628849 15526 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:19:27.632784 18571 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:27.614670 18535 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:27.558713 18453 consensus_queue.cc:1048] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:19:27.643735 18564 raft_consensus.cc:2804] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:27.644052 18167 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.645437 18317 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:27.647289 18167 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:27.643633 18571 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:27.626327 18458 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.618989 18541 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:27.651222 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:27.656111 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:27.656064 15642 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:27.648759 18559 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.641331 18377 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:27.650354 15522 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:27.649685 15524 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:27.617837 18551 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:27.652872 15519 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:19:27.628553 18554 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:27.672863 15457 debug-util.cc:398] Leaking SignalData structure 0x7b080003b4a0 after lost signal to thread 15552
W20260709 13:19:27.674455 15457 debug-util.cc:398] Leaking SignalData structure 0x7b080003b4e0 after lost signal to thread 15553
W20260709 13:19:27.651226 18591 log.cc:927] Time spent T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:19:27.532565 18582 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:27.614897 18432 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:27.658988 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:19:27.675534 15457 debug-util.cc:398] Leaking SignalData structure 0x7b08003f1a40 after lost signal to thread 15627
I20260709 13:19:27.673275 15519 pending_rounds.cc:85] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 7
I20260709 13:19:27.616609 18383 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.644241 18564 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
W20260709 13:19:27.683203 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 48691.1us
I20260709 13:19:27.682271 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:27.682628 15519 pending_rounds.cc:107] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:19:27.610302 18579 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:27.620916 18527 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.687832 15603 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:57344
I20260709 13:19:27.688331 18579 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:27.682897 15457 debug-util.cc:398] Leaking SignalData structure 0x7b08003f1a80 after lost signal to thread 15628
I20260709 13:19:27.688568 18527 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:27.689687 15457 debug-util.cc:398] Leaking SignalData structure 0x7b08003ce7a0 after lost signal to thread 15701
W20260709 13:19:27.690001 15457 debug-util.cc:398] Leaking SignalData structure 0x7b08001c2d00 after lost signal to thread 15702
I20260709 13:19:27.690562 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:27.690819 18563 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:27.691082 15457 debug-util.cc:398] Leaking SignalData structure 0x7b08003db3c0 after lost signal to thread 16266
W20260709 13:19:27.691375 15457 debug-util.cc:398] Leaking SignalData structure 0x7b08003db500 after lost signal to thread 16267
I20260709 13:19:27.691939 18560 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:27.691546 15597 raft_consensus.cc:2955] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:27.653832 18458 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:27.685773 15519 raft_consensus.cc:2889] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:27.658995 18541 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:27.684716 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:27.699227 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:27.708058 15519 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:27.692042 15457 debug-util.cc:398] Leaking SignalData structure 0x7b0800392820 after lost signal to thread 18419
I20260709 13:19:27.703231 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:27.688449 15603 raft_consensus.cc:606] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:27.670603 18535 raft_consensus.cc:697] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:19:27.709134 15457 debug-util.cc:398] Leaking SignalData structure 0x7b0800392920 after lost signal to thread 18430
I20260709 13:19:27.651010 18571 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:27.709450 15603 raft_consensus.cc:614] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:27.720608 15521 raft_consensus.cc:1240] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:27.722090 15521 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:27.624281 18450 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.725879 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:27.726644 15672 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
W20260709 13:19:27.709875 15457 debug-util.cc:398] Leaking SignalData structure 0x7b0800382180 after lost signal to thread 18480
I20260709 13:19:27.644086 18573 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:27.727770 15491 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:27.731665 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:27.733008 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:27.685855 18564 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:27.684252 18562 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:27.734345 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:19:27.734877 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 56466.5us
I20260709 13:19:27.737891 15668 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:27.719129 18541 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:27.694610 18557 raft_consensus.cc:3055] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:27.685276 18551 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:27.743305 18551 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.739792 16232 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:27.724599 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:27.745873 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:27.740877 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:19:27.752260 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:27.752962 15672 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:27.757593 16232 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:27.682955 18377 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.734282 18571 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:27.760874 15668 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:19:27.764163 16205 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:27.742113 18557 raft_consensus.cc:740] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:27.745379 18562 raft_consensus.cc:697] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:27.771395 15519 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:27.760725 18571 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:27.789937 15601 raft_consensus.cc:3055] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:27.790447 15601 raft_consensus.cc:740] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:27.683624 18554 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.759402 18481 consensus_queue.cc:1059] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:19:27.791275 15601 consensus_queue.cc:260] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.814393 15601 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:27.737540 18573 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.810082 18597 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
W20260709 13:19:27.812985 18552 log.cc:927] Time spent T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 1.032s	user 0.008s	sys 0.005s
I20260709 13:19:27.761704 18444 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:27.819200 18559 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:27.798847 18453 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:27.798659 18571 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:27.822916 15603 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:27.828855 18570 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:27.823568 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 59421.1us
W20260709 13:19:27.831928 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.840667 15527 raft_consensus.cc:1240] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:27.785836 18557 consensus_queue.cc:260] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.777551 18316 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:19:27.787727 18484 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:27.845259 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 57891.9us
I20260709 13:19:27.845713 18551 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:27.863716 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:27.864419 15522 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:27.864737 18522 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:27.866108 18432 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:27.868235 18377 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:27.871032 15601 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:27.796942 18535 consensus_queue.cc:237] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.876968 15673 raft_consensus.cc:1240] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:27.872597 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:27.879001 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:27.862519 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:27.863729 18554 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:27.881966 18383 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:27.872179 18458 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:19:27.897217 18582 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:27.903822 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.894953 15594 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:27.905527 18484 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:27.924281 18570 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:27.875805 18562 consensus_queue.cc:237] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.932937 18470 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:27.934813 16238 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1003) took 1257 ms. Trace:
I20260709 13:19:27.935245 16238 rpcz_store.cc:276] 0709 13:19:26.677854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.678358 (+   504us) service_pool.cc:224] Handling call
0709 13:19:26.681325 (+  2967us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:26.908701 (+227376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:26.908730 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:26.908771 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:26.908799 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.934571 (+1025772us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b54005370d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:27.934620 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:27.934754 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":522455808}
I20260709 13:19:27.943624 15670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57710 (request call id 700) took 1482 ms. Trace:
I20260709 13:19:27.944057 15670 rpcz_store.cc:276] 0709 13:19:26.461328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.606692 (+145364us) service_pool.cc:224] Handling call
0709 13:19:27.943410 (+1336718us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b540042b154. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:27.943558 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":400725248}
I20260709 13:19:27.918608 18444 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:27.945545 15675 raft_consensus.cc:1240] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:27.951629 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:27.952906 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:27.954869 15670 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
W20260709 13:19:27.961169 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.963950 18573 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:27.965972 15672 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:27.968246 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:27.970539 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:27.971979 15672 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:27.962316 18471 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:27.985517 15670 raft_consensus.cc:1240] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:27.991326 15672 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:27.993500 18508 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:27.994151 18508 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.996193 18508 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:27.965221 18571 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.999979 18571 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:27.964948 18541 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:28.002524 15675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 1928) took 1388 ms. Trace:
I20260709 13:19:28.019199 15675 rpcz_store.cc:276] 0709 13:19:26.613659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.659933 (+ 46274us) service_pool.cc:224] Handling call
0709 13:19:27.945373 (+1285440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:27.945985 (+   612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.002345 (+ 56360us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b540042b150. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:28.002470 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2955668864}
I20260709 13:19:28.003168 15669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57710 (request call id 699) took 1545 ms. Trace:
I20260709 13:19:28.022092 15669 rpcz_store.cc:276] 0709 13:19:26.461015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.461475 (+   460us) service_pool.cc:224] Handling call
0709 13:19:28.003016 (+1541541us) spinlock_profiling.cc:243] Waited 45.4 ms on lock 0x7b540021dc54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:28.003114 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104411008}
I20260709 13:19:28.023576 15491 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:28.024623 18479 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:28.003970 15594 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:28.027266 18378 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:28.027825 18378 raft_consensus.cc:697] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
W20260709 13:19:28.004982 18608 log.cc:927] Time spent T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:19:28.006644 18573 consensus_queue.cc:1048] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:28.027120 15491 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:28.031719 18519 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:28.031852 18481 consensus_queue.cc:1059] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:28.035164 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:28.035571 18331 raft_consensus.cc:3055] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:28.035985 18331 raft_consensus.cc:740] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:28.036754 18331 consensus_queue.cc:260] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:28.007642 18449 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:27.983302 18564 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:27.981691 16233 raft_consensus.cc:1240] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:28.028709 18378 consensus_queue.cc:237] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:28.015377 15599 raft_consensus.cc:1240] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:28.041184 16233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1001) took 1371 ms. Trace:
I20260709 13:19:28.017335 18289 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.041556 16233 rpcz_store.cc:276] 0709 13:19:26.670190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.680468 (+ 10278us) service_pool.cc:224] Handling call
0709 13:19:27.981269 (+1300801us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b580009ff98. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99e4fc0 00007f5ce99cf929 00007f5ce99cf3d2 00007f5ce2f46bd4
0709 13:19:27.981490 (+   221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:28.040911 (+ 59421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.041125 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7728512}
I20260709 13:19:28.042276 18289 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:28.043104 18581 log.cc:927] Time spent T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.650s	user 0.002s	sys 0.000s
I20260709 13:19:28.018029 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:28.044133 15602 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:28.040385 18564 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:19:28.040845 18583 log.cc:927] Time spent T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.649s	user 0.003s	sys 0.000s
I20260709 13:19:28.039767 18331 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:28.028714 15594 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:19:28.051640 15671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 698) took 1594 ms. Trace:
I20260709 13:19:28.082733 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:28.058516 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:28.074142 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:28.077875 15424 catalog_manager.cc:5697] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 4 to 6, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 6 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:19:28.078980 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:28.086788 15672 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:28.083407 15671 rpcz_store.cc:276] 0709 13:19:26.457676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.459964 (+  2288us) service_pool.cc:224] Handling call
0709 13:19:26.460213 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:26.528926 (+ 68713us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:26.528955 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:26.528993 (+    38us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:26.529020 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.051387 (+1522367us) spinlock_profiling.cc:243] Waited 68.5 ms on lock 0x7b540009bfd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:28.051436 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:28.051574 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":157630208}
I20260709 13:19:28.028632 18541 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:28.095796 15674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 692) took 1651 ms. Trace:
I20260709 13:19:28.104835 15674 rpcz_store.cc:276] 0709 13:19:26.444392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.446360 (+  1968us) service_pool.cc:224] Handling call
0709 13:19:28.095109 (+1648749us) spinlock_profiling.cc:243] Waited 9.89 ms on lock 0x7b5800012398. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99e4fc0 00007f5ce99cf929 00007f5ce99cf3d2 00007f5ce2f46bd4
0709 13:19:28.095319 (+   210us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:28.095490 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:28.095511 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:28.095549 (+    38us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:28.095575 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.095628 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:28.095738 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22738688}
I20260709 13:19:28.207865 18289 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:28.160758 15672 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:19:28.159740 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:28.214288 15670 raft_consensus.cc:1240] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 4. Current term is 5. Ops: []
W20260709 13:19:28.165313 15525 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1763) took 1751 ms (client timeout 1522 ms). Trace:
I20260709 13:19:28.221470 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:28.223340 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:28.224752 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:28.226516 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:28.227083 15674 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:28.225778 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:28.228127 15672 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 5: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:28.228600 15671 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:28.216331 16232 raft_consensus.cc:1240] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:28.217103 18516 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:28.230891 18516 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:28.230831 18504 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:28.230120 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:28.232788 16203 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:28.233038 18516 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:28.233891 15599 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 6: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:28.115094 16235 raft_consensus.cc:1240] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 9. Current term is 10. Ops: []
W20260709 13:19:28.235405 15676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 1791) took 2133 ms (client timeout 1836 ms). Trace:
W20260709 13:19:28.221580 15525 rpcz_store.cc:269] 0709 13:19:26.413845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.413972 (+   127us) service_pool.cc:224] Handling call
0709 13:19:28.165055 (+1751083us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b540029d1d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:28.165235 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3286872320}
I20260709 13:19:28.154906 15594 raft_consensus.cc:1275] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:19:28.237900 15525 raft_consensus.cc:1240] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:28.238294 15567 leader_election.cc:400] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7.
I20260709 13:19:28.238651 15567 leader_election.cc:403] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:19:28.234328 15641 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:28.239424 15519 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:28.233956 16203 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:28.240841 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:28.241868 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:19:28.236003 15676 rpcz_store.cc:269] 0709 13:19:26.086580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.086674 (+    94us) service_pool.cc:224] Handling call
0709 13:19:28.159580 (+2072906us) spinlock_profiling.cc:243] Waited 9.21 ms on lock 0x7b5800012398. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99d10a0 00007f5ce99d0984 00007f5ce2f4b6d4 00007f5ce2f4b662
0709 13:19:28.219458 (+ 59878us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21185792}
I20260709 13:19:28.243782 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:28.244048 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:28.244525 15669 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 6: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:28.245379 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:28.245723 15671 raft_consensus.cc:1240] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:28.246850 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:28.247076 15673 raft_consensus.cc:1240] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:28.247253 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:28.247880 15599 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:19:28.250012 15641 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:28.250654 15568 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:28.251216 15567 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:28.251149 15641 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:28.250689 15674 raft_consensus.cc:1240] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:28.254232 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:28.254813 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:28.202116 18473 raft_consensus.cc:3055] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:28.259969 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:28.257627 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:28.258327 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:28.249225 16231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1006) took 1559 ms. Trace:
I20260709 13:19:28.123554 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:28.216499 18450 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:28.129637 18557 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:28.278128 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 44260.5us
I20260709 13:19:28.279351 16203 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:28.279644 17744 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.277752 15597 raft_consensus.cc:1240] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:28.280133 17744 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:28.280012 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 63396.2us
I20260709 13:19:28.283942 15493 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:28.285252 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:28.289500 17744 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:28.264830 18564 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:28.267472 15567 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:28.270215 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:28.273867 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:28.240505 18474 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.291590 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:28.271679 18074 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:28.259997 18473 raft_consensus.cc:740] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:28.273025 18365 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:28.274130 18445 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:28.286013 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:28.313678 17744 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.300419 15567 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:28.270967 16235 raft_consensus.cc:1240] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:28.314229 17744 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:28.312420 15568 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:28.283105 18492 consensus_queue.cc:1059] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:19:28.287940 18571 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:28.282721 18568 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:28.286341 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:28.285362 18289 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:28.285660 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:28.289984 18440 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:28.277602 18551 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7.
I20260709 13:19:28.268076 16231 rpcz_store.cc:276] 0709 13:19:26.684443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.687752 (+  3309us) service_pool.cc:224] Handling call
0709 13:19:28.052563 (+1364811us) spinlock_profiling.cc:243] Waited 4.29 ms on lock 0x7b580009ff98. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99e4fc0 00007f5ce99cf929 00007f5ce99cf3d2 00007f5ce2f46bd4
0709 13:19:28.071218 (+ 18655us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:28.071463 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:28.071489 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:28.071535 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:28.071563 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.071623 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:28.243327 (+171704us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9862528}
I20260709 13:19:28.457909 18385 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
W20260709 13:19:28.462224 18613 log.cc:927] Time spent T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:19:28.457815 16231 raft_consensus.cc:1240] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 4. Current term is 5. Ops: []
W20260709 13:19:28.465148 18612 log.cc:927] Time spent T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 13:19:28.470129 15674 raft_consensus.cc:1240] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:28.477612 18580 raft_consensus.cc:3055] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:28.478204 18580 raft_consensus.cc:740] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:28.479235 18580 consensus_queue.cc:260] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:28.480593 18580 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:28.482869 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:28.390669 18131 consensus_queue.cc:1059] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:19:28.396723 18504 consensus_queue.cc:1059] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 13:19:28.446954 18473 consensus_queue.cc:260] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:28.496076 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 159079us
I20260709 13:19:28.494773 15491 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:19:28.501684 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.373234 18575 consensus_queue.cc:1059] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:19:28.398166 15669 raft_consensus.cc:1240] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:28.286562 15493 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:28.297200 18318 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:28.311434 18472 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:19:28.392093 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:28.506067 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 219566us
I20260709 13:19:28.337344 18582 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:28.287652 18177 consensus_queue.cc:1059] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:19:28.384702 15568 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:28.312582 18474 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:28.284992 18596 consensus_queue.cc:1048] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:19:28.370761 17744 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:28.401834 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 85190us
W20260709 13:19:28.403384 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.342059 18611 consensus_queue.cc:1059] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:28.372635 15602 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:19:28.316040 18445 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:28.369515 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 69166.3us
I20260709 13:19:28.519454 15567 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:28.382408 18470 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:28.506529 18559 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:28.314373 18477 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:28.506848 16232 raft_consensus.cc:2393] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:28.508111 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:28.290910 18457 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.096s
I20260709 13:19:28.509405 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:28.509654 15674 raft_consensus.cc:1240] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:28.465373 18440 raft_consensus.cc:697] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:28.520435 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:28.502923 15600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53324 (request call id 1698) took 1833 ms (client timeout 2111 ms). Trace:
I20260709 13:19:28.564107 15676 raft_consensus.cc:2393] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 7: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:28.544123 15567 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:28.510814 18516 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:19:28.545179 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 227163us
W20260709 13:19:28.562618 15600 rpcz_store.cc:269] 0709 13:19:26.666911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.667021 (+   110us) service_pool.cc:224] Handling call
0709 13:19:28.499282 (+1832261us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54002ad0d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:28.499975 (+   693us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":415312256}
I20260709 13:19:28.551095 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:28.552256 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:28.553361 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:28.553711 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:28.570001 15528 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:28.570429 15525 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:28.556815 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:28.558066 15526 raft_consensus.cc:1240] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:28.538440 18445 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:28.513885 18492 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.566334 15567 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:28.573822 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:28.583665 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:28.542024 18179 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:28.575577 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:28.555459 18535 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.586038 15523 raft_consensus.cc:2393] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:28.576941 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:28.577489 15493 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:28.577905 15603 raft_consensus.cc:1240] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: []
W20260709 13:19:28.574836 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 54792.6us
I20260709 13:19:28.582018 16205 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:28.583665 15567 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:28.518743 18563 consensus_queue.cc:1059] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:19:28.561100 18582 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:28.610821 15528 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:28.585189 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:28.607235 15567 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:28.559826 18318 raft_consensus.cc:697] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:28.620512 18318 consensus_queue.cc:237] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:28.581285 18516 raft_consensus.cc:697] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:28.561192 18421 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:28.623739 18421 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:28.555591 18439 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:19:28.626139 18548 consensus_queue.cc:1059] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:19:28.626675 18479 raft_consensus.cc:3055] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:28.571796 18131 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:28.627100 18479 raft_consensus.cc:740] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:28.627627 18131 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:28.628280 18131 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:28.627909 18479 consensus_queue.cc:260] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:28.559990 18551 consensus_queue.cc:1059] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:19:28.629088 18479 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:28.605845 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:28.575079 18559 raft_consensus.cc:697] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:28.563498 18595 consensus_queue.cc:1059] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:19:28.611779 18378 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:19:28.571300 18519 consensus_queue.cc:1059] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:19:28.617976 18431 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.581713 18177 consensus_queue.cc:1059] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:19:28.571461 18504 consensus_queue.cc:1059] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:19:28.622848 18516 consensus_queue.cc:237] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:28.587533 18617 log.cc:927] Time spent T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.291s	user 0.000s	sys 0.001s
I20260709 13:19:28.625857 18421 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:28.636801 18472 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:28.656013 18431 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:28.686872 18564 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:28.687407 18559 consensus_queue.cc:237] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:28.611217 18535 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:28.690884 18601 raft_consensus.cc:3055] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:28.691331 18601 raft_consensus.cc:740] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:28.691573 18535 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:28.692214 18601 consensus_queue.cc:260] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:28.693367 18601 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:28.677851 18440 consensus_queue.cc:237] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:28.699910 18431 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:28.606354 18492 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:28.702080 18492 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:28.585556 18568 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:28.610663 18582 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:28.703286 18582 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:28.704607 18608 log.cc:927] Time spent T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:19:28.636831 18563 raft_consensus.cc:3055] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:28.612005 18317 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:19:28.705749 18563 raft_consensus.cc:740] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:28.705893 18553 raft_consensus.cc:3055] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:28.706374 18553 raft_consensus.cc:740] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:28.637488 18439 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:28.706787 18563 consensus_queue.cc:260] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:28.707160 18553 consensus_queue.cc:260] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:28.707821 18439 raft_consensus.cc:697] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:28.708257 18563 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:28.709002 18541 raft_consensus.cc:3055] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:28.708701 18439 consensus_queue.cc:237] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:28.709381 18541 raft_consensus.cc:740] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:28.710176 18541 consensus_queue.cc:260] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:28.711323 18541 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:28.712673 18611 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:28.715647 18554 raft_consensus.cc:3055] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:28.716022 18554 raft_consensus.cc:740] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:28.716801 18554 consensus_queue.cc:260] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:28.718837 18470 raft_consensus.cc:3055] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:28.719254 18470 raft_consensus.cc:740] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:28.720013 18470 consensus_queue.cc:260] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:28.721170 18470 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:28.725647 18474 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:28.717792 18554 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:28.734824 18553 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:28.756798 18582 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:28.759253 18582 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:28.712328 18519 raft_consensus.cc:3055] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:28.760880 18519 raft_consensus.cc:740] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:28.608201 15522 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 4: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:28.581753 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:28.799321 15528 raft_consensus.cc:1240] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:28.586946 15568 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:28.615656 15598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 567) took 1730 ms. Trace:
W20260709 13:19:29.001135 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 358268us
I20260709 13:19:29.000922 15598 rpcz_store.cc:276] 0709 13:19:26.885041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.275077 (+390036us) service_pool.cc:224] Handling call
0709 13:19:28.499413 (+1224336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:28.499618 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:28.499644 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:28.499687 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:28.499726 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.499784 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:28.615455 (+115671us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54002ad0d0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:28.615582 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2815177472}
W20260709 13:19:28.816808 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:29.003458 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 435097us
I20260709 13:19:29.003229 18519 consensus_queue.cc:260] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:29.004678 18519 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:28.830847 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:28.635264 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 53345.1us
I20260709 13:19:28.856950 18445 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.008298 16205 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:29.008685 18445 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:28.859850 16234 raft_consensus.cc:1240] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:29.009302 15642 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:28.860795 16238 raft_consensus.cc:1240] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:28.868173 15673 raft_consensus.cc:1240] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 13:19:28.874162 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.886991 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:28.888572 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:28.954581 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:28.999634 18508 consensus_queue.cc:1059] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:19:28.815135 15602 raft_consensus.cc:1240] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:29.017340 18557 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:29.021872 18445 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:29.022696 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.022984 18508 consensus_queue.cc:1243] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer 3ef186ec12554a1d87814af0a0ace9a4 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:19:29.023803 18508 raft_consensus.cc:3055] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:29.024221 18508 raft_consensus.cc:740] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
W20260709 13:19:29.025970 18620 log.cc:927] Time spent T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.345s	user 0.000s	sys 0.002s
I20260709 13:19:29.028915 18534 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.030167 18289 raft_consensus.cc:3055] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:29.029866 18534 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.030686 17950 raft_consensus.cc:3055] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:29.031159 18587 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:29.032616 15523 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:29.038436 15526 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:29.047052 15673 raft_consensus.cc:1275] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:19:29.048730 18628 consensus_queue.cc:1048] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:29.049728 15599 raft_consensus.cc:1275] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:19:29.050036 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:29.051441 15597 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:29.051970 16236 raft_consensus.cc:1240] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:29.051800 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:29.052520 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:29.053740 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:29.054108 15669 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:29.048779 16234 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 5. Current term is 6. Ops: []
W20260709 13:19:29.066713 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 400675us
W20260709 13:19:29.068235 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 481228us
I20260709 13:19:29.051872 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:29.058527 17950 raft_consensus.cc:740] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:29.030714 18289 raft_consensus.cc:740] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:29.086784 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:29.025859 18508 consensus_queue.cc:260] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.088135 18444 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:29.088644 18508 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:29.077198 18631 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:29.090066 18421 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:29.091010 18421 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.091434 18473 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:29.092237 18534 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:29.092883 18534 consensus_queue.cc:1059] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:29.072144 18628 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:29.109368 18378 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:29.118834 18289 consensus_queue.cc:260] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.120893 18289 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:29.125622 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:29.126343 15525 raft_consensus.cc:1240] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:29.071569 18557 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:29.128252 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:29.128273 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:29.130538 18557 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:29.134055 15603 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:29.134805 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:29.136631 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:19:29.142086 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 402759us
I20260709 13:19:29.142637 18440 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:29.143369 18440 raft_consensus.cc:697] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:29.145889 15519 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:19:29.150519 15424 catalog_manager.cc:5697] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:29.153687 15424 catalog_manager.cc:5697] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 3 to 4, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 4 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:29.155509 18421 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:29.156075 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:29.158597 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:29.161592 18074 raft_consensus.cc:3055] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:29.144912 18440 consensus_queue.cc:237] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.162102 18074 raft_consensus.cc:740] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
W20260709 13:19:29.163275 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.163611 18628 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:29.166664 16239 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:29.110420 18378 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:29.165954 18611 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:29.165223 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 59031.9us
I20260709 13:19:29.169493 16235 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:29.173334 16236 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:19:29.172509 17950 consensus_queue.cc:260] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.172283 18611 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:29.171130 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 412987us
I20260709 13:19:29.189538 15524 raft_consensus.cc:2410] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:19:29.177480 15522 raft_consensus.cc:1275] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:19:29.163407 18074 consensus_queue.cc:260] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.112088 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:19:29.183686 18479 consensus_meta.cc:287] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.554s	user 0.016s	sys 0.009s
I20260709 13:19:29.207551 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:29.179826 18378 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:29.178184 18421 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:29.180203 18167 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.173894 18587 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:29.178138 17744 consensus_queue.cc:1059] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:19:29.222476 15603 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:29.172343 18450 consensus_queue.cc:1059] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:19:29.225509 15493 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:29.235292 15676 raft_consensus.cc:1240] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:29.229575 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:29.240017 15670 raft_consensus.cc:1275] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:29.236536 18421 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.242897 15671 raft_consensus.cc:1275] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:29.244110 18421 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:29.246357 18074 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:29.253480 16232 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:19:29.250085 15568 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:29.238117 18378 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:29.265712 15566 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:29.270856 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:29.272269 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:29.274547 18566 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:29.274921 18318 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:29.276026 18566 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:29.277098 17950 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:29.278332 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:29.279402 18474 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:29.280113 18474 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:29.281348 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 55099us
I20260709 13:19:29.282958 15568 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:29.282958 18553 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:29.284071 18474 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:29.284641 18474 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:29.284541 18556 consensus_queue.cc:1059] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:29.267894 18167 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:29.279562 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.261448 18421 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:29.288084 15522 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:29.291615 18611 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:29.295714 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 287828us
I20260709 13:19:29.287048 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:29.234431 18587 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:29.267525 18508 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:29.285704 18556 raft_consensus.cc:3055] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:29.285604 18378 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:29.312958 18177 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:29.314606 18474 consensus_queue.cc:1059] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:19:29.322124 18508 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:29.324545 18508 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:29.326730 18474 consensus_queue.cc:1059] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:29.328382 16238 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1010) took 1062 ms. Trace:
I20260709 13:19:29.328892 16238 rpcz_store.cc:276] 0709 13:19:28.265970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.266760 (+   790us) service_pool.cc:224] Handling call
0709 13:19:28.272844 (+  6084us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:29.014295 (+741451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.328067 (+313772us) spinlock_profiling.cc:243] Waited 587 ms on lock 0x7b5400640ad4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:29.328296 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1350875392}
I20260709 13:19:29.303711 18628 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:29.301071 18566 consensus_queue.cc:1059] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:19:29.311902 18556 raft_consensus.cc:740] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:29.328766 18474 raft_consensus.cc:3055] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:29.339773 18474 raft_consensus.cc:740] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:29.339946 18602 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:19:29.340865 18474 consensus_queue.cc:260] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:29.341526 18179 raft_consensus.cc:3055] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:29.342001 18179 raft_consensus.cc:740] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:29.343173 18179 consensus_queue.cc:260] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.344272 18179 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:29.345088 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.347415 15597 raft_consensus.cc:1240] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: []
W20260709 13:19:29.354405 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 177869us
I20260709 13:19:29.379390 16238 raft_consensus.cc:1240] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:29.379828 15602 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:19:29.380484 18647 log.cc:927] Time spent T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:19:29.380813 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:29.381625 16236 raft_consensus.cc:2393] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 6: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:29.385605 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:29.386408 15594 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:29.387554 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:29.379854 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.390455 18474 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:29.400226 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.403120 15493 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:29.404239 18516 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:29.374356 18602 raft_consensus.cc:3055] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:29.407617 18602 raft_consensus.cc:740] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:29.408665 18602 consensus_queue.cc:260] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.410378 18587 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:29.413684 18170 consensus_queue.cc:1059] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:29.376781 18378 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:29.414808 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:29.414100 18290 raft_consensus.cc:3055] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:29.415884 18290 raft_consensus.cc:740] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:29.416905 18290 consensus_queue.cc:260] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.418450 18290 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:29.419553 15566 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:29.420895 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 96824.2us
W20260709 13:19:29.422842 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 44502us
I20260709 13:19:29.424139 18566 consensus_queue.cc:1048] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:29.424806 16236 raft_consensus.cc:1275] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:19:29.426703 18481 raft_consensus.cc:3055] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:29.427107 18481 raft_consensus.cc:740] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:29.428015 18481 consensus_queue.cc:260] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.429054 18481 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:29.430160 18553 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:29.433065 18534 raft_consensus.cc:3055] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:29.433504 18534 raft_consensus.cc:740] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:29.434170 18170 raft_consensus.cc:3055] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:29.434629 18170 raft_consensus.cc:740] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:29.434579 18534 consensus_queue.cc:260] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.436249 18534 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:29.435854 18170 consensus_queue.cc:260] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:29.437216 18170 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:29.443024 15424 catalog_manager.cc:5697] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 2 to 3, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 3 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:19:29.444423 18167 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:29.444664 15424 catalog_manager.cc:5697] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 3 to 5, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 5 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:19:29.458142 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.462018 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:29.462793 16232 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:29.469092 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:29.474475 15673 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:29.474502 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:29.480677 15669 raft_consensus.cc:1240] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:29.483361 15674 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:29.368574 18556 consensus_queue.cc:260] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.491030 18556 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:29.503108 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:29.515414 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.540161 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:29.540555 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:29.538789 16205 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:29.501988 18481 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:19:29.564846 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:29.544284 15424 catalog_manager.cc:5697] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:19:29.582835 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:29.584431 16232 raft_consensus.cc:1275] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:29.580879 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:29.582203 15568 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:29.586898 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:29.575086 18602 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:29.596906 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 191199us
I20260709 13:19:29.598361 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:29.605373 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:29.605969 15677 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:29.606104 16239 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:29.607725 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:29.609615 16238 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:19:29.610319 18627 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:29.610939 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:29.610993 18627 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.611862 15603 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:29.614342 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:19:29.614908 15426 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58802 (request call id 258) took 26087 ms (client timeout 15000 ms). Trace:
I20260709 13:19:29.615195 18290 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:29.615612 18290 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:29.616019 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:29.616462 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:29.616616 18627 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:29.615995 18290 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:29.626508 15642 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:29.626655 15677 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:29.627533 18573 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:29.630726 18492 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:29.589160 15424 catalog_manager.cc:5697] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:19:29.610803 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 106960us
W20260709 13:19:29.635936 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.588407 18566 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:29.584110 18167 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:19:29.654045 18553 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:29.658527 18553 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.660776 18553 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:29.662456 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:29.663226 16238 raft_consensus.cc:2393] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 8: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:29.670951 16204 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
W20260709 13:19:29.635066 15426 rpcz_store.cc:269] 0709 13:19:03.527258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.540123 (+ 12865us) service_pool.cc:224] Handling call
0709 13:19:29.614671 (+26074548us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:21.662986 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:21.714463 (+ 51477us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:21.714506 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:19:21.714525 (+    19us) tablet.cc:662] Decoding operations
0709 13:19:21.782044 (+ 67519us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:21.782111 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:21.782143 (+    32us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:19:21.783210 (+  1067us) tablet.cc:701] Row locks acquired
0709 13:19:21.783226 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:19:21.783379 (+   153us) write_op.cc:270] Start()
0709 13:19:21.783508 (+   129us) write_op.cc:276] Timestamp: P: 1783603161783331 usec, L: 0
0709 13:19:21.783527 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:19:21.784237 (+   710us) log.cc:844] Serialized 5854 byte log entry
0709 13:19:22.347715 (+563478us) op_driver.cc:464] REPLICATION: finished
0709 13:19:22.380487 (+ 32772us) write_op.cc:301] APPLY: starting
0709 13:19:22.380566 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:22.380847 (+   281us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:22.380969 (+   122us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:22.451772 (+ 70803us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:22.493164 (+ 41392us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=15
0709 13:19:22.493210 (+    46us) write_op.cc:312] APPLY: finished
0709 13:19:22.493539 (+   329us) log.cc:844] Serialized 107 byte log entry
0709 13:19:22.493710 (+   171us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:22.493869 (+   159us) write_op.cc:454] Released schema lock
0709 13:19:22.493950 (+    81us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:22.600398 (+106448us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:22.600444 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":82458,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":32528,"apply.run_cpu_time_us":9597,"apply.run_wall_time_us":194033,"num_ops":15,"prepare.queue_time_us":151578,"prepare.run_cpu_time_us":23439,"prepare.run_wall_time_us":271899,"raft.queue_time_us":311099,"raft.run_cpu_time_us":2731,"raft.run_wall_time_us":4707,"replication_time_us":564048,"thread_start_us":544963,"threads_started":4,"wal-append.queue_time_us":188434,"wal-append.run_cpu_time_us":17201,"wal-append.run_wall_time_us":1531141}]]}
I20260709 13:19:29.657820 15493 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:29.676573 15424 catalog_manager.cc:5697] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:19:29.680325 18566 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.681993 15491 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:29.682282 15424 catalog_manager.cc:5697] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:29.684569 15424 catalog_manager.cc:5697] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } }
I20260709 13:19:29.687803 16205 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:29.690583 18508 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:29.690963 15597 raft_consensus.cc:1275] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:29.691514 15493 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:29.694492 18628 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:29.695910 18628 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.697943 18481 raft_consensus.cc:2804] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:29.698491 18481 raft_consensus.cc:697] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:29.699478 18481 consensus_queue.cc:237] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.700542 15424 catalog_manager.cc:5697] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 2 to 3, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 3 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
W20260709 13:19:29.702011 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:29.702378 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 108013us
I20260709 13:19:29.702960 18378 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:29.703357 18378 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:29.703735 18378 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:29.707019 18492 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.707571 18492 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.707906 18628 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:29.702698 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.682196 18541 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:19:29.664738 18290 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.681725 18629 consensus_queue.cc:1048] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:19:29.746374 15522 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 LEADER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:29.725544 18508 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:29.749909 15602 raft_consensus.cc:1275] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:29.727536 18479 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:29.729784 15424 catalog_manager.cc:5697] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 2 to 4, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66), now has a pending config: VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65), VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66), VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67), VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 4 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: NON_VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: true } } } pending_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:29.727850 18628 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:29.749593 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:29.754344 17744 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:29.754920 17744 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:29.757071 18604 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:29.753155 15424 catalog_manager.cc:5697] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 1 to 4, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 4 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:29.720811 18167 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:19:29.757203 17744 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:29.758265 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:29.682005 18566 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.773311 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:29.774554 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 83144.3us
I20260709 13:19:29.747174 18573 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:29.757531 18604 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:29.752662 18479 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:29.774698 15424 catalog_manager.cc:5697] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 1 to 4, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 4 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:19:29.782366 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 296749us
W20260709 13:19:29.802454 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.795567 15566 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:29.752668 18628 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:29.820128 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.792219 18604 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:29.796278 18074 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:29.847385 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:29.848156 15675 raft_consensus.cc:2393] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 8: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:29.849022 18378 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:29.833570 16205 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:29.851907 18378 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:29.839231 16236 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:29.853039 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:29.807034 18587 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.856171 18587 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.856972 18074 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:29.857481 18074 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:29.827250 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 497216us
I20260709 13:19:29.829458 18474 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:29.803843 18479 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:29.834484 15493 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:29.836141 18566 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:29.837172 18629 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:29.837481 18440 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:29.840291 15672 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:19:29.841463 18450 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:29.843227 18492 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:29.847169 18628 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:29.823963 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 53244.3us
I20260709 13:19:29.854142 15642 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:29.855008 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:29.807379 15424 catalog_manager.cc:5697] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:19:29.860136 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.861656 15675 raft_consensus.cc:2393] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 5: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:29.847282 18290 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:29.872141 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:29.854894 18573 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:19:29.891443 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.895538 17744 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:29.892227 15600 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:29.927215 17744 raft_consensus.cc:697] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:29.887849 15424 catalog_manager.cc:5697] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:29.870105 18474 raft_consensus.cc:697] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:29.945408 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:29.946193 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:29.948822 16239 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:29.949123 18611 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.006s	sys 0.019s
W20260709 13:19:29.949589 18611 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.006s	sys 0.019s
I20260709 13:19:29.950354 16236 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:29.951820 15642 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:29.958170 16239 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:19:29.946406 15424 catalog_manager.cc:5697] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:19:29.961575 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 70326.5us
W20260709 13:19:29.965173 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:29.967694 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.889329 18450 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.971385 18290 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:30.007194 17744 consensus_queue.cc:237] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:29.998723 18534 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:29.994026 18573 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:30.021670 15567 leader_election.cc:336] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.024s (SENT)
W20260709 13:19:30.019086 15674 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.009s	sys 0.020s
W20260709 13:19:30.027210 15674 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.010s	sys 0.020s
I20260709 13:19:30.014250 18534 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:30.029323 15567 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:30.019871 18628 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:30.035473 15641 leader_election.cc:336] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.522s (SENT)
I20260709 13:19:30.045215 16205 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:30.030210 18519 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:30.025941 18541 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:19:30.038764 18534 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:30.035285 18290 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.058899 18179 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:30.122109 15493 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:30.122635 15602 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:19:30.029453 18474 consensus_queue.cc:237] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:30.125303 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 112811us
W20260709 13:19:30.127657 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 104824us
I20260709 13:19:30.066923 18587 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:30.121073 18556 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:30.130522 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:30.144551 15595 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:30.142045 18179 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:30.147159 18179 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:30.146517 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 97569.7us
I20260709 13:19:30.118227 18628 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:30.150909 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 151038us
W20260709 13:19:30.151041 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 199306us
I20260709 13:19:30.137161 18508 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:19:30.157626 15567 leader_election.cc:336] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.086s (SENT)
I20260709 13:19:30.138469 18595 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:30.166285 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 112132us
I20260709 13:19:30.130754 18519 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.150358 18601 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:30.155334 18602 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:30.176057 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:30.178900 15672 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:30.160460 18378 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:30.160782 18508 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:30.191332 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:30.192727 18654 log.cc:927] Time spent T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
W20260709 13:19:30.195719 18663 log.cc:927] Time spent T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:19:30.172511 18556 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.184131 18602 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:30.214233 18650 log.cc:927] Time spent T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
W20260709 13:19:30.214848 18657 log.cc:927] Time spent T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
W20260709 13:19:30.215267 18655 log.cc:927] Time spent T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 13:19:30.236465 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.216348 18556 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:30.205403 18508 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:30.243523 15596 raft_consensus.cc:1275] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:19:30.230901 18659 log.cc:927] Time spent T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.209s	user 0.001s	sys 0.001s
I20260709 13:19:30.214187 18450 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:30.250969 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 107155us
I20260709 13:19:30.213886 18604 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.256937 18519 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:30.224792 18179 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:30.262387 18604 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:30.178121 18595 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:30.272454 18662 log.cc:927] Time spent T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:19:30.274547 18628 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:30.272506 18508 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.230687 18602 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:30.290309 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 58555.7us
I20260709 13:19:30.318008 15567 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:30.301219 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 125779us
I20260709 13:19:30.306226 18290 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:30.310465 15668 raft_consensus.cc:1240] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:30.312579 15603 raft_consensus.cc:1240] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:19:30.316604 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:30.322275 18676 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:19:30.323330 18508 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:30.325290 18595 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:30.326119 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:30.326962 16235 raft_consensus.cc:2393] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 9: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:30.328459 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:30.329101 15527 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:30.331681 18479 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.334376 15525 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:30.334522 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:30.335417 15603 raft_consensus.cc:2393] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 5: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:30.335632 18479 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:30.337420 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:30.338090 15603 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:30.342228 15595 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:19:30.343403 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:30.347422 15598 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:19:30.344347 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.346760 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:30.350860 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:30.351029 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:30.346468 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:30.320097 18179 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:30.356962 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:30.354195 15527 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:30.350773 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:30.361675 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:30.349548 15493 connection.cc:256] RPC call timeout handler was delayed by -1.3023426199999903s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15000000000000002s for a response
W20260709 13:19:30.362349 15493 connection.cc:256] RPC call timeout handler was delayed by -1.0287426200000027s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1804s for a response
I20260709 13:19:30.362479 16232 raft_consensus.cc:2368] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 5. Current term is 6.
I20260709 13:19:30.363744 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:30.364962 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:30.348459 18573 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:30.352056 18170 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:30.337304 18074 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.376722 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:30.377414 15595 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:30.370592 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:30.358812 18628 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.360729 18508 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:30.382329 18587 consensus_queue.cc:1059] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:30.384196 15493 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:30.387351 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:30.390220 16205 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:30.370888 18519 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.394292 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:30.394739 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:30.402290 18669 log.cc:927] Time spent T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.230s	user 0.000s	sys 0.002s
I20260709 13:19:30.403328 18378 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:30.404047 18690 log.cc:927] Time spent T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:19:30.412855 15669 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:30.423961 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:30.414587 16204 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:30.428295 15525 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:30.441545 15526 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:19:30.408345 18170 raft_consensus.cc:697] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:30.412001 18378 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:30.443418 15519 raft_consensus.cc:3055] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:30.424381 18541 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:19:30.443953 15519 raft_consensus.cc:740] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:30.452278 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:30.455487 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 213990us
W20260709 13:19:30.460207 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 62804.4us
I20260709 13:19:30.415226 18519 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.426748 18628 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:30.432968 18508 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:30.470127 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.467724 15519 consensus_queue.cc:260] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.479350 18056 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:30.479923 18056 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:30.480289 18056 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:30.473697 18652 log.cc:927] Time spent T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.312s	user 0.003s	sys 0.002s
I20260709 13:19:30.492904 18508 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:30.493557 18508 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:30.494710 15519 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:30.471333 18688 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:30.495755 18688 raft_consensus.cc:697] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:30.496824 18688 consensus_queue.cc:237] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:30.499542 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.503134 15519 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:30.504534 18074 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:30.505427 18170 consensus_queue.cc:237] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.507822 18519 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:30.510622 18628 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:30.523093 15568 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:30.526924 15525 raft_consensus.cc:1240] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 6. Current term is 7. Ops: []
W20260709 13:19:30.530704 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 146628us
I20260709 13:19:30.471266 15642 leader_election.cc:400] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 5. Current term is 6.
I20260709 13:19:30.531694 15642 leader_election.cc:403] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:30.535055 18566 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:30.535215 18684 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:30.536382 18601 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:30.516476 18602 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:30.586092 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 124804us
I20260709 13:19:30.546736 18289 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:30.582173 18628 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.594236 18628 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:30.595397 18439 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.595973 18439 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.597958 18439 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:30.598001 18602 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:30.592370 18056 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.589690 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:30.600394 18056 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:30.603346 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:30.603986 15597 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:30.603962 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:30.613003 15597 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:30.615757 16205 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:30.616479 16232 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:19:30.618069 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:30.618479 15596 raft_consensus.cc:1240] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:30.622231 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:30.623308 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:30.624156 15596 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:30.625149 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:30.626374 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:30.626924 16239 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:30.623005 18553 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:30.583738 18378 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.630952 18553 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.616376 18167 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:19:30.631467 15493 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:30.628871 18179 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.633351 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:19:30.631381 18696 log.cc:927] Time spent T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.055s	user 0.000s	sys 0.003s
I20260709 13:19:30.633857 18179 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:30.635414 18689 consensus_queue.cc:1059] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:30.636088 18179 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:30.637207 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:30.639438 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:30.629523 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:30.644838 15676 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:19:30.598196 18678 consensus_queue.cc:1059] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:19:30.630414 18611 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:30.640223 18439 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.617707 18534 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.683604 18534 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:30.660050 18290 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:30.684326 18290 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:30.684695 15673 raft_consensus.cc:2393] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 6: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:30.685149 18439 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.666146 18628 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:30.684741 18290 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:30.666565 18678 raft_consensus.cc:3055] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:30.690191 18678 raft_consensus.cc:740] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:30.690793 18289 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.691293 18678 consensus_queue.cc:260] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.668675 18611 raft_consensus.cc:697] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:30.664043 18074 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:19:30.670805 15493 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:30.710654 18611 consensus_queue.cc:237] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.711447 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:30.712013 15493 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:30.712965 18702 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:30.713563 18702 raft_consensus.cc:697] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:30.712060 16239 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:19:30.714027 18678 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:30.714699 18680 consensus_queue.cc:1059] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:30.713577 18628 raft_consensus.cc:2804] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:30.672729 18553 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:30.668601 18519 raft_consensus.cc:3055] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:30.715534 18628 raft_consensus.cc:697] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:30.715770 18519 raft_consensus.cc:740] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:30.714625 18702 consensus_queue.cc:237] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.716769 18519 consensus_queue.cc:260] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.705397 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:30.705895 15601 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:30.717427 18628 consensus_queue.cc:237] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.718410 15491 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:30.718675 16238 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
W20260709 13:19:30.706766 18705 log.cc:927] Time spent T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.115s	user 0.003s	sys 0.000s
I20260709 13:19:30.706641 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:30.720598 16236 raft_consensus.cc:2393] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:30.721120 18508 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:30.721536 18508 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:30.713253 15493 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:30.722430 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:30.705843 18706 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:30.691110 18439 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:30.709484 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:30.719482 18289 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.729521 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:30.737632 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:30.740245 16239 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:30.721889 18508 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:30.752014 15642 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:30.770591 15642 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:30.750147 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:30.769701 15602 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
W20260709 13:19:30.773782 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 60617.9us
I20260709 13:19:30.746259 18689 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:30.777963 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:30.774605 15493 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:30.784409 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:30.783188 18519 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:30.786495 18378 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:30.788270 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:30.792533 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:30.794708 16238 raft_consensus.cc:1240] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:30.797374 18378 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:30.799055 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:30.800029 18289 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:30.776597 18573 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:30.800812 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:30.801462 15672 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:30.801455 16234 raft_consensus.cc:1240] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:30.806566 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:30.807255 15669 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:19:30.810957 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:30.811630 15673 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:30.773016 18587 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:30.771275 18318 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:19:30.817365 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:30.789788 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:30.822825 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:30.825582 18701 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:30.836717 15522 raft_consensus.cc:1240] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:30.783923 18290 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.848692 18440 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.849316 18440 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.856460 18056 raft_consensus.cc:3055] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:30.857012 18056 raft_consensus.cc:740] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:30.858081 18701 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:30.858691 18701 raft_consensus.cc:697] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:30.859625 18701 consensus_queue.cc:237] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.860244 16203 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:30.862186 18611 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:30.862587 18611 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:30.862958 18611 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:30.865783 15523 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:30.823740 18587 raft_consensus.cc:697] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:30.866900 18587 consensus_queue.cc:237] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.823549 18573 raft_consensus.cc:697] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:30.868518 18573 consensus_queue.cc:237] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:30.870505 18698 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:30.871007 18698 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:30.871352 18698 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:30.873359 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.846011 16231 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:30.898098 15568 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:30.898324 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 75556.2us
I20260709 13:19:30.900413 15523 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:30.908371 15491 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:30.919286 15668 raft_consensus.cc:1275] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:19:30.897048 18179 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.897418 17950 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.964640 15521 raft_consensus.cc:1240] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:30.962265 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:30.965559 18712 log.cc:927] Time spent T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:19:30.970963 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 59750us
W20260709 13:19:30.971159 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 57334.5us
I20260709 13:19:30.915386 18553 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:30.912480 18708 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.870548 18611 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.005141 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:31.009719 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.980856 18553 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:30.969200 18179 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:30.979544 17950 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.024978 18553 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:31.029801 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:31.031548 18707 log.cc:927] Time spent T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 13:19:31.058704 18056 consensus_queue.cc:260] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:31.032354 18720 log.cc:927] Time spent T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:19:31.033701 15595 raft_consensus.cc:1240] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:31.046934 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:31.047614 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.049280 18179 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:31.074400 15599 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:31.069578 15600 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:31.032684 18719 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:31.060515 18056 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:31.084690 15521 raft_consensus.cc:1275] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:31.013367 18473 consensus_queue.cc:1059] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 13:19:31.077410 18629 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.112118 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:31.119778 15600 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:19:31.115245 18729 consensus_queue.cc:1059] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:31.116777 18519 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.123628 18519 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:31.124636 18729 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:31.126693 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:31.127346 15519 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:31.127933 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:31.128307 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:31.003388 18708 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.129559 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:31.130151 15521 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:31.083578 18719 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.139986 15526 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:31.140730 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:31.144433 18611 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:31.139662 18629 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.145255 18179 consensus_queue.cc:1048] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:31.147322 18629 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:31.147543 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:31.148077 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:31.149175 18708 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:31.151134 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 63149.9us
I20260709 13:19:31.149788 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:31.155257 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:31.160025 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:31.057879 18553 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:31.167099 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 173588us
I20260709 13:19:31.141106 18519 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:31.175694 16205 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:31.179548 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:31.180413 15602 raft_consensus.cc:2393] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 5: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:31.150697 18290 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:31.151441 18719 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:31.189273 15523 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:31.194988 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:31.195750 15523 raft_consensus.cc:2368] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 5. Current term is 6.
I20260709 13:19:31.198050 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:31.192963 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:31.201134 15601 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:31.211565 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:19:31.213158 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 48378.7us
I20260709 13:19:31.219516 15523 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:31.202836 18681 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:31.226188 16204 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:31.207866 18680 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.245977 18440 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:31.249830 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:31.250691 15519 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 5: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:31.225525 18681 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.232129 18481 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.238633 18289 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:31.269169 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.280121 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:31.275091 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:31.282287 15670 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:31.278102 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 67123us
I20260709 13:19:31.302912 15600 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:31.304487 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:31.306833 15526 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:31.307987 15519 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:19:31.308847 16205 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:31.310842 15677 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 6. Current term is 7. Ops: []
W20260709 13:19:31.261826 18508 consensus_meta.cc:287] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.512s	user 0.006s	sys 0.000s
I20260709 13:19:31.322286 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:31.322576 15599 raft_consensus.cc:1240] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:31.261833 18678 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:31.266920 18681 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:31.353597 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 44842us
I20260709 13:19:31.359444 15523 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:31.266263 18553 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:31.354664 18508 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:31.366614 18508 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:31.274962 18698 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.367408 15528 raft_consensus.cc:2368] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8.
I20260709 13:19:31.368856 18698 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:31.312474 18721 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:31.239692 18680 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.267369 18481 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.270846 18289 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.378350 18721 raft_consensus.cc:697] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:31.395385 18721 consensus_queue.cc:237] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.380867 18680 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:31.373842 18719 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:31.398249 15524 raft_consensus.cc:2368] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7.
I20260709 13:19:31.399667 18719 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:31.388496 18167 consensus_meta.cc:287] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.746s	user 0.007s	sys 0.003s
I20260709 13:19:31.400657 18167 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 5. Current term is 6.
I20260709 13:19:31.401036 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:31.402773 18167 consensus_queue.cc:1059] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:31.392923 18289 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:31.393086 18678 raft_consensus.cc:697] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:19:31.407054 18727 log.cc:927] Time spent T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.232s	user 0.002s	sys 0.000s
I20260709 13:19:31.408111 18678 consensus_queue.cc:237] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.385681 18681 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.363871 18473 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:31.413853 15676 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:31.418955 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:31.429105 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:31.429806 16231 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:31.430276 15603 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 6: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:31.431202 15669 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
W20260709 13:19:31.420246 15567 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.595s (SENT) [suppressed 20 similar messages]
I20260709 13:19:31.433329 15568 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:31.436931 15670 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
W20260709 13:19:31.435483 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 209395us
I20260709 13:19:31.440075 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:31.436474 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:31.440418 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:31.442384 16239 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:31.415535 18473 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.451508 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:31.452255 15672 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:31.441787 15599 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:31.465576 16205 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:31.454303 18473 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:31.453728 18629 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.470373 18289 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:31.476581 16239 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:31.495664 18481 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:31.498210 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:31.499071 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.490168 18289 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.562041 15523 raft_consensus.cc:3055] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:31.562345 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:31.484510 18629 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.562485 15523 raft_consensus.cc:740] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:31.499497 18736 raft_consensus.cc:2804] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:31.559307 18289 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:31.578567 15526 raft_consensus.cc:1240] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:31.603694 15672 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:19:31.623032 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:31.624755 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:31.623834 15520 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:31.574687 18736 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.640043 15523 consensus_queue.cc:260] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.631392 18736 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:31.662664 18735 log.cc:927] Time spent T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 13:19:31.663601 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.681686 15523 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:31.676549 18719 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:31.684548 18719 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:31.686805 18719 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:31.698556 18629 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:31.759169 15523 raft_consensus.cc:1275] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:19:31.759625 15523 pending_rounds.cc:85] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 6
I20260709 13:19:31.788582 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:31.791625 16231 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:31.759874 15523 pending_rounds.cc:107] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:19:31.793512 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:31.795539 15523 raft_consensus.cc:2889] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:31.795778 15521 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:31.800781 16240 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:19:31.812047 15567 leader_election.cc:336] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.559s (SENT)
I20260709 13:19:31.812552 15567 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:31.814520 15521 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:31.814972 18177 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:31.816884 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:31.817526 15674 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:31.820889 16231 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
W20260709 13:19:31.830179 18739 log.cc:927] Time spent T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.284s	user 0.008s	sys 0.000s
I20260709 13:19:31.822909 18289 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:31.840018 18741 log.cc:927] Time spent T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.209s	user 0.000s	sys 0.000s
I20260709 13:19:31.842710 18289 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:31.849341 18473 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.851488 18473 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:31.858930 18736 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.861245 18736 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:31.868914 17950 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:31.871003 18492 raft_consensus.cc:3055] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:31.871508 18492 raft_consensus.cc:740] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:31.872489 18492 consensus_queue.cc:260] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.873911 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:31.874547 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 83511.7us
I20260709 13:19:31.874055 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:31.878069 18748 consensus_queue.cc:1059] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:31.878222 18573 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:31.880098 15526 raft_consensus.cc:1240] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:31.879978 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:31.880667 15521 raft_consensus.cc:2376] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8. Re-sending same reply.
I20260709 13:19:31.881712 15523 raft_consensus.cc:1240] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:31.882978 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:31.883854 15521 raft_consensus.cc:2393] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:31.889568 15674 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:19:31.890905 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:31.892127 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:31.893121 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:31.891031 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:31.895087 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:31.895246 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:31.896520 18601 raft_consensus.cc:3055] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:31.897099 18601 raft_consensus.cc:740] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:31.897300 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:31.897838 16231 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:31.900200 18726 log.cc:927] Time spent T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.766s	user 0.004s	sys 0.009s
I20260709 13:19:31.901965 16234 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:31.896544 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:31.903613 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:31.904567 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:31.907727 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:31.908315 15595 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:31.909166 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:31.910531 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:31.911103 15603 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:19:31.913393 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:31.909742 15601 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:19:31.914170 15521 raft_consensus.cc:2368] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7.
I20260709 13:19:31.915000 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:31.915532 15521 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:31.917564 15594 raft_consensus.cc:1275] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:19:31.919457 18290 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:31.922796 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:31.923400 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:31.924091 15528 raft_consensus.cc:2368] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7.
I20260709 13:19:31.924906 15601 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:31.927994 16231 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:19:31.929323 15603 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:31.929888 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:31.932333 16205 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:31.933846 18688 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:19:31.927862 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.934242 18688 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.934640 18688 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:31.935094 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:31.935957 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.936803 15491 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:31.936954 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:31.937498 18719 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:31.937574 15599 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:19:31.938098 18719 raft_consensus.cc:697] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:31.939122 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:31.938927 18719 consensus_queue.cc:237] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:31.939324 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:31.939966 15599 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:19:31.941721 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:31.942737 18601 consensus_queue.cc:260] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.944103 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:31.944137 18601 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:31.946175 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:31.944430 18684 consensus_queue.cc:1175] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" responder_term: 7 status { last_received { term: 7 index: 8 } last_committed_idx: 8 last_received_current_leader { term: 7 index: 8 } }
I20260709 13:19:31.947559 18318 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:31.948055 18318 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.924832 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:31.952126 18318 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:31.953054 16203 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:31.953366 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:31.953980 18744 consensus_queue.cc:1059] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:31.954397 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:31.954938 15601 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:31.955116 18744 raft_consensus.cc:3055] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:31.955523 18744 raft_consensus.cc:740] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:31.956430 18744 consensus_queue.cc:260] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:31.955173 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:31.957283 18736 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:31.957448 15673 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:31.957592 18744 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:31.957885 18736 raft_consensus.cc:697] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:31.958619 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:31.959383 15671 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 6: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:31.959038 18736 consensus_queue.cc:237] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.963850 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:31.970160 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:31.971295 18318 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:31.971796 16205 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:31.972024 18678 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:31.964128 15642 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:31.974145 17744 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:31.974489 16205 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:31.974648 17744 raft_consensus.cc:697] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:31.974046 18541 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:31.975551 18541 raft_consensus.cc:697] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:31.975417 17744 consensus_queue.cc:237] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.976490 18541 consensus_queue.cc:237] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.976979 18748 raft_consensus.cc:2804] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:31.977535 18748 raft_consensus.cc:697] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:31.978368 18748 consensus_queue.cc:237] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.980707 18439 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:31.981295 18439 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.983805 18439 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:31.984692 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:31.985724 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:31.937618 15491 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:31.988948 18628 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:31.990581 18177 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:19:31.990711 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 53163.3us
I20260709 13:19:31.991170 18177 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.992053 16235 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:19:31.992067 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:31.992990 15595 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 8: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:31.993306 18177 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:31.967633 18688 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:31.996721 15595 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:19:31.995908 15567 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:31.999230 18733 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:32.000300 16236 raft_consensus.cc:1240] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:32.004088 15567 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:32.005442 18177 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:32.006211 15595 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:32.006486 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:32.007839 15567 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:32.010056 15601 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:19:32.013361 16235 raft_consensus.cc:1240] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:31.953986 15599 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:31.923611 15523 raft_consensus.cc:2393] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:31.966601 15594 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:31.967900 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:31.968128 15603 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:31.968487 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:31.969857 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:32.041990 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:32.042582 15568 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:32.043141 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:32.043798 15568 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:32.048185 15521 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:32.051674 18289 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:32.052369 18289 raft_consensus.cc:697] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:32.057807 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:32.053359 18289 consensus_queue.cc:237] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:32.041069 15641 leader_election.cc:336] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.545s (SENT)
I20260709 13:19:32.039981 15567 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:32.088387 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 45215.1us
I20260709 13:19:32.087777 15641 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:32.094521 15493 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:32.099012 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 59494us
I20260709 13:19:32.100416 15526 raft_consensus.cc:1275] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:19:32.106067 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:32.106459 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:32.107959 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 59113.4us
W20260709 13:19:32.111419 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.119033 15525 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:19:32.120072 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 60980.6us
I20260709 13:19:32.121593 16203 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:32.122888 16203 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:32.107026 16237 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:32.125783 15642 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:32.088176 18553 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:19:32.137789 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 96742.3us
I20260709 13:19:32.141158 16239 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:19:32.144196 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:32.144243 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:32.144526 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:32.146885 15670 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:32.148964 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:32.151556 15524 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:32.163882 18771 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:32.087585 18736 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.123s
I20260709 13:19:32.138314 18573 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:32.137069 15603 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:19:32.165653 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 125754us
I20260709 13:19:32.129487 18602 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:32.183651 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 139926us
W20260709 13:19:32.189495 15567 leader_election.cc:336] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.505s (SENT)
I20260709 13:19:32.191910 15670 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:32.150565 18290 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:32.144549 18628 raft_consensus.cc:697] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:32.149578 18611 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:32.199816 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:32.212193 15603 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:19:32.209591 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:32.217506 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1796) took 1170 ms. Trace:
I20260709 13:19:32.223824 15603 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:19:32.218384 15522 rpcz_store.cc:276] 0709 13:19:31.046705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.214606 (+1167901us) service_pool.cc:224] Handling call
0709 13:19:32.214928 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:32.217116 (+  2188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:32.217146 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:32.217231 (+    85us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:32.217262 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.217326 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:32.217444 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:32.165601 18745 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:19:32.229425 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 90298.3us
W20260709 13:19:32.233230 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:32.230464 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 116796us
W20260709 13:19:32.231385 15493 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.235996 15493 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:32.243278 18775 consensus_queue.cc:1048] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:32.245004 18775 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:32.250187 15493 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:32.251730 15493 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:32.215442 18611 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:32.213984 18290 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:32.213356 18688 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:32.260313 15566 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:32.218289 18733 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:19:32.275573 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 227528us
I20260709 13:19:32.213003 18602 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:32.281462 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:32.282141 15603 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:32.258949 18611 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:32.266405 18290 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:32.285622 18628 consensus_queue.cc:237] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:32.289502 15603 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:19:32.278286 18775 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:32.297204 18779 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:32.298075 18629 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:32.275871 18719 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:32.303100 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:32.272807 15493 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:32.315306 15493 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:32.316684 15594 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:32.283058 18771 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:19:32.320294 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 167410us
I20260709 13:19:32.284823 18553 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:32.327059 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.329831 15493 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:19:32.342197 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 87720.8us
I20260709 13:19:32.331202 18768 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:32.346773 18768 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:32.347193 18768 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:32.284276 18733 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:32.316965 18573 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:19:32.344116 18774 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:32.364135 15493 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:32.364537 15599 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
W20260709 13:19:32.353452 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 93226.3us
I20260709 13:19:32.354671 18719 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:32.384063 15493 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:32.338922 18553 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:32.370350 15566 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:32.386015 15493 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:19:32.392765 15567 leader_election.cc:336] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.029s (SENT)
I20260709 13:19:32.393249 15567 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:32.396111 18771 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:32.396814 18771 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:32.397953 18791 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:32.399638 18771 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:32.400975 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:32.401870 16237 raft_consensus.cc:2393] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 7: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:32.402120 18784 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:32.368705 18177 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:32.403421 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:32.404738 18611 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:32.407429 18784 raft_consensus.cc:697] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:32.408372 18784 consensus_queue.cc:237] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:32.413625 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:32.419389 18611 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:32.391201 18733 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:32.424764 18602 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:32.427173 18177 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:19:32.435334 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.436964 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:19:32.447873 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 61957.3us
I20260709 13:19:32.387231 18775 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:32.452740 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:32.384569 18785 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:32.453694 15493 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:32.453377 15673 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:32.452181 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:32.454497 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:32.456534 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 147989us
I20260709 13:19:32.456072 15677 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:32.458700 15641 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:32.465875 18573 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:32.471057 18553 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:32.474764 15493 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:32.477438 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:32.480473 15493 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:32.482167 18785 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:32.483038 15493 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:32.481204 18628 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:32.455904 18719 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:19:32.476680 18775 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:32.489403 15493 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:32.453133 18774 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:19:32.491276 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:32.492511 15493 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:32.493366 18774 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:32.491340 18785 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:32.489328 18628 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:32.488929 18681 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:32.494910 15493 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:32.495712 18628 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:32.497213 15493 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:32.499092 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 106361us
I20260709 13:19:32.598441 15671 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:19:32.602876 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.611142 18793 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.611786 18793 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:32.616511 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:32.626794 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 129647us
I20260709 13:19:32.627825 15493 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:19:32.633291 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 135218us
I20260709 13:19:32.634039 15641 leader_election.cc:400] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 5. Current term is 6.
I20260709 13:19:32.634701 15641 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:32.637552 15641 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:32.659291 15528 raft_consensus.cc:3055] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:32.659767 15528 raft_consensus.cc:740] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:32.657812 18793 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:32.616539 18772 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:19:32.660573 15528 consensus_queue.cc:260] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:32.669253 15528 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:32.664752 18573 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:32.616328 18785 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:32.674962 18785 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:32.677102 18785 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:32.677637 18785 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:32.635627 18774 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:19:32.693756 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 149006us
I20260709 13:19:32.728389 15673 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:32.732215 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:32.746259 15601 raft_consensus.cc:1275] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:19:32.748324 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 110852us
I20260709 13:19:32.749289 15641 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:32.666426 18290 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:32.732035 18785 raft_consensus.cc:697] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:32.740978 18772 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:32.753744 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:32.757529 15598 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:32.753088 15641 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:32.740489 18784 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:32.759573 18729 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:32.766351 18290 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:32.767088 15668 raft_consensus.cc:1240] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:32.768132 18772 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:32.769558 18492 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:32.750730 18792 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:32.786931 18772 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:32.792336 18768 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:32.793298 15669 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:32.795168 18785 consensus_queue.cc:237] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:32.804440 15528 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 8. (term mismatch)
I20260709 13:19:32.804893 15528 pending_rounds.cc:85] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 7
I20260709 13:19:32.805126 15528 pending_rounds.cc:107] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 13:19:32.805380 15528 raft_consensus.cc:2889] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:32.814714 16205 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:32.816216 18318 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:32.816785 18318 raft_consensus.cc:697] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:32.817726 18318 consensus_queue.cc:237] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:32.820238 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 477011us
I20260709 13:19:32.831785 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:32.840837 15673 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:32.843538 18601 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:32.846488 16236 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2afa8e112a744f218227526d861b98c3"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:19:32.830181 18784 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:32.852717 15672 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:32.831287 18698 raft_consensus.cc:2764] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 13:19:32.858219 15603 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
W20260709 13:19:32.859608 15676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57710 (request call id 761) took 2123 ms (client timeout 2573 ms). Trace:
W20260709 13:19:32.860108 15676 rpcz_store.cc:269] 0709 13:19:30.729116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.729270 (+   154us) service_pool.cc:224] Handling call
0709 13:19:32.852150 (+2122880us) spinlock_profiling.cc:243] Waited 665 ms on lock 0x7b540046d554. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:32.852286 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1530191232}
I20260709 13:19:32.846639 18573 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:32.869596 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 241858us
I20260709 13:19:32.872258 15493 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:32.875712 15493 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:32.876567 16236 raft_consensus.cc:606] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:32.882608 15641 leader_election.cc:400] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8.
I20260709 13:19:32.882952 15641 leader_election.cc:403] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:19:32.854249 18761 consensus_queue.cc:1059] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:19:32.879936 18775 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:32.885816 16231 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:19:32.886387 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:32.888391 16238 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:32.875532 18784 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:32.889251 16240 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:32.890990 16239 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:32.882169 18628 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:32.901872 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1752) took 1004 ms. Trace:
I20260709 13:19:32.902403 15526 rpcz_store.cc:276] 0709 13:19:31.897759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.931771 (+ 34012us) service_pool.cc:224] Handling call
0709 13:19:32.092030 (+160259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:32.100919 (+  8889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.901634 (+800715us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b540029ccd0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:32.901801 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":367955328}
I20260709 13:19:32.912209 16203 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:32.915063 16203 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:32.882346 18719 consensus_queue.cc:1059] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:19:32.883944 18573 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:19:32.922706 17744 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:32.913105 18628 raft_consensus.cc:697] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:32.914189 18167 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8.
I20260709 13:19:32.918006 18450 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:32.928604 18450 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:32.929021 18450 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:32.932646 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 53993.5us
I20260709 13:19:32.933971 15641 leader_election.cc:400] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7.
I20260709 13:19:32.934415 18289 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:32.934950 18289 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:32.934347 16238 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:32.927497 18761 raft_consensus.cc:3055] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:32.937986 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:32.938442 18761 raft_consensus.cc:740] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:32.939438 18733 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:32.939343 18761 consensus_queue.cc:260] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:32.940042 18733 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:32.942153 18289 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:19:32.943883 18761 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:32.944234 18628 consensus_queue.cc:237] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:32.946085 18481 consensus_queue.cc:1048] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:19:32.932947 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 162393us
I20260709 13:19:32.934314 15641 leader_election.cc:403] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:19:32.920787 18601 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:32.952467 15602 raft_consensus.cc:1275] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:19:32.938738 15601 raft_consensus.cc:1240] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:19:32.957193 18450 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:32.957621 18792 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7.
I20260709 13:19:32.958036 15670 raft_consensus.cc:1240] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:32.961149 15595 raft_consensus.cc:1275] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:19:32.963006 15567 leader_election.cc:400] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7.
I20260709 13:19:32.963582 15567 leader_election.cc:403] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:19:32.987931 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.005800 18450 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:33.005810 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:33.006739 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 56092.5us
I20260709 13:19:33.007548 15567 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:33.008785 18733 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:33.009157 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:33.009982 15675 raft_consensus.cc:2393] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 7: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:33.011283 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:33.013666 18601 raft_consensus.cc:697] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:33.025007 15567 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:33.034416 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:33.039737 15528 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:33.041584 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:33.043131 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:33.029165 18556 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7.
I20260709 13:19:33.043910 15526 raft_consensus.cc:2376] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6. Re-sending same reply.
I20260709 13:19:33.044768 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:33.045027 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:33.045701 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:33.048669 15642 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:33.051543 15641 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:33.056838 15595 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:33.058399 15520 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:33.063223 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:33.038502 18777 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:33.074882 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:33.084743 16234 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:33.075738 15526 raft_consensus.cc:2376] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6. Re-sending same reply.
I20260709 13:19:33.086704 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:33.082329 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:33.088398 18289 consensus_queue.cc:1048] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:19:33.106086 18601 consensus_queue.cc:237] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:33.041498 18792 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:19:33.112226 18797 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:33.116130 15598 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:19:33.119889 15672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 789) took 1115 ms. Trace:
I20260709 13:19:33.122073 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:33.137959 16235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57258 (request call id 714) took 1144 ms. Trace:
I20260709 13:19:33.138983 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:33.065189 18689 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:33.140142 18744 consensus_queue.cc:1059] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:33.065698 18677 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:33.064851 18775 consensus_queue.cc:1059] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:19:33.146998 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:33.149773 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:33.156385 17950 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:33.166936 16237 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:33.161659 15527 raft_consensus.cc:2376] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6. Re-sending same reply.
I20260709 13:19:33.169752 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1751) took 1272 ms. Trace:
I20260709 13:19:33.162343 16235 rpcz_store.cc:276] 0709 13:19:31.993103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.993617 (+   514us) service_pool.cc:224] Handling call
0709 13:19:31.994126 (+   509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:32.013809 (+ 19683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.137670 (+1123861us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b54005136d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:33.137857 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44275200}
I20260709 13:19:33.172636 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:33.170132 15527 rpcz_store.cc:276] 0709 13:19:31.897476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.121744 (+1224268us) service_pool.cc:224] Handling call
0709 13:19:33.169685 (+ 47941us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.160530 18771 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:33.176960 16234 raft_consensus.cc:2368] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 7. Current term is 8.
W20260709 13:19:33.177551 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 51914.2us
I20260709 13:19:33.179241 15568 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:33.160760 15672 rpcz_store.cc:276] 0709 13:19:32.004361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.004964 (+   603us) service_pool.cc:224] Handling call
0709 13:19:32.852390 (+847426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:32.853202 (+   812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.119611 (+266409us) spinlock_profiling.cc:243] Waited 847 ms on lock 0x7b540046d550. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:33.119787 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1948232960}
I20260709 13:19:33.182966 15668 raft_consensus.cc:1275] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:19:33.185428 15568 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:33.189509 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:33.194490 18708 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:33.200407 18768 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:33.175369 18689 raft_consensus.cc:697] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:33.204777 15523 raft_consensus.cc:3055] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:33.260628 15523 raft_consensus.cc:740] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:33.206112 15600 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:19:33.208161 18602 raft_consensus.cc:993] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9: : Instructing follower d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:19:33.210081 18774 raft_consensus.cc:3055] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:33.282397 18774 raft_consensus.cc:740] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:33.283263 18774 consensus_queue.cc:260] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:33.284637 18774 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:33.211483 15528 raft_consensus.cc:2368] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10.
I20260709 13:19:33.219034 18761 consensus_queue.cc:1048] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:19:33.287695 18787 consensus_queue.cc:1059] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:33.239619 18553 raft_consensus.cc:3055] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:33.288602 18787 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:33.288928 18553 raft_consensus.cc:740] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:33.290093 18566 raft_consensus.cc:3055] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:33.290587 18566 raft_consensus.cc:740] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:33.291343 18566 consensus_queue.cc:260] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:33.292434 18566 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:33.295688 15669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57710 (request call id 793) took 1238 ms. Trace:
I20260709 13:19:33.296094 15669 rpcz_store.cc:276] 0709 13:19:32.057451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.057569 (+   118us) service_pool.cc:224] Handling call
0709 13:19:33.295500 (+1237931us) spinlock_profiling.cc:243] Waited 735 ms on lock 0x7b54003fb454. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:33.295594 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1689569024}
W20260709 13:19:33.300704 18802 log.cc:927] Time spent T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.184s	user 0.004s	sys 0.000s
I20260709 13:19:33.303050 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:33.303750 15668 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:33.303117 18553 consensus_queue.cc:260] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:33.311694 18553 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:33.317785 16203 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:19:33.263779 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 78467.3us
I20260709 13:19:33.278873 18602 raft_consensus.cc:1081] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Signalling peer d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:19:33.279937 18689 consensus_queue.cc:237] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:33.281081 15523 consensus_queue.cc:260] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:33.330047 15523 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:33.331290 18801 log.cc:927] Time spent T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.220s	user 0.002s	sys 0.000s
I20260709 13:19:33.274914 18772 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:33.106547 15493 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:19:33.338428 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 289308us
W20260709 13:19:33.341068 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 234625us
I20260709 13:19:33.342860 15493 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:33.343340 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:33.344969 15595 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:33.349071 15598 raft_consensus.cc:1240] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:33.356710 15523 raft_consensus.cc:1275] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:19:33.358055 15523 pending_rounds.cc:85] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 6
I20260709 13:19:33.358959 15523 pending_rounds.cc:107] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:19:33.359954 15523 raft_consensus.cc:2889] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:33.331337 18601 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:33.376875 15567 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:33.382637 15599 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:19:33.383773 18177 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:33.390355 15567 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:33.403517 15567 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:33.367502 18761 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:33.367871 18791 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:33.409168 15567 leader_election.cc:400] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10.
I20260709 13:19:33.418646 15567 leader_election.cc:403] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:33.376154 18601 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:33.445008 15520 raft_consensus.cc:1240] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:33.406599 15519 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2afa8e112a744f218227526d861b98c3"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:56840
I20260709 13:19:33.367115 18768 consensus_queue.cc:1048] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:19:33.367295 18775 consensus_queue.cc:1048] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:19:33.447072 15519 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:33.451339 15519 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:33.376624 18628 consensus_queue.cc:1059] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:19:33.418462 18772 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:33.415185 18179 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:33.445338 18601 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:33.474937 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 101006us
I20260709 13:19:33.477154 15519 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:33.479565 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 396167us
I20260709 13:19:33.414754 18708 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:19:33.450109 18771 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:33.488725 15493 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.488976 15602 raft_consensus.cc:1240] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:33.436005 18808 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10.
I20260709 13:19:33.498472 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:33.502785 15519 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:33.505008 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.446696 18611 consensus_queue.cc:1048] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:19:33.476029 18761 raft_consensus.cc:3055] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:33.522423 15568 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:33.480897 18719 consensus_queue.cc:1059] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:19:33.471190 18179 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:33.531165 18808 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:33.560561 16231 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:19:33.561009 16231 pending_rounds.cc:85] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9: Aborting all ops after (but not including) 6
I20260709 13:19:33.561213 16231 pending_rounds.cc:107] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:19:33.561446 16231 raft_consensus.cc:2889] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:33.534008 18805 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:19:33.554268 18677 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:19:33.528277 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:33.566069 16234 raft_consensus.cc:3055] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:33.566411 16234 raft_consensus.cc:740] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:33.567142 16234 consensus_queue.cc:260] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:33.568073 16234 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:33.555642 18179 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:33.529011 15595 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:33.530637 16240 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:19:33.531196 15568 leader_election.cc:400] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 7. Current term is 8.
I20260709 13:19:33.576081 15568 leader_election.cc:403] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:19:33.576849 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 47792.1us
I20260709 13:19:33.577521 15568 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:33.577762 18793 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:19:33.578704 15568 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:33.535018 16237 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:33.530083 15668 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:33.526777 16235 raft_consensus.cc:1240] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:33.548710 18131 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.552369 16205 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:33.583374 15519 raft_consensus.cc:1240] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:33.584532 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:33.511816 18761 raft_consensus.cc:740] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:33.588032 15601 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:33.584349 16203 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:33.597863 15671 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:33.599525 16237 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:19:33.511687 18797 consensus_queue.cc:1059] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 13:19:33.509176 18771 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:33.641639 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 57373us
I20260709 13:19:33.642987 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:33.643893 16235 raft_consensus.cc:2393] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 8: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:19:33.650461 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 51159.5us
I20260709 13:19:33.605998 18056 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:33.596194 18289 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:19:33.681286 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 142209us
W20260709 13:19:33.687276 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 150802us
I20260709 13:19:33.603417 18803 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:19:33.647491 18177 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:33.652093 18481 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:33.642117 18179 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.606501 18450 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 13:19:33.596046 18131 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:33.677052 18289 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:33.607592 18777 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 13:19:33.802640 15568 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:33.874476 15671 raft_consensus.cc:1275] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:19:33.878379 15597 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:19:33.663307 18808 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.169s
I20260709 13:19:33.711819 18677 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:33.800570 18056 raft_consensus.cc:697] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:33.867878 18785 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:33.899353 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 235243us
I20260709 13:19:33.814073 18289 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:33.680701 18761 consensus_queue.cc:260] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:33.912019 18819 log.cc:927] Time spent T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:19:33.818512 18179 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:33.906318 18481 raft_consensus.cc:697] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:33.955201 16234 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
W20260709 13:19:33.995373 18821 log.cc:927] Time spent T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
I20260709 13:19:33.996407 18771 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:33.997212 18800 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:33.997825 18800 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:34.000021 18800 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:34.002336 18811 log.cc:927] Time spent T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 13:19:34.003160 18131 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:34.005318 18601 consensus_meta.cc:287] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.518s	user 0.008s	sys 0.000s
I20260709 13:19:34.005797 18601 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:34.008001 18601 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:34.009298 18481 consensus_queue.cc:237] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:33.912317 18677 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:33.936813 18785 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:34.057356 18826 log.cc:927] Time spent T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:19:34.063470 18785 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:34.068087 18179 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:34.019066 18056 consensus_queue.cc:237] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:34.074860 18793 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 7. Current term is 8.
I20260709 13:19:34.057226 18780 raft_consensus.cc:3055] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:34.075906 18780 raft_consensus.cc:740] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:34.076871 18780 consensus_queue.cc:260] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:34.073643 18677 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:34.079239 18289 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:34.081399 18289 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:34.107323 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 304785us
I20260709 13:19:34.108902 18553 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:34.112722 15594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 1754) took 1175 ms. Trace:
W20260709 13:19:34.114055 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 243226us
I20260709 13:19:34.113179 15594 rpcz_store.cc:276] 0709 13:19:32.937000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.982836 (+ 45836us) service_pool.cc:224] Handling call
0709 13:19:32.983259 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:33.010343 (+ 27084us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.010374 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:33.015204 (+  4830us) log.cc:844] Serialized 26 byte log entry
0709 13:19:33.424102 (+408898us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:33.868150 (+444048us) log.cc:844] Serialized 14 byte log entry
0709 13:19:34.112368 (+244218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.112448 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:34.112489 (+    41us) raft_consensus.cc:1685] finished
0709 13:19:34.112501 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.112624 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":269922,"prepare.queue_time_us":137898,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":26529,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"thread_start_us":787778,"threads_started":2,"wal-append.queue_time_us":470095}
I20260709 13:19:34.115386 18733 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:34.119330 18785 consensus_queue.cc:1048] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:19:34.119612 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 66823.5us
W20260709 13:19:34.120729 18816 log.cc:927] Time spent T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.485s	user 0.000s	sys 0.007s
W20260709 13:19:34.121632 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 199082us
I20260709 13:19:34.126353 15525 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 13:19:34.126739 18777 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:34.127709 16237 raft_consensus.cc:606] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:34.129832 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:34.130998 16234 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:34.126756 15525 pending_rounds.cc:85] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 9
I20260709 13:19:34.133251 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:34.133710 15525 pending_rounds.cc:107] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:19:34.134011 15525 raft_consensus.cc:2889] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:34.140465 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 497808us
I20260709 13:19:34.145505 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:34.149766 15524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 843) took 1839 ms. Trace:
I20260709 13:19:34.150314 15524 rpcz_store.cc:276] 0709 13:19:32.310356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.310694 (+   338us) service_pool.cc:224] Handling call
0709 13:19:32.322691 (+ 11997us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:32.323031 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:32.323062 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:32.323521 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:19:33.202838 (+879317us) spinlock_profiling.cc:243] Waited 3.39 ms on lock 0x7b1800458f20. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab0d1 00007f5ce49c6a0f
0709 13:19:33.424405 (+221567us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:33.424444 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.424522 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:34.120050 (+695528us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:19:34.149502 (+ 29452us) raft_consensus.cc:1685] finished
0709 13:19:34.149525 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.149664 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1449,"raft.queue_time_us":693,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":13692,"spinlock_wait_cycles":32036480,"thread_start_us":221415,"threads_started":1,"wal-append.queue_time_us":1200465}
I20260709 13:19:34.153851 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:34.158514 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:34.141772 15493 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.179070 18824 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:34.179694 18824 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:34.186336 18825 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:34.188125 18824 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:34.194666 18553 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:34.197203 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:34.198925 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:34.205892 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 65610.8us
W20260709 13:19:34.205958 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 63647.1us
I20260709 13:19:34.214972 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:34.220039 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:34.222167 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:34.222807 15668 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:34.223917 15670 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:34.225809 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:34.226102 15597 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:34.227512 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:34.227057 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 486560us
I20260709 13:19:34.233628 15493 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:34.333719 15600 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:34.335978 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:34.336633 16237 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:34.346024 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 112504us
I20260709 13:19:34.338954 15568 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:34.360924 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:34.362497 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:34.365897 15600 raft_consensus.cc:2368] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 8. Current term is 9.
I20260709 13:19:34.334159 18850 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.365938 15491 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:34.376523 15668 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:19:34.341426 18803 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:34.343974 18784 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:34.485493 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 116011us
I20260709 13:19:34.361353 18553 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:19:34.512341 18822 log.cc:927] Time spent T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 13:19:34.512996 18847 raft_consensus.cc:993] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9: : Instructing follower d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:19:34.379611 18800 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:19:34.514079 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 187746us
I20260709 13:19:34.357250 18824 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:34.484179 18846 log.cc:927] Time spent T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
I20260709 13:19:34.515744 15642 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:34.513392 18789 log.cc:927] Time spent T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.124s	user 0.004s	sys 0.006s
I20260709 13:19:34.509464 18170 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:19:34.536512 18824 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:34.484719 18850 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:34.558801 18800 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:34.545884 18634 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.404s
I20260709 13:19:34.550500 18689 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:34.588666 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 883897us
I20260709 13:19:34.620800 18830 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:34.626099 18687 raft_consensus.cc:1081] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Signalling peer d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:19:34.630133 16237 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
W20260709 13:19:34.632861 18852 log.cc:927] Time spent T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:19:34.641307 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.645972 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:34.646770 15670 raft_consensus.cc:2368] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 8. Current term is 9.
I20260709 13:19:34.642164 18803 raft_consensus.cc:697] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:34.648648 18803 consensus_queue.cc:237] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:34.787137 15595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 1979) took 1096 ms. Trace:
I20260709 13:19:34.787583 15595 rpcz_store.cc:276] 0709 13:19:33.690626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.692379 (+  1753us) service_pool.cc:224] Handling call
0709 13:19:33.803574 (+111195us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:34.130970 (+327396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:34.130999 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:34.131402 (+   403us) log.cc:844] Serialized 26 byte log entry
0709 13:19:34.333264 (+201862us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:34.333301 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.662404 (+329103us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b540016ba54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:34.662463 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:34.662518 (+    55us) raft_consensus.cc:1685] finished
0709 13:19:34.662534 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.787042 (+124508us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":111126,"raft.queue_time_us":111884,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":94,"spinlock_wait_cycles":496473088,"thread_start_us":201513,"threads_started":1,"wal-append.queue_time_us":352944}
W20260709 13:19:34.804102 18859 log.cc:927] Time spent T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 13:19:34.614853 18800 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:34.808023 18800 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:34.808697 18800 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.809216 18800 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:34.790012 18850 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:34.836397 18775 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.860287 18775 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:34.862454 18775 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:34.864827 18775 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.863327 18855 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:34.865486 18855 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:34.867816 15566 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:34.868863 15566 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:19:34.867781 15677 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.010s	sys 0.033s
W20260709 13:19:34.840577 18853 log.cc:927] Time spent T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.289s	user 0.003s	sys 0.000s
W20260709 13:19:34.900573 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:34.903544 15596 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.008s	sys 0.003s
W20260709 13:19:34.904784 15596 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.008s	sys 0.003s
I20260709 13:19:34.866365 18824 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:34.906888 18854 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:34.887629 18775 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:34.907366 18854 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:34.907716 18854 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:34.908772 18775 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:34.909017 18863 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.910133 18721 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:34.910288 18849 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:34.910701 18721 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:34.911571 18830 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:34.912889 18721 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:34.887780 18850 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:34.910787 18849 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:34.912134 18830 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:34.917732 18850 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:34.954062 18831 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:34.909556 18863 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:34.960971 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 426722us
I20260709 13:19:34.922096 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:34.941109 15671 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.089s	sys 0.089s
I20260709 13:19:34.921331 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:34.920900 15491 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:19:34.963392 15671 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.089s	sys 0.092s
I20260709 13:19:34.963924 15668 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
W20260709 13:19:34.965983 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 45189.1us
I20260709 13:19:34.966588 18855 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:34.919703 18824 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:34.929948 18785 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:34.975734 18863 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:34.967415 18873 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:34.975869 18855 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:19:34.976560 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 107769us
W20260709 13:19:34.970021 15493 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.981703 15491 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:34.985769 15524 raft_consensus.cc:1240] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:34.987695 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:34.962487 18831 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:34.989872 18775 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:34.990513 18775 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:34.983814 18873 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:34.991688 18873 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:35.003214 18849 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.009171 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:35.010766 18850 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:35.011502 18800 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.012040 18800 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:35.012552 18800 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.014223 18874 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:35.019656 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:35.021402 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:35.018358 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:35.023998 15522 raft_consensus.cc:2410] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:19:34.983733 18855 raft_consensus.cc:697] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:35.031320 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:35.032173 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:35.032223 15528 raft_consensus.cc:2393] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 6: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:35.034425 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:35.014748 18874 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.028424 18830 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:34.986169 18824 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:35.034132 18849 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:34.987074 18785 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:35.039935 18797 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.048964 18874 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:35.039556 18854 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:35.066936 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:35.067267 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:35.066334 16237 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:19:35.069105 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 82892.2us
I20260709 13:19:35.071270 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:35.075214 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:35.062467 18849 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:35.061146 18880 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.086484 15598 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:19:35.081207 15676 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:35.087332 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:35.086232 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:35.092285 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:35.095155 16203 leader_election.cc:400] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 8. Current term is 9.
I20260709 13:19:35.095531 16203 leader_election.cc:403] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:19:35.095077 15594 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 6: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:35.097946 15674 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:35.105069 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:35.085383 18868 log.cc:927] Time spent T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:19:35.069332 18876 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:35.107933 18880 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.085902 18800 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:35.115195 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 86044.3us
I20260709 13:19:35.117805 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:35.107399 18736 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 8. Current term is 9.
I20260709 13:19:35.126150 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:35.065619 18797 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.130074 18831 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.131048 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:35.088807 18878 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:19:35.119990 18800 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:35.085722 18855 consensus_queue.cc:237] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.135290 18879 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:35.163765 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:35.149417 18877 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.113502 18876 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.151125 18800 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.164723 18881 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:35.182080 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.226652 18885 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.169256 18805 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:35.161522 18879 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:35.254408 18800 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:35.233551 18869 log.cc:927] Time spent T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:19:35.175109 18824 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:35.261620 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:35.233728 18880 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.230510 18785 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:35.230852 18878 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.234737 18881 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.268976 18886 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.269340 18881 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.269901 18881 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:35.271572 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:35.273851 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:35.276518 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:35.277614 15493 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:35.279320 18744 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.279963 18744 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.290712 18628 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:35.291528 18628 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:35.227319 18885 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.294759 18885 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.295806 18830 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.296346 18830 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.297000 18676 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.298075 15528 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:35.298411 18830 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.298620 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:35.300479 16205 leader_election.cc:400] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 8. Current term is 9.
I20260709 13:19:35.300913 18736 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:35.302480 18678 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:19:35.310458 18882 log.cc:927] Time spent T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:19:35.316341 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:35.316710 18879 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:35.317055 15672 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:19:35.317206 18879 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.256302 18884 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.318420 18884 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.320163 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:35.320596 18884 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.321326 18886 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.323460 18886 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.237622 18805 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:35.329126 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:35.233947 18883 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.170643 18877 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.297533 18676 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.329965 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:35.403345 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 125786us
I20260709 13:19:35.329783 15677 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:19:35.330374 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:35.422569 15493 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:35.422973 15671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49094 (request call id 1855) took 1197 ms. Trace:
I20260709 13:19:35.421716 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:35.335822 15491 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:35.429828 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 94095us
I20260709 13:19:35.430595 15491 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:35.416854 15567 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:35.425843 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:35.431562 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:35.260671 18854 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:35.434023 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:35.433943 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:35.436187 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:35.438030 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:35.428036 18888 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:35.438730 18889 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:35.436182 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:35.424072 15671 rpcz_store.cc:276] 0709 13:19:34.227173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.227432 (+   259us) service_pool.cc:224] Handling call
0709 13:19:34.227772 (+   340us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:34.229800 (+  2028us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:34.229833 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:34.239534 (+  9701us) log.cc:844] Serialized 52 byte log entry
0709 13:19:34.964551 (+725017us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:34.964595 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.964667 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:35.414385 (+449718us) raft_consensus.cc:1685] finished
0709 13:19:35.414409 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:35.422910 (+  8501us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":149842,"raft.queue_time_us":104830,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":9994,"thread_start_us":592522,"threads_started":1,"wal-append.queue_time_us":679267}
I20260709 13:19:35.325771 18805 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:35.440166 16235 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:19:35.264779 18800 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:35.443432 15491 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:35.442719 15672 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:35.401710 18744 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.442233 18678 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.426642 18849 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:35.317718 18879 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.437091 18884 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.266990 18824 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.188s
I20260709 13:19:35.440928 18628 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:35.439230 18797 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.440820 18876 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.399407 18874 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:35.427287 18883 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.486650 18874 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:35.467296 18888 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.478157 18889 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.487532 18883 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.484882 18684 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:35.447916 18736 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:35.488318 18684 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.488691 18736 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.490522 18684 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:35.490710 18736 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:35.491441 16239 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:19:35.487437 15522 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 6: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:35.474431 18886 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:35.482955 18628 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.494335 18628 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:35.494825 18877 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.495546 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:35.496336 18676 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:35.496634 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:35.496901 18894 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:35.482650 18884 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.481653 15672 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:35.381177 18878 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.507018 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:35.499676 18884 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.490432 18761 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:35.486817 18775 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.512490 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:35.497403 18894 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:35.513163 15598 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:19:35.456430 18800 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:35.475276 18678 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.487447 18888 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:35.518862 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.487895 18889 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.526391 18889 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.535101 18628 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.539063 18879 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:35.539535 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 96321.9us
I20260709 13:19:35.540599 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:35.540751 15491 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:35.543382 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 126607us
I20260709 13:19:35.553892 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:35.554682 15594 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:35.559103 16235 raft_consensus.cc:606] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:35.559614 16235 raft_consensus.cc:614] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:19:35.619115 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 70709.8us
I20260709 13:19:35.622367 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:35.623904 15672 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:35.515197 18894 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:35.625486 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 194749us
I20260709 13:19:35.627827 18825 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.627988 18874 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:35.619141 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 71587.4us
I20260709 13:19:35.628443 18874 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:35.629638 18824 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.630215 18824 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.631153 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:35.628458 18825 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.637317 18440 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:35.637952 18440 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.640182 18440 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:35.665259 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:35.668118 18440 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:35.672646 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:35.668931 18440 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.680920 18440 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:35.741251 15672 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:19:35.744760 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:35.745680 15674 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 6: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:35.752836 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:35.744285 18573 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:35.751864 18842 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:35.755851 18890 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:35.755944 18842 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.756551 18890 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.757776 18573 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:35.758200 18900 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.758841 18900 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.758864 18890 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.755227 18879 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:35.755115 18780 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:35.760268 18879 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:35.765545 18879 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:19:35.769215 15668 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:35.761811 18900 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.779220 18878 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.788352 18879 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:35.788471 18888 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.789009 18879 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:35.791392 18879 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:35.794539 18800 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:35.618949 18719 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.797658 18719 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.802400 18678 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:35.619014 18785 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.803473 18785 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.557171 18867 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:35.830509 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:35.832301 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 44219.3us
I20260709 13:19:35.833809 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:35.834506 15594 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:19:35.769022 18889 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.838303 18879 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.852118 18889 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.852483 18879 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.854318 18889 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.854643 18879 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.852365 18885 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.857887 15566 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:35.810029 18785 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.862429 15493 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:35.855383 18885 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.829109 18878 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.828581 18733 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:35.869396 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:35.871451 18885 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.876886 16239 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:19:35.886340 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:35.886906 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:35.888208 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:35.888900 15528 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:35.849828 18867 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.890441 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:35.892120 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:35.856833 18775 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.901129 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:35.864938 18879 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:35.914076 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:35.916314 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:35.916915 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:35.882220 18785 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:35.919231 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:35.920553 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:35.920986 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:35.921844 15525 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:35.922307 15523 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:35.875084 18628 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:35.924702 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:35.924933 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:35.925580 15525 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 3.
I20260709 13:19:35.925992 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:35.889763 18736 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:35.898471 18849 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.930778 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:35.934091 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:35.934885 15525 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:19:35.944617 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:35.890704 18878 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.919011 18890 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:35.926941 18855 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:35.929427 18884 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:35.956898 15567 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:35.896369 18733 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:35.931250 18879 raft_consensus.cc:697] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:35.905514 18867 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.917157 18775 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.966923 15567 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:35.971068 15567 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:35.939258 18736 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.946835 18849 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.972978 15567 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:35.961036 18890 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:35.978480 15526 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:35.974723 15596 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:35.974350 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:35.970095 18872 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.983573 15567 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:35.984477 15567 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:35.985296 18879 consensus_queue.cc:237] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:35.989195 18904 log.cc:927] Time spent T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:19:35.975705 18877 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:35.990761 18877 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:35.991160 18877 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:35.999719 18553 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:35.973773 18784 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:36.004130 15567 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:35.963563 18855 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:36.012987 15594 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:35.977020 18777 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:35.962849 18884 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:36.016381 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:36.021956 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:36.022670 15594 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:19:36.011214 18899 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:36.013839 18553 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:36.010528 18784 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:36.041963 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:36.043277 15594 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:36.015230 18777 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:36.046480 18906 log.cc:927] Time spent T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:19:36.010771 18872 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:36.056378 18884 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:36.057564 15493 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:36.058288 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:36.037163 18553 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:19:36.058557 15424 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49132 (request call id 338) took 16692 ms (client timeout 15000 ms). Trace:
I20260709 13:19:36.060425 18177 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.062564 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:36.047398 18777 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:36.073768 15493 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:36.074256 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:36.063634 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:36.063853 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:36.046818 18784 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:36.046195 18556 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:36.081501 18556 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:36.078336 18761 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:36.082454 18761 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:36.082841 18761 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:36.082985 15675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 834) took 1011 ms. Trace:
I20260709 13:19:36.061781 18892 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:36.084254 15675 rpcz_store.cc:276] 0709 13:19:35.071364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.071467 (+   103us) service_pool.cc:224] Handling call
0709 13:19:35.097211 (+ 25744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:35.150128 (+ 52917us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:35.150155 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:35.150195 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:35.150223 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.082778 (+932555us) spinlock_profiling.cc:243] Waited 52.7 ms on lock 0x7b54003886d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:36.082821 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:36.082930 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":121134464}
I20260709 13:19:36.084923 18894 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:36.075718 18177 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:36.081984 18556 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:36.086843 18849 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:36.087479 18177 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:36.088195 18775 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:36.089254 18736 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:36.084448 18892 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:36.090765 18890 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:36.092451 18892 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:36.093591 18892 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:36.094394 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1771) took 1066 ms. Trace:
I20260709 13:19:36.094779 15522 rpcz_store.cc:276] 0709 13:19:35.027701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.031937 (+  4236us) service_pool.cc:224] Handling call
0709 13:19:36.094170 (+1062233us) spinlock_profiling.cc:243] Waited 454 ms on lock 0x7b54003d9fd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:36.094298 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1045040384}
I20260709 13:19:36.094095 18892 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:36.072190 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:36.099225 15594 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:36.101267 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.102428 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:36.104009 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:36.070315 18774 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:36.105075 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:36.074565 18553 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:36.111663 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:36.118723 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 45025.6us
W20260709 13:19:36.071504 15424 rpcz_store.cc:269] 0709 13:19:19.414375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.493196 (+ 78821us) service_pool.cc:224] Handling call
0709 13:19:36.058453 (+16565257us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:30.385098 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:30.385775 (+   677us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:30.385839 (+    64us) write_op.cc:435] Acquired schema lock
0709 13:19:30.385859 (+    20us) tablet.cc:662] Decoding operations
0709 13:19:30.514001 (+128142us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:30.514084 (+    83us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:30.514113 (+    29us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:19:30.515493 (+  1380us) tablet.cc:701] Row locks acquired
0709 13:19:30.515535 (+    42us) write_op.cc:260] PREPARE: finished
0709 13:19:30.515700 (+   165us) write_op.cc:270] Start()
0709 13:19:30.515850 (+   150us) write_op.cc:276] Timestamp: P: 1783603170515640 usec, L: 0
0709 13:19:30.515869 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:19:30.516619 (+   750us) log.cc:844] Serialized 7395 byte log entry
0709 13:19:31.542665 (+1026046us) op_driver.cc:464] REPLICATION: finished
0709 13:19:31.830272 (+287607us) write_op.cc:301] APPLY: starting
0709 13:19:31.830369 (+    97us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:31.830719 (+   350us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:31.830852 (+   133us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:31.832913 (+  2061us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:31.833305 (+   392us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=18
0709 13:19:31.833358 (+    53us) write_op.cc:312] APPLY: finished
0709 13:19:31.833791 (+   433us) log.cc:844] Serialized 125 byte log entry
0709 13:19:31.846310 (+ 12519us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:31.846498 (+   188us) write_op.cc:454] Released schema lock
0709 13:19:31.847135 (+   637us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:31.847567 (+   432us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:31.847594 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2905551,"reported_tablets":41,"child_traces":[["op",{"apply.queue_time_us":268758,"apply.run_cpu_time_us":4518,"apply.run_wall_time_us":17299,"num_ops":18,"prepare.queue_time_us":114721,"prepare.run_cpu_time_us":16696,"prepare.run_wall_time_us":199993,"raft.queue_time_us":403332,"raft.run_cpu_time_us":83359,"raft.run_wall_time_us":328372,"replication_time_us":968022,"thread_start_us":670753,"threads_started":4,"wal-append.queue_time_us":142920,"wal-append.run_cpu_time_us":32609,"wal-append.run_wall_time_us":2172684}]]}
I20260709 13:19:36.121119 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.125283 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:36.126185 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:36.128250 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.132836 16238 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:36.132290 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.135977 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.138163 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.138823 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:36.139117 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:36.140378 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:36.144369 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:36.110939 18899 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.124212 18774 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:36.150844 18845 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:36.151038 18774 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:36.151485 18845 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:36.153560 18845 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:36.145511 18440 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:36.154376 18888 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:36.154671 18440 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:36.156667 18440 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:36.157639 18873 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:36.161734 15668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49100 (request call id 2029) took 1176 ms. Trace:
I20260709 13:19:36.127542 18885 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:36.123696 18775 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:36.166595 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.167685 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.168670 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.164274 18855 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:36.168035 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.178545 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:36.162151 15668 rpcz_store.cc:276] 0709 13:19:34.985543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.986452 (+   909us) service_pool.cc:224] Handling call
0709 13:19:36.161574 (+1175122us) spinlock_profiling.cc:243] Waited 781 ms on lock 0x7b540026bbd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:36.161675 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1795860352}
I20260709 13:19:36.179095 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.180974 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:36.168380 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.179013 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.181862 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:36.182238 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:36.185063 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:36.186236 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:36.186501 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:36.186956 15526 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:36.187064 15527 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:36.136319 18878 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:36.198451 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.199194 16233 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:19:36.199676 18777 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:36.200232 18604 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.200764 18604 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:36.187717 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:36.179313 18899 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:36.191200 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:36.191885 18908 log.cc:927] Time spent T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:19:36.204018 18899 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:36.205987 18777 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:36.168534 18867 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:36.175246 18842 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:36.212503 15595 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:19:36.184528 15567 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:36.190620 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:36.213763 15521 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:36.214556 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.191283 18784 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:36.191756 18733 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:36.194701 18877 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:36.226545 16205 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:36.170416 18439 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:36.188844 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.183322 18736 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:36.179962 18602 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:36.166448 18913 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:36.218345 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.237776 16231 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:36.219353 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.220952 15641 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:36.221570 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:36.126053 18556 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:36.224519 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:36.225291 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:36.197376 18719 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:36.169878 18849 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:36.166661 18872 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:36.244890 15601 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:36.230315 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:36.166896 18775 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:36.246498 15599 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:19:36.239789 15641 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:36.251941 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.254657 15596 raft_consensus.cc:1240] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:36.236179 18177 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:36.257771 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.258555 16232 raft_consensus.cc:2393] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 7: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:36.232550 15567 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:36.234439 15568 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:36.234961 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:36.235721 15522 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:56840
I20260709 13:19:36.241791 15600 raft_consensus.cc:2393] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 7: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:36.242668 15521 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:36.260277 15566 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:36.244357 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.214910 18850 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:36.238278 18676 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:36.264233 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.282483 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.268180 15568 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:36.273505 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:36.275542 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:36.283483 15519 raft_consensus.cc:2955] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:36.277148 15528 raft_consensus.cc:1240] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:36.278398 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:36.279862 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:36.286618 15524 raft_consensus.cc:2410] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:19:36.244957 18780 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:36.287075 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:36.228299 18439 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:36.229390 18761 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:36.239861 18736 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:36.245035 18913 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:36.251457 18849 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:36.245103 18602 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:36.857511 18777 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:36.858057 18777 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:36.858542 18777 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:36.281519 18676 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:36.860584 18676 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:36.784225 18719 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:36.332481 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:36.425439 15640 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.14.23.126:39251, user_credentials={real_user=slave}} blocked reactor thread for 57242.2us
W20260709 13:19:36.504108 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 228630us
W20260709 13:19:36.345377 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 85187.5us
I20260709 13:19:36.289077 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:36.397432 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 131550us
I20260709 13:19:36.593506 15522 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:36.878822 15522 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:36.360525 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:36.881081 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:36.874974 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 606830us
I20260709 13:19:36.858217 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:36.264303 18845 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:36.256938 18892 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:36.902531 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:36.634114 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:36.907891 18913 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:36.264501 18899 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.910272 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:36.900702 15641 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:36.888959 15568 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:36.913895 15521 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.001s	sys 0.014s
W20260709 13:19:36.918274 15521 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.002s	sys 0.014s
I20260709 13:19:36.283747 18891 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:36.283458 18884 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:19:36.350433 18882 log.cc:927] Time spent T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:19:36.915386 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:36.367693 18918 log.cc:927] Time spent T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:19:36.919898 15568 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:36.922106 15524 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:19:36.900231 18845 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:36.935464 18845 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:36.874258 18780 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:36.349879 18879 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:36.331931 16233 raft_consensus.cc:1275] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:19:36.367960 18894 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:36.877808 18719 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:36.928243 18439 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:36.535188 18553 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:36.952240 15568 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:36.953022 18553 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:36.953171 18913 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:36.928964 18842 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:36.929255 18797 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:36.909723 18440 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:19:36.932173 18919 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:36.921526 18892 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:36.926260 18891 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:36.367108 18775 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:36.656333 18698 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:36.361965 18556 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:36.359861 18784 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:36.360809 18604 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:36.938071 15524 pending_rounds.cc:85] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 8
I20260709 13:19:36.973480 15524 pending_rounds.cc:107] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:19:36.973770 15524 raft_consensus.cc:2889] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:36.289757 18850 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:36.656803 18877 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:36.939090 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:36.942544 15525 raft_consensus.cc:1275] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:19:36.948266 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.972127 15491 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:36.975909 16231 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:19:36.954437 15568 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:36.955379 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:36.958209 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:36.944103 18780 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:36.942742 18880 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:36.945859 18879 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:36.925415 18899 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:36.957115 18797 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:36.784236 18907 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:36.980988 18850 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:36.970335 18891 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:36.973026 18698 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:36.984977 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:36.967732 18926 log.cc:927] Time spent T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:19:36.986600 15568 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:36.989481 18719 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:36.984473 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:36.959965 18842 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:36.982854 18775 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:36.988431 18879 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:36.987315 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:37.000735 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1770) took 1974 ms. Trace:
I20260709 13:19:37.005578 15568 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:36.987782 18556 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:36.994288 18797 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:37.011489 15519 rpcz_store.cc:276] 0709 13:19:35.027255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.027445 (+   190us) service_pool.cc:224] Handling call
0709 13:19:35.027977 (+   532us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:19:35.033606 (+  5629us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:19:35.047713 (+ 14107us) log.cc:844] Serialized 14 byte log entry
0709 13:19:36.211529 (+1163816us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b1800340fc0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce45b34bc 00007f5ce190148b 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191b455 00007f5ce4a03060 00007f5ce4a15716 00007f5ce4a26287
0709 13:19:36.211627 (+    98us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:19:36.239480 (+ 27853us) log.cc:844] Serialized 608 byte log entry
0709 13:19:36.239664 (+   184us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:36.740897 (+501233us) log.cc:844] Serialized 14 byte log entry
0709 13:19:36.741203 (+   306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.741293 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:37.000519 (+259226us) raft_consensus.cc:1685] finished
0709 13:19:37.000543 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:37.000680 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2083,"prepare.queue_time_us":16882,"prepare.run_cpu_time_us":147,"prepare.run_wall_time_us":276,"raft.queue_time_us":115661,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":2112,"spinlock_wait_cycles":5393152,"thread_start_us":96483,"threads_started":2,"wal-append.queue_time_us":98805}
I20260709 13:19:37.020694 18602 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:37.007697 18877 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:37.027324 18698 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:37.031054 15568 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:37.053965 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 176455us
I20260709 13:19:37.055140 16205 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:37.056223 16205 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:37.054442 15642 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:37.058097 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:37.060048 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:37.062633 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:37.064144 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:37.060590 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:37.066531 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:37.057725 16205 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:37.033244 18880 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:37.071031 16205 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:37.033370 18877 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:37.003626 18907 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.054752 18761 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:37.066828 18440 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:37.076844 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:37.033596 18923 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:37.035485 18849 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:37.098841 18849 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:37.072320 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:37.080298 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:37.102634 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 201988us
I20260709 13:19:37.054374 18439 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:37.087692 18877 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:37.113353 18842 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:37.114362 18573 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:37.116079 18935 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:37.116640 18935 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:37.065660 18784 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:19:37.092617 18440 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:37.122309 18440 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:37.065929 18602 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:37.126085 18931 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:37.126611 18931 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:37.127007 18931 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:37.071322 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:37.059729 18556 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:37.075459 18318 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:37.073671 18880 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:19:37.087384 15493 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.68:39627 timed out after 1.653s (SENT) [suppressed 5 similar messages]
I20260709 13:19:37.106541 18439 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.078387 18736 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:37.138334 15522 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.126147 18602 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:19:37.163285 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 194876us
I20260709 13:19:37.125583 18676 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:37.160718 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 106361us
I20260709 13:19:37.165683 15642 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:37.166839 16204 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:37.142011 18318 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:37.182684 16237 raft_consensus.cc:1240] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:37.155431 18880 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:37.171789 15642 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:37.175861 15672 raft_consensus.cc:1275] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:19:37.180250 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:37.186200 18877 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.201241 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:37.202701 18944 consensus_queue.cc:1059] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:37.203509 18944 consensus_queue.cc:1243] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer d3c4f5bd76e0445795bad5f6dd93ff69 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:19:37.206696 18935 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:37.207363 18935 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:37.193719 18848 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:37.167404 18842 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:37.210665 18848 raft_consensus.cc:3055] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:37.211441 18848 raft_consensus.cc:740] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:37.212306 18848 consensus_queue.cc:260] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.213472 18848 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:37.103381 18923 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.188428 15669 raft_consensus.cc:1275] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:19:37.138154 18934 consensus_queue.cc:1059] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:19:37.161293 18849 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:37.160251 18573 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:37.166618 18602 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:37.254402 18943 raft_consensus.cc:3055] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:37.254874 18943 raft_consensus.cc:740] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:37.194967 18777 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.255674 18943 consensus_queue.cc:260] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.256958 18943 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:37.257838 18777 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:37.169637 18907 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:37.260015 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:37.210620 15642 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:37.209647 18935 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:37.143399 18845 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.188535 18318 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:37.210989 18842 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.226130 18899 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:37.192281 18556 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:37.305023 18556 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:37.306223 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.307408 18556 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:37.170853 18698 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:37.308305 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.307512 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.203521 18877 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:37.261092 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 51176.7us
I20260709 13:19:37.251959 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:37.319824 15642 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:37.270279 18903 log.cc:927] Time spent T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 1.318s	user 0.010s	sys 0.010s
I20260709 13:19:37.264758 18879 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:37.271754 18874 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:37.273988 18602 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.274857 18439 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:37.276721 18676 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:37.349635 15601 raft_consensus.cc:1240] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:37.350811 18602 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:37.355041 15595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53324 (request call id 1794) took 1176 ms. Trace:
I20260709 13:19:37.355631 15595 rpcz_store.cc:276] 0709 13:19:36.178145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.178278 (+   133us) service_pool.cc:224] Handling call
0709 13:19:37.354798 (+1176520us) spinlock_profiling.cc:243] Waited 32.6 ms on lock 0x7b540016b554. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:37.354930 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":74918272}
I20260709 13:19:37.261792 18849 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.264755 18880 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.250842 18892 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:37.287247 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.288535 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:37.290239 15566 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:37.364383 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 74239.2us
I20260709 13:19:37.267445 18891 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.372468 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.373209 15669 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:37.326050 18939 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:37.309128 18440 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.378027 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 878) took 3456 ms. Trace:
I20260709 13:19:37.281673 18797 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:37.294214 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 100135us
I20260709 13:19:37.300616 16235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:52820 (request call id 4) took 1860 ms. Trace:
I20260709 13:19:37.380998 16235 rpcz_store.cc:276] 0709 13:19:35.439913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.440049 (+   136us) service_pool.cc:224] Handling call
0709 13:19:37.300405 (+1860356us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b5400621e54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a05853 00007f5ce99d360a 00007f5ce2f597d4 00007f5ce2f59762 00007f5ce2f59691 00007f5ce2f5960c 00007f5ce2f588c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:37.300515 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":252302208}
I20260709 13:19:37.303582 18923 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:37.304162 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:37.309729 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.252076 18573 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.306154 15596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 703) took 1297 ms. Trace:
I20260709 13:19:37.317353 16236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53976 (request call id 732) took 1126 ms. Trace:
I20260709 13:19:37.323484 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.386749 16204 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:37.386422 16236 rpcz_store.cc:276] 0709 13:19:36.190504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.190606 (+   102us) service_pool.cc:224] Handling call
0709 13:19:36.190857 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:36.923392 (+732535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:36.923420 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:36.923457 (+    37us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:36.923489 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:37.317115 (+393626us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x7b5400240ed4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:37.317166 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:37.317293 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1684314496}
I20260709 13:19:37.389089 18797 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:37.374920 18848 consensus_queue.cc:1059] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:19:37.265033 18945 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:37.386323 15596 rpcz_store.cc:276] 0709 13:19:36.008704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.017522 (+  8818us) service_pool.cc:224] Handling call
0709 13:19:36.050668 (+ 33146us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:36.255189 (+204521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:37.305899 (+1050710us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b5400439254. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:37.306084 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":468676992}
I20260709 13:19:37.396804 18676 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:37.417443 15676 raft_consensus.cc:2410] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:19:37.422483 18934 consensus_queue.cc:1059] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:37.423429 18676 raft_consensus.cc:3055] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:37.295918 18744 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:37.425978 18744 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:37.427138 16235 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:19:37.427737 16235 raft_consensus.cc:606] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:37.428073 16235 raft_consensus.cc:614] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:37.428252 18744 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:37.280172 18842 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:37.365957 15566 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:37.429455 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 63608.5us
I20260709 13:19:37.429541 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:37.431689 18784 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:37.368678 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:37.433187 18914 consensus_queue.cc:1048] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:37.267231 18780 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.434408 15566 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:37.435446 18780 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:37.308245 18698 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.373489 18899 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:37.290887 15421 catalog_manager.cc:5697] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 4 to 6, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 6 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:37.378499 15520 rpcz_store.cc:276] 0709 13:19:33.919111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.919248 (+   137us) service_pool.cc:224] Handling call
0709 13:19:33.940191 (+ 20943us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:33.940521 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.940549 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:33.940987 (+   438us) log.cc:844] Serialized 26 byte log entry
0709 13:19:35.474649 (+1533662us) spinlock_profiling.cc:243] Waited 3.8 ms on lock 0x7b1800015280. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab0d1 00007f5ce49c6a0f
0709 13:19:35.618420 (+143771us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:35.618464 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:35.619689 (+  1225us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:37.269667 (+1649978us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:19:37.375665 (+105998us) raft_consensus.cc:1685] finished
0709 13:19:37.375688 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:37.375821 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":76,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":9805,"spinlock_wait_cycles":13624320,"thread_start_us":143557,"threads_started":1,"wal-append.queue_time_us":1969518}
I20260709 13:19:37.368383 18892 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.437140 15522 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:37.437670 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:56840 (request call id 895) took 1202 ms. Trace:
I20260709 13:19:37.446931 15522 rpcz_store.cc:276] 0709 13:19:36.235474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.235603 (+   129us) service_pool.cc:224] Handling call
0709 13:19:37.436024 (+1200421us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b54003cf5d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a02a3d 00007f5ce99d2cd0 00007f5ce2f57254 00007f5ce2f571e2 00007f5ce2f57111 00007f5ce2f5708c 00007f5ce2f56342 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:37.437584 (+  1560us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":765932416}
I20260709 13:19:37.293897 18825 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:37.448211 15597 raft_consensus.cc:1240] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:37.424548 18880 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:37.290642 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:37.435698 18676 raft_consensus.cc:740] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:37.452224 16235 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:19:37.452824 16235 raft_consensus.cc:606] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:37.438407 18892 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:37.389879 15670 raft_consensus.cc:1240] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:37.439088 18440 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:37.440155 15600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53324 (request call id 1791) took 1378 ms. Trace:
I20260709 13:19:37.454692 15600 rpcz_store.cc:276] 0709 13:19:36.061871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.074022 (+ 12151us) service_pool.cc:224] Handling call
0709 13:19:37.439951 (+1365929us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b540030dc54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:37.440085 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":377746176}
I20260709 13:19:37.455345 18573 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:37.454382 15673 raft_consensus.cc:1240] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:37.433238 18854 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:19:37.441943 18883 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:37.439873 15421 catalog_manager.cc:5697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 3 to 6, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 6 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:37.462989 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:37.444334 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:37.463256 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 70423.7us
I20260709 13:19:37.463750 16236 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:37.445370 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.464295 16204 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:37.398669 18931 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.389367 18318 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.456516 18698 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:37.451803 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.453202 18676 consensus_queue.cc:260] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:37.460494 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 137055us
I20260709 13:19:37.461748 15566 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:37.462601 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.465953 15568 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:37.464677 15677 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:37.481871 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:37.484395 15566 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:37.482445 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:19:37.486721 16204 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:37.488288 16231 raft_consensus.cc:2393] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 12: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:37.489948 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.489542 15421 catalog_manager.cc:5697] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:37.490676 15671 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:37.490780 15566 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:37.462530 18883 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:37.487682 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:37.506661 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.513748 15421 catalog_manager.cc:5697] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 4 to 5, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 5 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:37.518417 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:37.332140 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 244783us
I20260709 13:19:37.489514 18604 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:37.477815 18732 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:19:37.495038 18892 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:37.495428 18473 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:37.526784 15642 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:37.522585 15421 catalog_manager.cc:5697] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 2 to 7, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 7 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:37.543928 15421 catalog_manager.cc:5697] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:37.499305 18676 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:37.499584 18880 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:37.514537 18883 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:37.556787 15566 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:37.514024 18744 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:37.557987 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:37.518200 18784 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:37.532837 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:37.568248 15596 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
W20260709 13:19:37.534818 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.568629 15642 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:37.537770 18946 log.cc:927] Time spent T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:19:37.546294 15567 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:37.566412 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:37.557904 15421 catalog_manager.cc:5697] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:37.571031 17744 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:37.571647 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:37.572096 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:37.586428 15527 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:37.573251 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:37.575543 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:37.577512 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:37.531528 18845 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:37.573041 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:37.542268 18824 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:37.592618 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:37.594264 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.595500 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:37.597524 16238 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1074) took 1377 ms. Trace:
I20260709 13:19:37.598599 18318 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:37.597946 16238 rpcz_store.cc:276] 0709 13:19:36.219816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.239526 (+ 19710us) service_pool.cc:224] Handling call
0709 13:19:36.240544 (+  1018us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:36.926982 (+686438us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:36.956731 (+ 29749us) log.cc:844] Serialized 14 byte log entry
0709 13:19:37.166911 (+210180us) log.cc:844] Serialized 14 byte log entry
0709 13:19:37.171219 (+  4308us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:37.171271 (+    52us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:37.171311 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:37.597311 (+426000us) spinlock_profiling.cc:243] Waited 686 ms on lock 0x7b54008507d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:37.597357 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:37.597468 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21019,"prepare.queue_time_us":83867,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":85,"raft.queue_time_us":71735,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":11381,"spinlock_wait_cycles":1579668992,"thread_start_us":236082,"threads_started":3,"wal-append.queue_time_us":203313}
W20260709 13:19:37.600203 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 54003.6us
I20260709 13:19:37.601289 15567 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:37.601853 18931 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:37.602449 15567 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:37.603434 15421 catalog_manager.cc:5697] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:37.604951 15421 catalog_manager.cc:5697] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 1 to 4, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 4 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:37.558135 18744 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:37.607952 18744 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:37.609007 18929 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:37.609731 18929 consensus_queue.cc:1059] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:37.611145 18929 raft_consensus.cc:3055] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:37.611538 18929 raft_consensus.cc:740] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:37.579198 18777 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:37.610248 18892 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:37.612448 18777 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:37.610235 18880 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:37.555699 18891 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:37.585069 18899 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:37.614017 15421 catalog_manager.cc:5697] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 5 to 8, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 8 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } }
I20260709 13:19:37.584754 18849 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:37.579311 18439 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:37.616144 18919 raft_consensus.cc:993] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9: : Instructing follower d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:19:37.597573 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:37.617192 18684 raft_consensus.cc:1081] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Signalling peer d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:19:37.619614 15597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 711) took 1400 ms. Trace:
I20260709 13:19:37.620039 15597 rpcz_store.cc:276] 0709 13:19:36.218810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.230836 (+ 12026us) service_pool.cc:224] Handling call
0709 13:19:37.447991 (+1217155us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:37.448677 (+   686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:37.619420 (+170743us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b540030dc50. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:37.619550 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2798586624}
I20260709 13:19:37.620458 16203 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:37.593741 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:37.570353 15642 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:37.619117 15567 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:37.615520 15421 catalog_manager.cc:5697] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 1 to 4, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 4 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:37.632292 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:19:37.682272 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 58847.4us
W20260709 13:19:37.682186 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 50000.8us
W20260709 13:19:37.682186 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 63136.9us
I20260709 13:19:37.683980 15567 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:37.685488 15567 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:37.644452 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:37.690090 15596 raft_consensus.cc:2393] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 6: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:37.646615 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:37.664517 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:37.698079 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:37.693802 15493 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:37.699012 15602 raft_consensus.cc:2393] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 12: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:37.706305 15527 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:56840
I20260709 13:19:37.706939 15527 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:37.707293 15527 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:37.709236 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:37.713048 15493 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:37.585222 18784 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:37.681299 18899 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:37.612376 18929 consensus_queue.cc:260] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:37.614729 18777 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:37.727859 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 84916.6us
I20260709 13:19:37.693719 18684 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:37.729128 18440 consensus_queue.cc:1059] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:37.692524 18880 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:37.730605 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:37.697031 18824 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:37.731513 18824 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:37.731912 18824 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:37.734463 15421 catalog_manager.cc:5697] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 2 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65), VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66), VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67), VOTER dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 4 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:37.736805 15421 catalog_manager.cc:5697] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:37.738245 18874 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:37.738786 18874 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:37.740984 18874 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:37.741465 18874 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:19:37.743204 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 172909us
I20260709 13:19:37.743810 18907 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:37.746097 18845 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:37.713775 18775 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:37.746937 18775 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:37.747295 18775 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:37.747527 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.748274 15676 raft_consensus.cc:2393] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 8: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:37.681134 18891 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:19:37.758076 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 44282.3us
I20260709 13:19:37.727026 15671 raft_consensus.cc:1240] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:37.696519 18892 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:37.730185 18440 raft_consensus.cc:3055] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:37.711663 18556 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:19:37.751642 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 53055.5us
I20260709 13:19:37.754601 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.729194 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:37.758870 16203 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:37.775045 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.775501 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.775581 18883 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:37.776298 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.776753 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.777499 15673 raft_consensus.cc:2393] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 5: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:37.777379 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.778285 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.778985 15598 raft_consensus.cc:2393] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 8: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:37.779423 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.780262 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:37.780499 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.781080 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:37.782017 18891 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:37.721686 18784 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:37.764936 15527 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.785542 16203 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:37.778228 15421 catalog_manager.cc:5697] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:37.790259 15421 catalog_manager.cc:5697] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 2 to 5, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 5 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:37.793128 18931 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:37.793686 18931 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:37.794081 18931 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:37.741905 18874 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:37.804956 18874 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:37.772326 18440 raft_consensus.cc:740] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:37.811034 18440 consensus_queue.cc:260] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.812747 18440 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:37.787916 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.789219 15600 raft_consensus.cc:1240] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:37.816567 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.792517 15421 catalog_manager.cc:5697] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 3 to 5, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:37.818372 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:37.777748 18883 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:37.854408 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 81732.1us
W20260709 13:19:37.854952 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 69502.2us
I20260709 13:19:37.864920 16203 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:37.782608 18929 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:37.856328 16237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53976 (request call id 736) took 1614 ms. Trace:
I20260709 13:19:37.903609 16237 rpcz_store.cc:276] 0709 13:19:36.241976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.365343 (+123367us) service_pool.cc:224] Handling call
0709 13:19:36.365616 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:37.183126 (+817510us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:37.822329 (+639203us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b5400640ad4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:37.856267 (+ 33938us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1859898624}
I20260709 13:19:37.859705 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:37.906659 15527 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:37.923053 15421 catalog_manager.cc:5697] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 2 to 5, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 5 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:19:37.853972 18892 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:37.751187 18899 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:37.929435 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 226347us
W20260709 13:19:37.939564 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 226558us
I20260709 13:19:37.945833 15567 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:37.946796 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.948405 15493 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:37.950788 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 85988.4us
I20260709 13:19:37.952333 16203 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:19:37.958844 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 252985us
I20260709 13:19:37.935386 15421 catalog_manager.cc:5697] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 2 to 5, term changed from 1 to 3, dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:37.901412 18318 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:37.906329 18780 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.973543 16203 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:37.971779 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.979317 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.925982 18825 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:37.983843 18825 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:37.985728 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:37.985905 15421 catalog_manager.cc:5697] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:37.997877 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:38.000355 15642 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:38.004354 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 58384.5us
I20260709 13:19:38.010826 15421 catalog_manager.cc:5697] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:19:38.013710 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 53400.7us
I20260709 13:19:37.973182 18848 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:38.015607 15568 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:37.981766 18780 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:38.021076 15421 catalog_manager.cc:5697] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 2 to 4, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 4 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:19:38.023752 15421 catalog_manager.cc:5697] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 2 to 5, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 5 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:37.963780 18892 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:38.010600 18601 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:38.013792 18950 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:38.082298 15672 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:19:38.083210 15421 catalog_manager.cc:5697] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:19:38.024166 18965 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
W20260709 13:19:38.085021 18960 log.cc:927] Time spent T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:19:38.086336 18899 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:38.089217 18784 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:38.090392 18825 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:37.986835 18736 consensus_queue.cc:1059] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.091s
I20260709 13:19:38.094309 18824 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:38.026863 18874 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:38.106999 18318 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:19:38.108639 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 108558us
I20260709 13:19:38.136870 18780 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:38.138635 15603 raft_consensus.cc:1240] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:38.138818 18945 consensus_queue.cc:1059] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:19:38.114286 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 98751.5us
I20260709 13:19:38.139822 18945 raft_consensus.cc:3055] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:38.140197 18945 raft_consensus.cc:740] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
W20260709 13:19:38.133739 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 113196us
I20260709 13:19:38.109258 18950 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:38.145614 15567 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:38.139344 18824 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:38.078980 18966 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:38.141830 15597 raft_consensus.cc:1240] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:38.142242 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:38.142462 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:38.143006 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:38.143206 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:38.149514 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:38.111567 18698 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:38.140851 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:38.144923 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:38.132675 18913 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:38.161831 15595 raft_consensus.cc:1240] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: []
W20260709 13:19:38.119313 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 159663us
I20260709 13:19:38.162343 15602 raft_consensus.cc:1240] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:38.109618 18931 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:38.151738 15596 raft_consensus.cc:1240] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:38.112268 18775 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:38.156294 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:38.164278 18972 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:38.180565 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:38.191636 15601 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
W20260709 13:19:38.164812 18962 log.cc:927] Time spent T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:19:38.167624 18899 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:38.169461 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:38.163457 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 66765.9us
W20260709 13:19:38.146553 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.190706 16237 raft_consensus.cc:2393] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 11: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
W20260709 13:19:38.199224 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 225759us
I20260709 13:19:38.227813 18780 consensus_queue.cc:1059] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:19:38.175165 18680 raft_consensus.cc:3055] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:38.231706 18680 raft_consensus.cc:740] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:38.231899 15642 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:38.232174 17744 consensus_queue.cc:1059] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:38.149009 18950 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:38.147367 18929 consensus_queue.cc:1048] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
W20260709 13:19:38.225849 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 262572us
I20260709 13:19:38.173851 18879 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:38.230643 16203 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:38.223986 15599 raft_consensus.cc:1240] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12. Ops: []
W20260709 13:19:38.226428 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 63594.5us
I20260709 13:19:38.239122 15493 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:38.246134 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 253969us
I20260709 13:19:38.248040 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:38.248674 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:38.249429 15668 raft_consensus.cc:2393] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 8: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:38.222306 18891 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:19:38.252113 15493 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:38.252998 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:38.254004 16237 raft_consensus.cc:2393] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 10: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:38.232196 18892 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:38.258723 18939 raft_consensus.cc:3055] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:38.259158 18939 raft_consensus.cc:740] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:38.260030 18939 consensus_queue.cc:260] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:38.261400 18939 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:38.262193 18849 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:38.262605 18849 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:38.262912 18849 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:38.180975 18951 consensus_queue.cc:1059] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 13:19:38.267380 18951 raft_consensus.cc:3055] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:38.267765 18951 raft_consensus.cc:740] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:38.268568 18951 consensus_queue.cc:260] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:38.235039 18899 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:38.253010 18965 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:19:38.253832 18744 consensus_meta.cc:287] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.645s	user 0.010s	sys 0.000s
I20260709 13:19:38.269886 18951 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:38.284925 15601 raft_consensus.cc:1240] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:38.272229 18899 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:38.289073 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 47023.2us
I20260709 13:19:38.290026 15493 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:38.272143 16203 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:38.296782 16203 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:38.317514 15491 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:38.318946 15491 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:38.320178 15491 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:38.274605 18874 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:38.273756 18775 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:38.325814 15493 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:38.236730 18945 consensus_queue.cc:260] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:38.279122 18931 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:38.328379 15491 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:38.326462 18982 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:38.273319 18848 consensus_queue.cc:1059] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:19:38.272327 18977 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:38.296038 18965 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:38.268440 18975 consensus_queue.cc:1059] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
W20260709 13:19:38.338449 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 77746.7us
I20260709 13:19:38.340404 15493 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:38.326387 18976 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:38.273119 18744 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:38.328552 18967 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:38.297940 18843 consensus_queue.cc:1059] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:19:38.328862 18944 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:38.344523 16203 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:38.346978 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:38.347354 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:38.332024 18684 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:38.363729 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:38.365463 15595 raft_consensus.cc:1240] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:38.365477 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:38.366356 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:38.367123 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:38.368417 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:38.350431 18824 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:38.369822 15491 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:38.340404 18874 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:38.356371 18473 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:38.343844 18919 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:38.379092 15675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2058) took 1202 ms. Trace:
I20260709 13:19:38.360571 18744 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:38.369100 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:38.350091 18318 consensus_queue.cc:1059] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:38.370019 15522 raft_consensus.cc:2368] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7.
I20260709 13:19:38.353964 18975 raft_consensus.cc:3055] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:38.370527 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:38.372037 18825 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:38.375342 18849 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:38.382576 18684 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:38.399963 18849 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:38.362299 18965 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:38.390503 18318 raft_consensus.cc:3055] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:38.376950 18601 raft_consensus.cc:3055] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:38.400866 18318 raft_consensus.cc:740] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:38.401180 18601 raft_consensus.cc:740] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:38.401371 18950 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:38.401896 18318 consensus_queue.cc:260] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:38.402927 18318 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:38.403474 18950 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:38.384336 18980 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:38.400826 15528 raft_consensus.cc:2393] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 7: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:38.404801 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:38.405275 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:38.379520 15675 rpcz_store.cc:276] 0709 13:19:37.176552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.176689 (+   137us) service_pool.cc:224] Handling call
0709 13:19:37.177027 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:37.177244 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:37.680858 (+503614us) log.cc:844] Serialized 14 byte log entry
0709 13:19:38.378392 (+697534us) spinlock_profiling.cc:243] Waited 6.69 ms on lock 0x7b18004a2fc0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1898c0a 00007f5ceafd8219 00007f5ce1920a40 00007f5ce191b4e4 00007f5ce191b455 00007f5ce84033c7 00007f5ce4a159a7 00007f5ce4a250bf 00007f5ce4a25038
0709 13:19:38.378611 (+   219us) log.cc:844] Serialized 14 byte log entry
0709 13:19:38.378779 (+   168us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:38.378823 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:38.378864 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.378933 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:38.379035 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33281,"prepare.queue_time_us":240501,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":18466,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":4308,"spinlock_wait_cycles":15394176,"thread_start_us":659602,"threads_started":2,"wal-append.queue_time_us":266999}
I20260709 13:19:38.386142 15523 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:19:38.407171 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:38.405403 15594 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:38.390430 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:38.400213 18684 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:38.385831 18975 raft_consensus.cc:740] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:38.415527 18784 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:38.417995 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:38.419596 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:38.420176 15524 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:19:38.393599 15491 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:38.422655 15491 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:38.425580 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:38.426704 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:38.426730 15594 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:38.428718 15603 raft_consensus.cc:2393] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 7: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:38.430146 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:38.430812 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:38.431270 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:38.430773 16238 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:19:38.413055 18975 consensus_queue.cc:260] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:38.432691 18975 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:38.425772 18964 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:38.435444 15491 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:38.443068 18967 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.450526 18967 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:38.452620 18967 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:38.457993 18965 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:38.443246 18850 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.462934 18850 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:38.463940 16240 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:19:38.464395 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:38.464995 18850 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:38.467844 15669 raft_consensus.cc:1240] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: []
W20260709 13:19:38.470583 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.473310 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:38.474166 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:38.474944 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:38.475292 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:38.476040 15668 raft_consensus.cc:2393] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 5: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:38.476620 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:38.477859 15675 raft_consensus.cc:1240] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:38.478888 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:38.479565 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:38.480000 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:38.483050 15523 raft_consensus.cc:1240] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:38.485309 15522 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:19:38.486488 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:38.486598 15491 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:38.489251 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:38.491739 16203 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:38.476831 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:38.495607 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:38.496212 15668 raft_consensus.cc:2393] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 7: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:38.498955 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:38.496778 18924 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:38.492204 18967 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:38.583226 15491 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:38.618835 15641 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:38.620025 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:38.505394 16236 raft_consensus.cc:1240] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:38.512183 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:38.505481 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:38.530803 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:38.621491 15521 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:38.620615 15491 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:38.622335 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:38.532464 17744 raft_consensus.cc:3055] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:38.581725 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:38.624056 16239 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:19:38.526572 16233 raft_consensus.cc:1240] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:38.627652 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:38.586668 18602 raft_consensus.cc:3055] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:38.523486 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:38.594349 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:38.613509 15567 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:38.587662 15600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 1800) took 1035 ms. Trace:
I20260709 13:19:38.631199 15596 raft_consensus.cc:2393] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 11: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:38.631253 15600 rpcz_store.cc:276] 0709 13:19:37.551850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.551974 (+   124us) service_pool.cc:224] Handling call
0709 13:19:37.552280 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:37.814289 (+262009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.587418 (+773129us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b540009e554. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:38.587594 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":531783936}
I20260709 13:19:38.633085 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:38.631733 15567 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:38.610869 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:38.635396 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:38.504652 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:38.637244 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:38.637498 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:38.623996 17744 raft_consensus.cc:740] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:38.638206 16240 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:19:38.637507 15567 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:38.638983 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:38.639106 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:38.639667 15567 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:38.581292 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:38.640548 15520 raft_consensus.cc:2393] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 7: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:38.614676 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:38.675834 15603 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:38.580551 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:38.504897 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:38.614753 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:38.677251 15599 raft_consensus.cc:2393] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 4: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:38.494421 18825 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:38.618382 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:38.512632 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:38.620586 15641 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:38.678141 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 57586.9us
I20260709 13:19:38.678915 15491 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:38.502959 18874 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:38.621407 18935 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:38.592993 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:38.623582 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:38.687286 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:38.687841 16232 raft_consensus.cc:3055] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:38.688064 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:38.688685 16236 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:38.592429 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:38.637029 18784 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:38.641707 18984 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:38.631482 18602 raft_consensus.cc:740] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:38.588714 18680 consensus_queue.cc:260] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:38.637637 18744 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:38.691510 18825 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:38.621546 18907 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:38.692417 18602 consensus_queue.cc:260] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:38.693641 18602 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:38.646605 15567 leader_election.cc:400] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7.
I20260709 13:19:38.696081 15567 leader_election.cc:403] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
W20260709 13:19:38.696554 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 50162.8us
I20260709 13:19:38.649319 15642 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:19:38.697291 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 48039us
W20260709 13:19:38.691051 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 70525.8us
I20260709 13:19:38.698172 18825 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:38.698575 15641 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:38.698801 15642 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:38.691231 18842 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:38.654212 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:38.699754 18845 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:38.699924 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:38.700253 18698 consensus_queue.cc:1059] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:38.699890 18842 consensus_queue.cc:1059] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:38.654819 17744 consensus_queue.cc:260] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:38.655268 15524 raft_consensus.cc:1240] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:38.701305 18842 consensus_queue.cc:1059] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:38.701560 18951 raft_consensus.cc:3055] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:38.641199 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:38.655623 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:38.702195 18951 raft_consensus.cc:740] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:38.645614 18965 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:38.657908 18963 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:38.659694 18945 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:38.703080 18951 consensus_queue.cc:260] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:38.660076 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:38.704176 16235 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:38.704581 15525 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:38.705483 16235 raft_consensus.cc:1240] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:19:38.501616 18850 consensus_queue.cc:1059] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.128s
I20260709 13:19:38.702198 18845 raft_consensus.cc:697] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:38.707381 18439 raft_consensus.cc:3055] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:38.707739 18439 raft_consensus.cc:740] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:38.674340 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:38.707875 18845 consensus_queue.cc:237] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:38.701092 18698 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:38.703202 18963 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:38.714700 18963 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:38.702950 17744 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:38.701143 18797 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:38.674093 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:38.681376 15601 raft_consensus.cc:2393] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 7: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:38.682454 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:38.677594 18976 consensus_queue.cc:1059] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:19:38.686271 18967 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:38.687005 18982 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:38.750898 18907 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:38.691134 15603 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:19:38.714123 18698 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:38.752609 18439 consensus_queue.cc:260] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:38.753484 18698 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:38.754102 18439 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:38.704246 18951 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:38.699458 15641 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:38.728971 15677 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:19:38.688104 16232 raft_consensus.cc:740] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:38.734202 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:38.755023 18965 raft_consensus.cc:3055] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:19:38.756412 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 76326.1us
W20260709 13:19:38.674090 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 55384.1us
I20260709 13:19:38.719066 18601 consensus_queue.cc:260] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:38.707644 16231 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:38.746032 15528 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:38.774416 17744 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:38.739476 15642 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:38.775270 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 75890us
I20260709 13:19:38.775959 15642 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:38.733601 16236 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:19:38.772991 15491 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:38.779701 15493 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:38.782572 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:38.782465 15568 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:38.782932 17744 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:38.784431 15493 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:38.698577 18680 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:38.785171 16232 consensus_queue.cc:260] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:38.755542 18945 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:38.756466 18744 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7.
I20260709 13:19:38.787598 16232 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:38.796898 15566 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:38.798195 15493 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:38.797217 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:38.800806 15642 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:38.774344 18965 raft_consensus.cc:740] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:38.781483 18797 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:38.805507 17744 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:38.805732 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:38.751158 18963 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:38.786953 18880 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:38.811864 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:38.787058 18825 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:38.802873 16204 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:38.785785 18935 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:38.805507 15493 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:38.815307 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:38.814860 15525 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:38.816210 15566 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:38.799762 16233 raft_consensus.cc:1240] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:38.800467 18601 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:38.805730 18945 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:38.801110 18964 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:38.786948 18976 consensus_queue.cc:1059] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:19:38.799785 18988 consensus_queue.cc:1059] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.074s
W20260709 13:19:38.874995 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 69562.2us
I20260709 13:19:38.807520 18883 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:38.884545 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 68396us
I20260709 13:19:38.870997 15520 raft_consensus.cc:1240] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:38.811900 18824 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:38.808192 18980 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:38.808019 18939 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:38.868741 18736 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:38.812285 18982 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:38.892403 15493 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:38.899219 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:38.870649 18880 raft_consensus.cc:697] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:38.892716 18854 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:38.808452 18979 consensus_queue.cc:1059] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 13:19:38.909487 18963 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:19:38.885535 18907 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:38.920159 18907 raft_consensus.cc:697] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:38.921094 18907 consensus_queue.cc:237] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:38.922230 18883 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:38.922900 18945 raft_consensus.cc:3055] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:38.923313 18945 raft_consensus.cc:740] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:38.924055 18945 consensus_queue.cc:260] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:38.925325 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:38.873886 18965 consensus_queue.cc:260] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:38.927330 18965 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:38.934155 15603 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:38.906806 16232 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:19:38.817448 18966 consensus_queue.cc:1059] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 13:19:38.808354 18950 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 13:19:38.916213 18980 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:38.918763 18880 consensus_queue.cc:237] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:38.901147 18775 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:38.904000 18824 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:38.942233 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 56895.4us
I20260709 13:19:38.912392 18854 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:19:38.948841 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.956748 18980 raft_consensus.cc:697] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
W20260709 13:19:38.962028 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:38.966863 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 151551us
I20260709 13:19:38.961099 18698 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:38.977988 16234 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:38.980301 15601 raft_consensus.cc:1240] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9. Ops: []
W20260709 13:19:38.982144 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.982252 16204 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:38.956980 18775 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:38.991510 18775 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:38.962260 18854 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:38.904040 18939 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.012794 15669 raft_consensus.cc:1240] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:38.972414 18940 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:39.018402 18824 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:39.019346 18849 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:39.019675 18950 consensus_queue.cc:1059] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:39.021663 18945 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:39.027469 18924 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:39.029773 16238 raft_consensus.cc:1240] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:39.014901 18602 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:38.994560 18980 consensus_queue.cc:237] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.026294 18940 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.058984 18698 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:39.061450 16237 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:19:39.067986 18939 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:39.079744 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:39.068547 18602 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.086709 17744 raft_consensus.cc:697] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:39.088546 15670 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:19:39.093022 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:39.080901 18935 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:39.089954 18602 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:39.095014 18825 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:19:39.150446 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 251002us
W20260709 13:19:39.154274 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.163647 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:39.164378 16232 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 10. Current term is 11. Ops: []
W20260709 13:19:39.166765 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.167043 15493 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:39.167730 17744 consensus_queue.cc:237] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.169974 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:39.172703 15493 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:39.189790 18698 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:39.194710 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:39.114952 18849 raft_consensus.cc:3055] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:39.207449 18964 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:39.207532 18976 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:39.208627 18964 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:39.209313 18964 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:39.107010 18935 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.211414 18994 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:39.213287 18935 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:39.218091 18849 raft_consensus.cc:740] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:39.219715 18849 consensus_queue.cc:260] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:39.228096 18975 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.231323 18964 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.209852 18999 consensus_queue.cc:1059] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:39.229137 18975 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.237084 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:39.241603 18602 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.243803 18854 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.257539 18775 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.261641 18964 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:39.221786 18849 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:39.274204 15528 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:19:39.281955 18602 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:39.285740 18883 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.286754 18883 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:39.287784 18775 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:39.290233 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:39.290907 18955 raft_consensus.cc:3055] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:39.291484 18955 raft_consensus.cc:740] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:39.293933 18967 raft_consensus.cc:3055] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:39.294468 18967 raft_consensus.cc:740] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:39.289736 18883 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:39.295796 18967 consensus_queue.cc:260] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:39.292933 18955 consensus_queue.cc:260] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.291046 16232 raft_consensus.cc:2368] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9.
I20260709 13:19:39.300547 18967 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:39.310767 19003 raft_consensus.cc:3055] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:39.316665 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:39.318334 18940 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:39.324882 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:39.326704 18892 raft_consensus.cc:3055] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:39.328141 15668 raft_consensus.cc:1240] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:39.329533 15674 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:19:39.332468 18933 raft_consensus.cc:3055] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:39.332638 18892 raft_consensus.cc:740] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:39.333029 18933 raft_consensus.cc:740] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:39.334084 18892 consensus_queue.cc:260] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.334444 18933 consensus_queue.cc:260] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.335420 18892 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:39.336397 18933 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:39.300661 18955 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:39.342389 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:39.344823 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:39.345667 15670 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:39.349810 18854 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:39.351320 15566 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:39.353509 18966 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:39.354210 18966 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.354714 18966 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:39.362802 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.328794 19003 raft_consensus.cc:740] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:39.383651 18854 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.384532 18854 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.386019 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:39.392429 15527 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:19:39.366142 19003 consensus_queue.cc:260] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.397985 19003 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:39.398852 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:39.398216 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:39.411947 15521 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:39.414285 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 45786.9us
I20260709 13:19:39.415448 15642 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:39.417078 18439 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:39.420822 18968 consensus_queue.cc:1059] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:39.422051 18968 raft_consensus.cc:3055] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:39.422452 18968 raft_consensus.cc:740] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:39.427402 18975 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:39.429463 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:39.429754 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:39.432828 18968 consensus_queue.cc:260] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.434814 18968 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:39.438834 15567 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:39.439998 18784 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:39.468180 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:39.469154 15672 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:39.470948 18854 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:39.480080 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:39.517657 16205 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:39.461340 18883 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.519376 18975 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:39.518813 18883 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.524183 18883 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:39.529755 15642 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:39.534876 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:39.563156 16232 raft_consensus.cc:2393] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 10: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
W20260709 13:19:39.536568 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.541982 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:39.565821 18842 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:39.517665 15641 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:39.566673 18842 raft_consensus.cc:697] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:39.567150 17744 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:39.567826 18842 consensus_queue.cc:237] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:39.568521 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 51130.6us
I20260709 13:19:39.570416 15641 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:39.571327 17744 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:39.583854 15566 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:39.590593 15521 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:19:39.549665 18784 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:39.550341 15596 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:39.551128 15668 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:19:39.594945 15674 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:19:39.566001 15642 leader_election.cc:400] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9.
I20260709 13:19:39.609674 15642 leader_election.cc:403] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:39.596714 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:39.619629 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:39.622735 15566 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:39.623695 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:39.625393 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:39.625593 17744 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:39.625320 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 59576.2us
I20260709 13:19:39.628847 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:39.626089 17744 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:39.635663 15528 raft_consensus.cc:1240] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9. Ops: []
W20260709 13:19:39.636777 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.627394 15602 raft_consensus.cc:1275] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:19:39.631356 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:39.641213 15520 raft_consensus.cc:2368] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9.
I20260709 13:19:39.630846 18849 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9.
I20260709 13:19:39.640658 18784 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:39.653087 15601 raft_consensus.cc:1240] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:39.657531 15524 raft_consensus.cc:1275] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:19:39.649290 18891 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:39.665802 15603 raft_consensus.cc:1240] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:39.665652 16234 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:19:39.662878 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:39.675074 16233 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:39.676934 15598 raft_consensus.cc:1240] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:39.664194 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:39.678007 15668 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:39.664078 18891 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.678509 16236 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:39.678776 15423 catalog_manager.cc:5697] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:39.663028 16204 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:39.685034 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 56856.6us
I20260709 13:19:39.686110 15567 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:39.686668 18744 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.687845 18963 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:39.688971 15568 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:39.689381 18891 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:39.689975 17744 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:39.666971 18842 consensus_queue.cc:1048] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:19:39.690934 18744 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.721309 18854 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:39.662925 18981 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:39.722426 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 102004us
I20260709 13:19:39.730264 18975 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:39.663496 18797 consensus_queue.cc:1059] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 13:19:39.733552 18797 raft_consensus.cc:3055] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:39.733955 18797 raft_consensus.cc:740] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:39.735117 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:39.735749 18975 raft_consensus.cc:697] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:39.736330 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:39.736968 16237 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:39.738143 18981 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:39.732520 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 134542us
I20260709 13:19:39.734848 18797 consensus_queue.cc:260] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:39.716940 18883 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:39.744513 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:39.736871 18975 consensus_queue.cc:237] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.746685 18676 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:39.749468 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 83040.2us
I20260709 13:19:39.757170 15603 raft_consensus.cc:2393] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 11: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:39.761286 18676 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.763839 18676 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:39.755324 18883 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.766360 15567 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:39.766749 18883 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:39.778529 16205 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:39.775801 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:39.658900 18966 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:39.788993 15602 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:39.780560 16237 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:39.791460 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:39.780906 15423 catalog_manager.cc:5697] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 2 to 5, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 5 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:39.805586 15568 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:39.761286 18744 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:39.788993 18784 raft_consensus.cc:697] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:39.818948 15528 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:39.802488 18854 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:19:39.826112 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 172523us
I20260709 13:19:39.835080 16204 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:39.839087 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 72869us
I20260709 13:19:39.840940 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:39.840575 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:39.852098 15520 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:39.840633 18797 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:39.857506 15493 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:39.845574 18676 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:39.868402 15521 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:39.871696 18732 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:39.872197 18732 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.872534 18732 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:39.873975 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:39.874236 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:39.875139 15597 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:39.882408 18907 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:39.886222 18880 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.887063 18880 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.891363 18907 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:39.892086 18907 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:39.895994 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:39.898056 18907 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:39.906409 18907 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:39.907328 18981 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:39.907485 18907 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:39.910948 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 66061.1us
I20260709 13:19:39.911405 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:39.911984 15641 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:39.914116 18849 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:39.914707 18933 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.915318 18933 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.917579 18933 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:39.918967 18907 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:39.921962 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:39.922600 16237 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:39.926689 15669 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:39.908273 18784 consensus_queue.cc:237] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.928836 15423 catalog_manager.cc:5697] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 3 to 6, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 6 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:39.867134 18676 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.938465 18676 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:39.943192 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:39.943480 15421 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49140 (request call id 360) took 19371 ms (client timeout 15000 ms). Trace:
I20260709 13:19:39.861585 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:39.945459 15423 catalog_manager.cc:5697] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:39.948433 15423 catalog_manager.cc:5697] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 3 to 6, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 6 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:39.951306 18966 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:39.952142 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:39.958899 18676 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.961958 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:39.962592 15521 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:19:39.959268 15423 catalog_manager.cc:5697] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 3 to 7, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 7 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:39.963444 18880 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:39.851502 19013 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:39.965580 19013 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.966001 19013 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:39.967482 15423 catalog_manager.cc:5697] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 3 to 6, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 6 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:39.957444 18883 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:39.943863 15421 rpcz_store.cc:269] 0709 13:19:20.572072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:20.572215 (+   143us) service_pool.cc:224] Handling call
0709 13:19:20.651840 (+ 79625us) spinlock_profiling.cc:243] Waited 66.6 ms on lock 0x7b5400369554. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4975090 00007f5ce4a13f3a 00007f5cea7ca1cc 00007f5cea8c837e 00007f5ce8d892a4 00007f5ce8d89232 00007f5ce8d89161 00007f5ce8d890dc 00007f5ce8d88392 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:39.943275 (+19291435us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:38.376812 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:38.377394 (+   582us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:38.377426 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:19:38.377446 (+    20us) tablet.cc:662] Decoding operations
0709 13:19:38.379054 (+  1608us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:38.379137 (+    83us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:38.379164 (+    27us) tablet.cc:685] Acquiring locks for 23 operations
0709 13:19:38.380813 (+  1649us) tablet.cc:701] Row locks acquired
0709 13:19:38.380841 (+    28us) write_op.cc:260] PREPARE: finished
0709 13:19:38.380979 (+   138us) write_op.cc:270] Start()
0709 13:19:38.381090 (+   111us) write_op.cc:276] Timestamp: P: 1783603178380936 usec, L: 0
0709 13:19:38.381109 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:19:38.407956 (+ 26847us) log.cc:844] Serialized 9111 byte log entry
0709 13:19:38.907989 (+500033us) op_driver.cc:464] REPLICATION: finished
0709 13:19:39.112620 (+204631us) write_op.cc:301] APPLY: starting
0709 13:19:39.112698 (+    78us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:39.113093 (+   395us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:39.113215 (+   122us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:39.121839 (+  8624us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:39.122269 (+   430us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=23
0709 13:19:39.122300 (+    31us) write_op.cc:312] APPLY: finished
0709 13:19:39.122580 (+   280us) log.cc:844] Serialized 158 byte log entry
0709 13:19:39.122724 (+   144us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:39.122854 (+   130us) write_op.cc:454] Released schema lock
0709 13:19:39.122893 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:39.123182 (+   289us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:39.123208 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":20906,"reported_tablets":43,"spinlock_wait_cycles":153127680,"child_traces":[["op",{"apply.queue_time_us":200241,"apply.run_cpu_time_us":4031,"apply.run_wall_time_us":10571,"num_ops":23,"prepare.queue_time_us":189232,"prepare.run_cpu_time_us":63922,"prepare.run_wall_time_us":207678,"raft.queue_time_us":222665,"raft.run_cpu_time_us":9366,"raft.run_wall_time_us":117450,"replication_time_us":526742,"thread_start_us":535177,"threads_started":4,"wal-append.queue_time_us":218733}]]}
I20260709 13:19:39.969856 15521 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:19:39.972229 18933 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:40.048184 18933 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.048529 18933 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:40.048830 16204 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:40.049686 18989 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:40.050264 18989 raft_consensus.cc:697] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:40.051203 18989 consensus_queue.cc:237] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.052737 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:40.053349 15598 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:39.973567 15641 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:39.973766 18601 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.978667 18854 raft_consensus.cc:697] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:40.055653 18601 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:39.986075 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 113425us
W20260709 13:19:40.057026 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 83559.1us
I20260709 13:19:40.057170 15493 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:40.055089 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:40.058074 15669 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:19:40.060756 15598 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:19:40.061971 18601 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:39.988426 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 182949us
I20260709 13:19:40.063673 17744 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:40.064296 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:40.065032 16237 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:19:40.065358 16238 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:40.066881 17744 consensus_queue.cc:1059] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:40.067564 16236 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:40.067924 18849 raft_consensus.cc:3055] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:40.068341 18849 raft_consensus.cc:740] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:40.068608 18880 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:40.056296 18854 consensus_queue.cc:237] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:40.069273 18849 consensus_queue.cc:260] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.069746 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:40.070451 16233 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:19:40.071395 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:40.071972 16234 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:40.068675 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:40.074280 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:40.074687 18849 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:40.075546 15596 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:40.075691 15601 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:40.077780 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:40.078800 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:40.069229 18880 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.080240 16237 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:40.080760 18880 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:40.064246 18676 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:40.071846 15598 consensus_queue.cc:237] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.083391 18933 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.084008 18967 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:40.085053 19023 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:40.086553 19023 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:40.086690 15642 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:40.087109 19023 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.087759 18842 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:40.088197 18842 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.088655 18842 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:40.089298 15669 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:40.040791 19020 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.105716 18945 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 13:19:40.094280 18907 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:40.114791 19020 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:39.977739 15423 catalog_manager.cc:5697] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:40.074299 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:40.099468 15493 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:40.116405 15594 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:19:40.091202 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:40.003273 18891 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.032244 18797 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.986804 18955 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.117574 19020 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:40.088276 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:40.098330 18842 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:40.074936 16232 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:19:40.118257 18797 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.092511 19023 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:40.119692 19023 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:40.120606 18797 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:40.120316 15423 catalog_manager.cc:5697] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:40.089838 15568 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:40.121620 19031 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:40.120677 18842 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:40.118000 18955 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.129199 15568 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:40.130004 19020 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:40.124150 19017 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:40.130813 19017 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.115785 18907 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:40.131237 19017 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:40.132087 16234 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:40.117800 16238 raft_consensus.cc:2393] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 7: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:40.133051 15493 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:39.997493 18732 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.135321 18732 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:40.135738 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:40.136459 15677 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:40.138247 16203 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:40.139956 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:40.140333 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:40.140677 15596 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:40.141043 16237 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:19:40.126955 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:40.127408 15674 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:19:40.130407 19020 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.148687 16233 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:40.148824 19020 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:40.095574 18880 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:19:40.119058 18891 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:40.141650 18676 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:40.159600 19020 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.160292 18676 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.139991 16205 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:40.090713 15641 leader_election.cc:400] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9.
W20260709 13:19:40.163628 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 73055.7us
I20260709 13:19:40.164695 18842 consensus_queue.cc:1048] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:40.160954 19031 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:40.166348 19020 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:40.166591 18684 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:40.144837 19017 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.017612 19003 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.130544 18955 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:40.146725 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:40.167824 19003 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:40.147539 18907 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:40.120193 19023 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:40.148182 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:40.167145 18684 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.169536 18684 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:40.169582 15525 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:40.171972 19023 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:40.172106 19017 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:40.175324 15568 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:40.157617 16233 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:19:40.178573 15596 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:40.030069 18883 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:40.190321 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:40.190482 15525 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:40.190956 16234 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:40.192145 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:40.192705 16238 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:40.192830 15566 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:40.193704 19021 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:40.010880 19013 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.194290 19021 raft_consensus.cc:697] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:40.194890 19018 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:40.194407 15568 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:40.195431 19018 raft_consensus.cc:697] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:40.195788 19013 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:40.195802 15568 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:40.196360 19018 consensus_queue.cc:237] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.197512 19013 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:40.196055 18981 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:40.199723 19013 raft_consensus.cc:697] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:40.199465 18684 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.200628 19013 consensus_queue.cc:237] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.201606 18684 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:40.202081 15597 raft_consensus.cc:2393] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 8: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:40.202009 18981 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.202646 18981 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:40.206516 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:40.206791 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:40.208051 16234 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:19:40.207502 15674 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:40.211301 16205 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:40.214367 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:40.215023 15668 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:40.215188 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:40.215696 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 103847us
I20260709 13:19:40.160924 18676 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:40.219413 15568 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:40.219663 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 84703.7us
I20260709 13:19:40.197618 18933 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:40.224207 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:40.225423 15677 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:40.226112 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:40.226745 16233 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:40.226446 15493 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:40.235673 15493 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:40.236568 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:40.235743 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:40.238206 16232 raft_consensus.cc:2393] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 13: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:40.218081 19003 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:40.218686 18968 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.240806 15594 raft_consensus.cc:2393] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 8: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:40.246106 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:40.244136 15566 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:40.247915 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:40.248116 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:40.249997 15597 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:40.222271 18732 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.224236 18883 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:40.252914 16233 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:19:40.254536 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:40.255227 15670 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:19:40.255318 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:40.256067 15669 raft_consensus.cc:2393] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 8: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:40.257416 19033 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:40.258262 19033 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:40.259244 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:40.199574 19021 consensus_queue.cc:237] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:40.226112 18967 raft_consensus.cc:697] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:40.267390 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:40.269769 16238 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:19:40.276571 16236 raft_consensus.cc:2393] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 8: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:40.282658 18955 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.268395 18854 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:40.283941 18854 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:40.284345 18854 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:40.237473 18907 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:19:40.256376 18975 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:40.287601 18975 raft_consensus.cc:697] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:40.288552 18975 consensus_queue.cc:237] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.283258 18955 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.304258 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:40.304967 15670 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
W20260709 13:19:40.309161 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 73585.4us
I20260709 13:19:40.298826 15668 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:19:40.315495 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:40.318265 15528 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:19:40.314556 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:40.319917 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:40.320802 15677 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:19:40.323520 15568 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:40.325754 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:40.326403 15520 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:40.329411 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:40.256193 19003 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:19:40.266089 18732 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.255857 18968 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.343250 15568 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:40.318665 18883 raft_consensus.cc:697] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:40.347474 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:40.336679 18797 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:40.351019 15527 raft_consensus.cc:1275] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:40.349805 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:40.354353 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:40.348634 18684 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:40.388167 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 63134.4us
I20260709 13:19:40.388648 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:40.397269 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 48459.7us
I20260709 13:19:40.347235 18891 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:19:40.437438 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 71589.7us
I20260709 13:19:40.439420 15525 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:19:40.460721 15597 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:40.437943 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:40.458078 15528 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:40.461861 15493 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:40.463126 15642 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:40.391733 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:40.474398 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:40.357726 18892 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:40.345388 19027 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:40.395263 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:40.482427 15602 raft_consensus.cc:2393] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 9: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
W20260709 13:19:40.396049 19025 log.cc:927] Time spent T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:19:40.484267 18842 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:40.488245 15668 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:40.357292 18684 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.499753 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:40.499310 15527 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:40.388535 19030 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:40.489267 18842 raft_consensus.cc:697] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:40.529070 15491 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:19:40.529807 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 212637us
I20260709 13:19:40.396749 18854 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.493832 19040 raft_consensus.cc:2764] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:19:40.499359 18891 raft_consensus.cc:697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:40.534013 16238 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:19:40.535939 18842 consensus_queue.cc:237] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:40.542210 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 80406.9us
W20260709 13:19:40.542984 19041 log.cc:927] Time spent T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:19:40.545249 15520 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:40.438299 18967 consensus_queue.cc:237] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.572871 15491 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:40.593578 15493 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:40.494361 19027 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.546119 18676 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.625572 19040 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:40.575233 19030 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:40.547490 19003 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:40.574193 18883 consensus_queue.cc:237] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.642163 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:40.662499 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 89714.6us
I20260709 13:19:40.618971 18907 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
W20260709 13:19:40.652245 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 144592us
I20260709 13:19:40.627302 18933 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:19:40.731971 16203 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:40.639719 19040 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.641510 19023 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:40.640645 18892 raft_consensus.cc:697] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:40.653038 18968 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:40.747624 15493 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:19:40.908654 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 161119us
I20260709 13:19:40.867527 19058 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.708124 19003 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:40.856029 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:40.869410 18981 consensus_meta.cc:287] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.666s	user 0.011s	sys 0.003s
I20260709 13:19:40.772440 19039 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.772253 15594 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:19:40.748530 19040 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:40.897822 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:40.898279 19050 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:40.899204 19049 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.899698 18684 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:40.900156 18892 consensus_queue.cc:237] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.736002 18891 consensus_queue.cc:237] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:40.885578 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 153649us
I20260709 13:19:40.958124 16237 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:40.959105 19052 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:19:40.959436 19042 log.cc:927] Time spent T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.249s	user 0.000s	sys 0.001s
I20260709 13:19:40.960237 16234 raft_consensus.cc:1275] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:40.917611 19003 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:40.962527 19064 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:40.963847 18676 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:40.954965 18981 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.965886 16238 raft_consensus.cc:1275] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:40.966353 18854 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:40.966485 19059 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:40.951535 19058 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:40.973362 15523 raft_consensus.cc:1275] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:19:40.959254 19035 log.cc:927] Time spent T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.400s	user 0.001s	sys 0.001s
I20260709 13:19:40.969318 15599 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:40.995312 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:40.999533 16237 raft_consensus.cc:2393] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 7: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:41.010843 15598 raft_consensus.cc:2393] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 13: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:41.012234 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:40.988951 18907 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.004184 15526 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:40.991166 18684 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:41.024266 15677 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:41.032892 16238 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:19:41.034834 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 58787.3us
I20260709 13:19:40.992435 19050 raft_consensus.cc:697] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:41.018220 19017 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:41.022171 15670 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:41.022013 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:41.023310 18955 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:41.024326 15599 raft_consensus.cc:1275] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:19:41.025279 15669 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:40.997643 19027 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:40.975270 19049 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:41.036846 15519 raft_consensus.cc:1275] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:19:41.046718 19049 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:41.047690 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 60497us
I20260709 13:19:41.049839 18933 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:41.050514 18933 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.052767 18933 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:41.053946 18732 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:41.037067 18684 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:40.999396 19030 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:40.971873 19039 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:41.073151 19056 log.cc:927] Time spent T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:19:41.073421 19039 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:41.041714 19059 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.989435 19064 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.078862 16205 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:41.081362 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:41.082973 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:41.083096 15599 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:41.002651 19021 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 13:19:41.086292 15668 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:19:41.101908 15528 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:41.109266 19073 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:41.109982 19073 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.042292 18968 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:19:41.114691 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:41.036242 18907 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.127874 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:41.121718 15673 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:41.118256 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:41.118703 16205 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:41.187023 18684 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:41.187327 19068 log.cc:927] Time spent T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:19:41.187603 19038 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.099974 19059 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.061805 19013 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:19:41.189805 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:41.198230 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:41.191144 19073 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:41.197209 16234 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:19:41.119503 15519 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:41.077155 19036 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:41.240903 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 45115.5us
I20260709 13:19:41.197129 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:41.192885 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:41.243019 15668 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:41.272961 16238 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:19:41.275038 15568 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:41.077008 19069 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.116s
I20260709 13:19:41.197597 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:41.215456 15677 raft_consensus.cc:1275] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:19:41.199391 19064 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:41.245038 18842 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:41.199182 19070 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:41.252375 19058 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:41.130226 18602 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:41.254849 15525 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:41.263326 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:41.264487 15528 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:19:41.272941 15602 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 8: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:41.244724 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:41.285820 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:41.255411 18967 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.289863 16236 raft_consensus.cc:2368] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 7. Current term is 8.
I20260709 13:19:41.290181 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:41.323874 16238 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:41.322261 15674 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:19:41.252115 18907 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:41.307611 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:41.212427 19020 consensus_queue.cc:1048] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
W20260709 13:19:41.314422 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 198336us
I20260709 13:19:41.287410 18933 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.316478 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:41.335843 16233 raft_consensus.cc:2393] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 8: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:41.313411 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:41.270159 19031 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.327260 15669 raft_consensus.cc:2410] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:19:41.338063 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:41.339229 15597 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:19:41.340549 16205 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:41.261624 19030 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:19:41.206230 19050 consensus_queue.cc:237] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.320920 18602 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.349383 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:41.354423 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:41.280181 19039 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:19:41.323071 18967 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.333123 19069 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:19:41.367532 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 54853.4us
I20260709 13:19:41.261991 19038 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.371537 15493 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:41.372802 15493 leader_election.cc:400] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 7. Current term is 8.
I20260709 13:19:41.373109 15493 leader_election.cc:403] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:41.376376 15668 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:41.363937 18602 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:41.364053 19084 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.379437 15528 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:41.385620 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:41.390365 15523 raft_consensus.cc:2393] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 10: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:41.351789 18981 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:41.356020 18797 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.320116 18842 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.285494 19036 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.347486 18676 consensus_queue.cc:1048] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:19:41.346464 18907 consensus_queue.cc:1059] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:19:41.344980 18933 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.330789 19070 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.377981 19069 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:41.409986 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:41.413970 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:41.368125 19039 consensus_queue.cc:1243] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer d3c4f5bd76e0445795bad5f6dd93ff69 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:19:41.458130 15671 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 10: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:41.461541 15525 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:19:41.421069 19066 log.cc:927] Time spent T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
W20260709 13:19:41.420506 19067 log.cc:927] Time spent T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.170s	user 0.002s	sys 0.000s
W20260709 13:19:41.417742 15598 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53310 (request call id 2045) took 1187 ms (client timeout 1545 ms). Trace:
I20260709 13:19:41.411733 18604 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:19:41.422770 19077 log.cc:927] Time spent T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:19:41.424532 19038 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:41.348152 19031 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:41.467223 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 57909.9us
W20260709 13:19:41.476013 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 219913us
I20260709 13:19:41.421187 19043 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:41.420912 19079 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:41.468942 15598 rpcz_store.cc:269] 0709 13:19:40.225320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.226016 (+   696us) service_pool.cc:224] Handling call
0709 13:19:41.412784 (+1186768us) spinlock_profiling.cc:243] Waited 754 ms on lock 0x7b540011e4d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:41.412954 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1733750016}
I20260709 13:19:41.396654 19058 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:41.421445 19065 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:41.455983 19069 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:41.421802 19061 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.116s
I20260709 13:19:41.466665 19062 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:41.411100 18797 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.388600 19076 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.325s
W20260709 13:19:41.579658 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 100054us
I20260709 13:19:41.474691 18604 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:41.587208 15642 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:41.475468 18907 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:41.604297 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:41.599619 15519 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:19:41.605001 15525 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:41.477751 19043 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.395141 19084 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:41.478331 19027 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:41.468863 19085 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 7. Current term is 8.
I20260709 13:19:41.586760 19065 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:41.638638 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 139400us
I20260709 13:19:41.617030 18604 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:41.619339 19043 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:41.487289 19079 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.640604 15641 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:41.634500 18907 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.598663 19062 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:41.713867 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 73376.7us
I20260709 13:19:41.714749 15641 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:19:41.716418 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 129324us
I20260709 13:19:41.647868 19027 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.656591 19049 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:19:41.662447 18981 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:19:41.705143 18967 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:41.720237 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 118332us
I20260709 13:19:41.704675 18945 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:41.711092 19058 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:41.724951 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:41.727152 15526 raft_consensus.cc:1275] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:19:41.731029 15566 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:41.734038 19084 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:41.735757 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:41.728402 18933 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:41.728303 19058 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:41.751600 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:41.727885 18967 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.733942 18797 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:41.725446 19017 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.748020 19093 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.750100 18981 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:41.732038 19080 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.192s
I20260709 13:19:41.783048 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:41.808746 19003 consensus_meta.cc:287] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.819s	user 0.009s	sys 0.000s
I20260709 13:19:41.809223 19003 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:41.809990 19093 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.812427 19093 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:41.813146 15527 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:19:41.783874 19090 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:41.784821 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:41.786684 19089 log.cc:927] Time spent T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:19:41.788916 18981 raft_consensus.cc:697] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:41.772917 19017 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:41.790769 18842 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:41.782776 19070 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:41.793346 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:41.778959 19082 log.cc:927] Time spent T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
I20260709 13:19:41.755496 19018 consensus_queue.cc:1048] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:19:41.798755 18604 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.771240 18967 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.800186 19098 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:41.822902 19090 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:41.862113 19090 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:41.838210 18883 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.009s	sys 0.008s
W20260709 13:19:41.863576 18883 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.010s	sys 0.008s
I20260709 13:19:41.805274 19036 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:41.813077 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:41.784317 19091 log.cc:927] Time spent T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:19:41.801357 19038 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:41.865972 15669 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:19:41.850080 18940 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:41.867084 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:41.866988 18940 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.867786 15669 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:19:41.868836 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:41.869169 18940 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:41.834857 15599 raft_consensus.cc:1240] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:41.869436 15669 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:19:41.840310 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:41.840183 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:41.842666 15677 raft_consensus.cc:1275] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:19:41.870704 15491 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:41.870695 15598 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
W20260709 13:19:41.798252 19107 log.cc:927] Time spent T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.075s	user 0.002s	sys 0.000s
I20260709 13:19:41.823311 15526 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:41.872706 15601 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:19:41.845008 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:41.845609 19098 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:41.873360 15674 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:19:41.873981 15601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 1839) took 1507 ms. Trace:
I20260709 13:19:41.874615 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:41.874809 19093 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.835378 18797 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.875337 19093 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.875484 18797 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.876291 18907 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:41.843636 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:41.870843 15673 raft_consensus.cc:2393] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 13: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:41.810612 18602 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.850169 18604 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:41.851143 18979 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:41.852001 18981 consensus_queue.cc:237] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.853235 19088 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:41.855322 19017 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:41.855207 19070 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:41.857348 19043 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.843580 19062 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:41.858150 19040 consensus_meta.cc:287] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.881s	user 0.001s	sys 0.009s
I20260709 13:19:41.858506 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:41.840729 19039 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:41.860215 19027 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:41.860351 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:41.863878 19065 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:41.866333 19038 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.847275 18967 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:41.844797 16204 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:41.873391 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:41.910989 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 66302.8us
I20260709 13:19:41.874382 15601 rpcz_store.cc:276] 0709 13:19:40.366683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.479384 (+112701us) service_pool.cc:224] Handling call
0709 13:19:40.485158 (+  5774us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:41.873124 (+1387966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:41.873708 (+   584us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b540016bf54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:41.873875 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3190589184}
I20260709 13:19:41.880962 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:41.882963 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:41.912221 15669 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:19:41.890806 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:41.902316 16203 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:41.876868 18907 consensus_queue.cc:1059] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:19:41.879702 19098 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:41.912662 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:41.914108 15491 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:41.911017 15526 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:19:41.913581 16204 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:41.917100 15677 raft_consensus.cc:2393] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 10: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:41.919210 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:41.915153 15566 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:41.920990 15566 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:41.891868 19070 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.927137 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:41.923993 16203 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:41.911651 19027 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.914388 18883 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.942757 15598 raft_consensus.cc:1275] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:19:41.934320 19093 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:41.934568 18601 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:41.935732 18907 consensus_queue.cc:1243] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer 5498ef3e89a941e39c25f68cf64e6b0f log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:19:41.887949 18979 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.928905 18676 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:41.926865 19062 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:41.947208 19027 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.926496 18967 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:41.926453 19017 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:41.957195 15520 pending_rounds.cc:85] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 7
I20260709 13:19:41.948462 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:41.948256 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:41.972532 15673 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 10: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:41.929045 19049 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:41.908787 19039 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:41.923897 19073 consensus_queue.cc:1048] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:19:41.907160 19040 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:41.922230 19058 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:42.002686 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:41.971431 15520 pending_rounds.cc:107] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:19:42.008925 15520 raft_consensus.cc:2889] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:42.009002 15673 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:42.006345 19046 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:41.890190 19088 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:42.039283 15525 raft_consensus.cc:2393] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 8: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:42.046828 16205 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:42.044883 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:42.038906 15566 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:41.955420 19072 raft_consensus.cc:3055] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:41.957647 18967 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.957885 19017 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.977175 18797 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:42.086362 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 156532us
I20260709 13:19:41.967859 18849 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.949185 18601 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.024627 19108 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:42.065563 15524 raft_consensus.cc:2410] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:19:41.947540 19033 consensus_queue.cc:1059] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.116s
I20260709 13:19:42.105218 16203 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:41.954849 18676 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.037683 18602 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:41.955626 19062 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:42.129437 19062 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:42.045020 19018 raft_consensus.cc:2955] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:42.131019 19059 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:42.094390 19046 raft_consensus.cc:697] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:42.132929 19046 consensus_queue.cc:237] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:42.108525 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 174498us
I20260709 13:19:41.956182 18979 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:42.037055 19027 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:42.037004 19069 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:42.035349 19039 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:42.031355 19038 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:42.135082 19038 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:42.136124 19069 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:42.092492 19072 raft_consensus.cc:740] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:42.093734 19049 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:42.038710 19043 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:42.032060 19040 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:42.069370 18604 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:42.114663 18676 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:42.106048 18601 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:42.109982 18849 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:42.039819 19058 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:42.051788 19063 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:42.094091 18967 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:42.116904 18440 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:42.094087 19017 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:42.120472 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1896) took 1060 ms. Trace:
I20260709 13:19:42.123005 19098 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:41.947973 18883 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:42.102399 19090 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:42.015481 19023 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:42.125527 19088 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:42.125952 19033 consensus_queue.cc:1243] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer 5498ef3e89a941e39c25f68cf64e6b0f log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:19:42.044449 19070 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:42.137589 19049 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.145733 19039 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.154523 15567 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:42.159626 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:42.209810 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 55390.8us
I20260709 13:19:42.160421 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:42.137565 19038 consensus_queue.cc:1048] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:19:42.181387 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:42.177637 15521 rpcz_store.cc:276] 0709 13:19:41.060222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.060738 (+   516us) service_pool.cc:224] Handling call
0709 13:19:41.389403 (+328665us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:41.478462 (+ 89059us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:41.478489 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:41.583995 (+105506us) log.cc:844] Serialized 26 byte log entry
0709 13:19:41.710013 (+126018us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:41.710051 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:42.119136 (+409085us) spinlock_profiling.cc:243] Waited 88.6 ms on lock 0x7b540029c2d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:42.119200 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:42.119258 (+    58us) raft_consensus.cc:1685] finished
0709 13:19:42.119273 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:42.120239 (+   966us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b540029c2d0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:42.120369 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":109,"raft.queue_time_us":161619,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":13354,"spinlock_wait_cycles":958881920,"thread_start_us":61370,"threads_started":1,"wal-append.queue_time_us":96105}
I20260709 13:19:42.180125 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:42.195516 19081 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:19:42.193892 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 57455.5us
I20260709 13:19:42.208732 15671 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
W20260709 13:19:42.211678 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 95786.6us
I20260709 13:19:42.176930 18440 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:42.213989 15674 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:19:42.218333 15601 raft_consensus.cc:2410] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:19:42.211241 19049 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:42.110816 19108 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:42.215211 19069 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:42.241259 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:42.243813 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:42.244755 15521 raft_consensus.cc:2393] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 10: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:42.251197 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:42.251924 15669 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:42.252982 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:42.227519 18440 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:42.256537 15526 raft_consensus.cc:3055] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:42.256955 15526 raft_consensus.cc:740] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:42.232059 19069 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.258292 19069 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:42.263302 18849 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:42.263607 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:42.274856 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:42.266003 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:42.182410 19063 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:42.207891 19023 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:42.257752 15526 consensus_queue.cc:260] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:42.304220 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:42.215725 19039 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:42.370522 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:42.371737 15669 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:42.370078 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 143430us
I20260709 13:19:42.299630 19062 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:42.448417 19062 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:42.448958 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:42.449242 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:42.448252 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:42.449828 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:42.368085 19013 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:42.449658 15676 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:42.367136 19088 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:42.449875 15675 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
W20260709 13:19:42.460975 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 158941us
I20260709 13:19:42.461972 15677 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:19:42.388319 18989 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.383649 16204 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:42.387495 19108 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:42.468147 15677 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:19:42.468597 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:42.471048 15675 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:42.461099 19113 log.cc:927] Time spent T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.620s	user 0.003s	sys 0.005s
I20260709 13:19:42.383543 19069 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:42.503994 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:42.460794 19103 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:42.508574 16237 raft_consensus.cc:2410] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:19:42.522305 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:42.523123 16236 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:19:42.524336 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:42.525086 15677 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:19:42.528157 15669 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:19:42.478590 19105 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:42.542976 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 129561us
I20260709 13:19:42.542078 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:42.542658 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 159153us
I20260709 13:19:42.556288 19126 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:19:42.558342 19116 log.cc:927] Time spent T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.330s	user 0.000s	sys 0.001s
I20260709 13:19:42.560817 18601 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:42.564538 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:42.565434 16236 raft_consensus.cc:2368] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11.
I20260709 13:19:42.461570 19013 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:42.465533 18440 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:42.514951 19070 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:42.573293 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:42.535564 18883 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:19:42.477715 18989 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:42.577524 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:42.477025 19088 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:42.583155 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:42.579411 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:42.582175 15603 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:42.534507 19098 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:42.600137 16238 raft_consensus.cc:2410] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:19:42.603875 16237 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:19:42.623869 16232 raft_consensus.cc:1240] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:42.625985 19079 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:42.582825 19110 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:42.628541 19110 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:42.631434 19110 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:42.632526 19134 log.cc:927] Time spent T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:19:42.634560 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:42.636117 19133 log.cc:927] Time spent T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 13:19:42.636835 19023 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:42.637228 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:42.637884 16235 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:19:42.604909 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:42.640733 19081 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:42.641309 19081 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:42.642297 19088 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:42.643913 19013 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:42.648520 18440 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:42.649911 19049 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:42.648546 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:42.654232 19069 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:42.655112 19058 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:42.655763 19058 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:42.658725 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:42.660037 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:42.660195 18601 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:42.661129 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:42.665166 19115 log.cc:927] Time spent T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.435s	user 0.001s	sys 0.001s
I20260709 13:19:42.600833 18981 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:42.666838 18981 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:42.673804 15676 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:19:42.600270 19070 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:42.573838 19105 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:42.535745 19103 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:42.959121 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:42.908442 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:42.610044 19018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:53300 (request call id 89) took 2535 ms. Trace:
I20260709 13:19:42.898888 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:42.867576 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:42.908473 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:42.960258 15603 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:19:42.907979 16240 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:19:42.893071 15602 raft_consensus.cc:2393] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 8: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:42.868732 15641 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:42.763731 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 88243.2us
I20260709 13:19:42.901338 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:42.763089 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:42.763091 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:42.959390 15675 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
W20260709 13:19:42.970603 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 101988us
I20260709 13:19:42.975641 15641 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:42.976766 15641 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:42.958801 19122 log.cc:927] Time spent T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.360s	user 0.001s	sys 0.000s
W20260709 13:19:42.878819 19114 log.cc:927] Time spent T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.278s	user 0.005s	sys 0.001s
I20260709 13:19:42.965291 16238 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:19:42.914179 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:42.974114 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:42.977491 16236 raft_consensus.cc:1813] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Handling vote request from an unknown peer 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:42.985419 16236 raft_consensus.cc:2410] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:19:42.898346 19110 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:42.926770 19033 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.800607 19058 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:42.986341 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:42.801411 19088 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:42.983824 16231 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
W20260709 13:19:42.979185 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 463906us
I20260709 13:19:42.969992 18849 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.972762 19108 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:42.980610 19017 consensus_meta.cc:287] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.780s	user 0.010s	sys 0.000s
I20260709 13:19:42.966562 19039 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:42.915189 19049 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:43.009980 15566 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:42.997781 18849 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:42.998935 19087 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:42.980677 18842 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:42.987332 19098 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:42.992890 18981 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:42.964339 19018 rpcz_store.cc:276] 0709 13:19:40.066800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.069681 (+  2881us) service_pool.cc:224] Handling call
0709 13:19:40.086322 (+ 16641us) log.cc:844] Serialized 494 byte log entry
0709 13:19:40.280133 (+193811us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b180038ae80. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce191e3f3 00007f5ce19207c1 00007f5ce1920779 00007f5ce1920709 00007f5ce19206d1 00007f5ce191f9cd 000000000038f354 000000000038f199 00007f5ce190183a
0709 13:19:42.599368 (+2319235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":98011,"raft.queue_time_us":2039,"raft.run_cpu_time_us":1542,"raft.run_wall_time_us":57135,"spinlock_wait_cycles":3948416,"thread_start_us":126235,"threads_started":1,"wal-append.queue_time_us":168111,"wal-append.run_cpu_time_us":1788,"wal-append.run_wall_time_us":1007905}
I20260709 13:19:42.913792 19081 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:43.017081 19087 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:43.009099 19110 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:43.037712 19031 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:43.037848 19105 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:43.020712 19098 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:43.018203 18842 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:43.016564 19139 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:43.044308 19103 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:43.014828 19017 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:43.013213 19048 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
W20260709 13:19:43.047067 18676 consensus_meta.cc:287] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.851s	user 0.010s	sys 0.000s
I20260709 13:19:43.011396 19088 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:43.047830 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1899) took 1571 ms. Trace:
I20260709 13:19:43.073897 19098 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:42.992681 19033 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:43.079293 18842 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:43.037910 19070 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:43.093775 15600 raft_consensus.cc:2368] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11.
I20260709 13:19:43.093803 19031 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:43.096423 19031 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:43.061628 15669 raft_consensus.cc:1240] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:43.063881 19087 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:43.042727 18989 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:43.074753 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:43.075376 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:43.105001 15674 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:43.081765 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:43.085623 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:43.048640 18967 consensus_meta.cc:287] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.852s	user 0.005s	sys 0.005s
I20260709 13:19:43.088275 15412 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bd36c2cfd99c45cea9e48096d54bed1a with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:43.086814 15528 rpcz_store.cc:276] 0709 13:19:41.476444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.667911 (+191467us) service_pool.cc:224] Handling call
0709 13:19:41.668210 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:41.724285 (+ 56075us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:41.724317 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:41.724355 (+    38us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:41.724386 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.047544 (+1323158us) spinlock_profiling.cc:243] Waited 55.9 ms on lock 0x7b54003d9ad4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:43.047595 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.047760 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":128529664}
I20260709 13:19:43.050330 18849 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:43.091017 19146 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:43.080279 19139 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:43.089988 18676 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:43.130272 19139 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:43.130237 19146 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:43.131819 18676 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:43.132890 19146 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:43.133741 16233 raft_consensus.cc:1240] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:19:43.134402 18967 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:43.136545 18967 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:43.093113 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 109348us
W20260709 13:19:43.118320 19120 log.cc:927] Time spent T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.663s	user 0.007s	sys 0.010s
W20260709 13:19:43.138084 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 164034us
I20260709 13:19:43.139488 15675 raft_consensus.cc:2393] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 8: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:43.156566 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:43.102288 19110 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:43.160329 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:43.117304 19017 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:43.162038 15594 raft_consensus.cc:2368] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9.
I20260709 13:19:43.117357 19033 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:43.162740 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1848) took 1538 ms. Trace:
I20260709 13:19:43.162236 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:43.118784 19088 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:43.165522 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:43.166337 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:43.164834 15523 rpcz_store.cc:276] 0709 13:19:41.624757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.875601 (+250844us) service_pool.cc:224] Handling call
0709 13:19:41.875909 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:41.876217 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:41.876245 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:41.876696 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 13:19:42.177473 (+300777us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:42.177511 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:42.177582 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:43.117614 (+940032us) spinlock_profiling.cc:243] Waited 5.38 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:19:43.158294 (+ 40680us) raft_consensus.cc:1685] finished
0709 13:19:43.158315 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.162673 (+  4358us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1059,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":10131,"spinlock_wait_cycles":12369792,"thread_start_us":300433,"threads_started":1,"wal-append.queue_time_us":424833}
I20260709 13:19:43.178536 19081 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:43.177321 19087 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:43.181016 19087 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:43.182868 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:43.184774 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:43.185014 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:43.202775 15676 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:43.193280 15642 leader_election.cc:400] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9.
I20260709 13:19:43.161197 19138 consensus_queue.cc:1059] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:19:43.210108 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:43.205834 15642 leader_election.cc:403] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:19:43.208483 15674 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:19:43.165462 19148 consensus_queue.cc:1048] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
W20260709 13:19:43.210748 19050 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.830s	user 0.044s	sys 0.069s
W20260709 13:19:43.220752 19050 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.045s	sys 0.070s
I20260709 13:19:43.222307 19147 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9.
I20260709 13:19:43.220644 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1011) took 1396 ms. Trace:
I20260709 13:19:43.229974 19137 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:43.230583 19137 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:43.232694 19137 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:43.233791 15526 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:43.216813 19048 raft_consensus.cc:3055] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:43.226792 15520 rpcz_store.cc:276] 0709 13:19:41.824079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.932179 (+108100us) service_pool.cc:224] Handling call
0709 13:19:41.945760 (+ 13581us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:42.102490 (+156730us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:42.102521 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:42.103144 (+   623us) log.cc:844] Serialized 52 byte log entry
0709 13:19:42.207400 (+104256us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:42.300845 (+ 93445us) log.cc:844] Serialized 14 byte log entry
0709 13:19:42.302211 (+  1366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.220281 (+918070us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b5400188154. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:43.220325 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:43.220372 (+    47us) raft_consensus.cc:1685] finished
0709 13:19:43.220459 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.220588 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":186,"prepare.queue_time_us":346468,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":37313,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":1469,"spinlock_wait_cycles":25754368,"thread_start_us":142216,"threads_started":2,"wal-append.queue_time_us":99301}
I20260709 13:19:43.235265 19048 raft_consensus.cc:740] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:43.262624 19048 consensus_queue.cc:260] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:43.245028 18842 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:43.267755 15526 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:19:43.271095 15526 pending_rounds.cc:85] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 10
I20260709 13:19:43.273738 18842 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:43.277626 15526 pending_rounds.cc:107] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:19:43.278095 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:43.278431 15526 raft_consensus.cc:2889] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:43.279618 19097 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:19:43.280052 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:43.280812 18842 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:43.283105 18940 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:43.283720 18940 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:43.285712 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1852) took 1463 ms. Trace:
I20260709 13:19:43.277835 16203 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:43.283849 18842 consensus_queue.cc:1059] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:43.291564 16203 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:19:43.336874 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 45395.3us
I20260709 13:19:43.337571 16203 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:43.299832 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:43.307085 15672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 949) took 1441 ms. Trace:
I20260709 13:19:43.314070 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:43.314663 15566 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:43.341071 15671 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 11: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:43.332679 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:43.343257 16234 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:19:43.331276 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1906) took 1477 ms. Trace:
I20260709 13:19:43.286582 15526 rpcz_store.cc:276] 0709 13:19:41.822834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.931800 (+108966us) service_pool.cc:224] Handling call
0709 13:19:41.946001 (+ 14201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:42.256440 (+310439us) spinlock_profiling.cc:243] Waited 8.4 ms on lock 0x7b540005c1a0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4982dc0 00007f5ce4a0d1c0 00007f5ce4a0e231 00007f5ce4a0ad8f 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c
0709 13:19:43.233356 (+976916us) spinlock_profiling.cc:243] Waited 932 ms on lock 0x7b580052acd0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce49670aa 00007f5ce49e9447 00007f5ce4a01fe7 00007f5ce4a037de 00007f5ce4a0db49 00007f5ce4a0e416 00007f5ce4a0ad8f 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4
0709 13:19:43.279435 (+ 46079us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.285637 (+  6202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2162081920}
I20260709 13:19:43.308866 16240 raft_consensus.cc:1240] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:43.352814 16203 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:43.306927 16238 raft_consensus.cc:2955] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:19:43.328630 19125 log.cc:927] Time spent T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.950s	user 0.006s	sys 0.010s
I20260709 13:19:43.285907 18940 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:43.384914 15668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2141) took 1496 ms. Trace:
I20260709 13:19:43.318617 18698 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:43.300275 19048 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:43.391223 15600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34102 (request call id 814) took 1128 ms. Trace:
I20260709 13:19:43.339186 15672 rpcz_store.cc:276] 0709 13:19:41.866010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.875053 (+  9043us) service_pool.cc:224] Handling call
0709 13:19:41.875329 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:41.883106 (+  7777us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:41.883142 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:41.891951 (+  8809us) log.cc:844] Serialized 26 byte log entry
0709 13:19:42.003776 (+111825us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:42.003820 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.306645 (+1302825us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b540026b6d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:43.306697 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:43.306745 (+    48us) raft_consensus.cc:1685] finished
0709 13:19:43.306760 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.306993 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4985,"raft.queue_time_us":6712,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":4769,"spinlock_wait_cycles":5521280,"thread_start_us":111400,"threads_started":1,"wal-append.queue_time_us":303378}
I20260709 13:19:43.351776 19128 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:43.351469 18989 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:43.328871 19118 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.291764 19061 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:43.350903 19138 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:43.387607 16203 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:43.352540 15527 rpcz_store.cc:276] 0709 13:19:41.854023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.976889 (+122866us) service_pool.cc:224] Handling call
0709 13:19:41.977252 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:42.002328 (+ 25076us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:42.002361 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:42.040211 (+ 37850us) log.cc:844] Serialized 26 byte log entry
0709 13:19:42.171661 (+131450us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:43.258568 (+1086907us) spinlock_profiling.cc:243] Waited 8.74 ms on lock 0x7b180033eda0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce45b34bc 00007f5ce190148b 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191b455 00007f5ce84033c7 00007f5ce4a159a7 00007f5ce4a250bf
0709 13:19:43.258734 (+   166us) log.cc:844] Serialized 14 byte log entry
0709 13:19:43.329263 (+ 70529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.329355 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:43.329401 (+    46us) raft_consensus.cc:1685] finished
0709 13:19:43.331056 (+  1655us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.331212 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":309030,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":9352,"raft.queue_time_us":25599,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":4006,"spinlock_wait_cycles":20097792,"thread_start_us":523592,"threads_started":3,"wal-append.queue_time_us":156821,"wal-append.run_cpu_time_us":2164,"wal-append.run_wall_time_us":1004985}
I20260709 13:19:43.385320 15668 rpcz_store.cc:276] 0709 13:19:41.889164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.051726 (+162562us) service_pool.cc:224] Handling call
0709 13:19:42.052060 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:42.052433 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:42.052462 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:42.052935 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 13:19:42.180559 (+127624us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:42.180601 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:42.180679 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:43.327263 (+1146584us) spinlock_profiling.cc:243] Waited 3.8 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:19:43.384591 (+ 57328us) raft_consensus.cc:1685] finished
0709 13:19:43.384617 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.384834 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":103,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"spinlock_wait_cycles":8748416,"thread_start_us":127241,"threads_started":1,"wal-append.queue_time_us":216304}
I20260709 13:19:43.391551 15600 rpcz_store.cc:276] 0709 13:19:42.263094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.263232 (+   138us) service_pool.cc:224] Handling call
0709 13:19:43.391028 (+1127796us) spinlock_profiling.cc:243] Waited 829 ms on lock 0x7b5400232654. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:43.391162 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1906631936}
I20260709 13:19:43.398224 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:43.388763 19126 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:43.388309 18981 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:43.385638 18842 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:43.327659 19139 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:43.448485 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:43.401798 19118 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:43.416741 18940 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:43.465322 18940 raft_consensus.cc:697] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:43.466317 18940 consensus_queue.cc:237] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:43.470938 19098 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:43.473774 19098 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:43.423641 19136 log.cc:927] Time spent T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.517s	user 0.004s	sys 0.000s
I20260709 13:19:43.287353 19072 consensus_queue.cc:260] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:43.484630 19072 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:43.485206 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:43.485984 15675 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 9: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:43.486950 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:43.488179 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:43.489395 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:43.490271 16205 leader_election.cc:400] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11.
I20260709 13:19:43.490585 16205 leader_election.cc:403] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:19:43.491107 15669 raft_consensus.cc:2393] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 10: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:43.492359 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:43.492360 19126 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:43.433055 15493 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:43.493698 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 60733.4us
I20260709 13:19:43.495024 15493 leader_election.cc:400] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11.
I20260709 13:19:43.495349 15493 leader_election.cc:403] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:43.495980 19081 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:43.496366 19081 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:43.496680 19081 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:43.498667 19059 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:19:43.431592 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:43.433396 18981 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:43.434907 19040 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:43.437889 15668 raft_consensus.cc:1240] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:43.455335 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:43.506584 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:43.459551 16239 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:19:43.460106 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:43.518559 15493 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:43.464531 19003 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:43.474390 18842 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:43.395047 18698 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:43.475677 19070 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:43.477869 15568 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:43.478077 19117 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:43.479638 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:43.483475 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:43.465641 19031 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:43.422572 19090 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:43.434446 19097 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:43.500695 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:43.543345 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:43.544210 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:43.500960 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:43.463967 19063 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:43.507094 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.521253 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:43.522006 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 105982us
I20260709 13:19:43.524457 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:43.399799 19061 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:43.522825 19027 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:43.402138 19128 raft_consensus.cc:697] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:43.531905 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:43.413371 19138 raft_consensus.cc:697] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:19:43.539741 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 61948.1us
I20260709 13:19:43.541977 19117 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:43.534650 19105 consensus_queue.cc:1059] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:19:43.522719 19033 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:43.544341 15520 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:43.478143 16204 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:43.550289 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:43.565328 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:43.569751 15491 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:43.573611 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:43.575544 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:43.576372 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:43.578714 15568 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:43.620970 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:19:43.624254 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 146183us
I20260709 13:19:43.592566 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:43.620507 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:19:43.628077 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 58432us
I20260709 13:19:43.629199 15668 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:43.630506 15523 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:43.627156 16204 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:43.631407 15672 raft_consensus.cc:2368] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10.
I20260709 13:19:43.632232 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:43.632844 15526 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:43.633205 19063 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:43.633729 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:43.634320 19110 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:43.634975 15527 raft_consensus.cc:2368] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10.
I20260709 13:19:43.630077 15525 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:43.554176 18784 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:43.534960 19040 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:43.637265 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 58609.9us
I20260709 13:19:43.656291 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:43.639801 15491 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:43.657446 15568 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:43.658056 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:43.658847 16233 raft_consensus.cc:2393] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 8: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:43.563720 19061 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:43.637622 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:43.627949 16240 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:43.562330 19108 raft_consensus.cc:3055] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:43.661873 16232 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:43.669211 15523 raft_consensus.cc:2393] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 6: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:43.572793 19003 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:43.678535 15668 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:19:43.683727 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:43.624415 19027 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.693666 16234 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:19:43.578186 19033 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:43.686546 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:43.543397 19097 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:43.709512 16232 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:19:43.656880 19038 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:43.659909 19018 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:43.566658 19070 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:43.749711 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:43.668831 19108 raft_consensus.cc:740] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:43.776026 19027 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:43.740619 15520 raft_consensus.cc:1275] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:19:43.799134 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:43.734786 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:43.800176 15526 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:19:43.621084 19069 consensus_queue.cc:1059] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 13:19:43.668675 19148 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:43.769940 15527 raft_consensus.cc:1240] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:43.770206 16239 raft_consensus.cc:1240] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:19:43.686312 18966 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:43.811826 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:43.686115 19003 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:43.775950 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:43.829352 15594 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:43.716922 19031 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:43.798203 15595 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:19:43.774036 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:43.830821 15523 raft_consensus.cc:2393] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 10: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:43.818292 16238 raft_consensus.cc:1240] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:43.826383 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:43.832377 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:43.776732 19033 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:43.836802 19148 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:43.856488 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:43.857292 15595 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 3.
I20260709 13:19:43.859799 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:43.860647 15523 raft_consensus.cc:2393] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 8: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:19:43.862329 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 148517us
I20260709 13:19:43.867024 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:43.870311 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 230565us
I20260709 13:19:43.885293 15491 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:43.892252 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:43.894484 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:43.888908 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:43.895439 15523 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
W20260709 13:19:43.844794 15423 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58802 (request call id 259) took 22153 ms (client timeout 15000 ms). Trace:
I20260709 13:19:43.897145 16203 leader_election.cc:400] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10.
I20260709 13:19:43.897521 16203 leader_election.cc:403] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:19:43.850814 15671 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:19:43.803213 19110 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:43.763907 16235 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:19:43.902007 16235 raft_consensus.cc:606] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:43.902207 15595 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:19:43.901087 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:43.898906 16203 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:43.859287 19031 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:43.909618 19017 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:43.911587 19031 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:43.912283 19031 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:43.915167 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:43.915876 15671 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:43.917805 15568 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:43.919155 19049 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:43.919706 19049 raft_consensus.cc:697] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:43.920629 19049 consensus_queue.cc:237] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:43.906414 15642 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:43.929654 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:43.935076 15671 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:19:43.940613 16204 leader_election.cc:400] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10.
I20260709 13:19:43.942863 16204 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:43.889436 19139 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:19:43.890942 19168 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:43.804536 19138 consensus_queue.cc:237] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:43.946079 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:43.932356 19039 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:43.957955 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:43.968248 15523 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:43.869302 19108 consensus_queue.cc:260] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:43.959771 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:43.934485 19157 log.cc:927] Time spent T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:19:43.804685 19128 consensus_queue.cc:237] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:43.934135 19166 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:43.944885 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:43.965461 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:43.899089 19118 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:43.966233 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:43.897258 15423 rpcz_store.cc:269] 0709 13:19:21.691575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.720978 (+ 29403us) service_pool.cc:224] Handling call
0709 13:19:23.210749 (+1489771us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b5400369554. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4975090 00007f5ce4a13f3a 00007f5cea7ca1cc 00007f5cea8c837e 00007f5ce8d892a4 00007f5ce8d89232 00007f5ce8d89161 00007f5ce8d890dc 00007f5ce8d88392 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:43.844703 (+20633954us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:40.484495 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:40.485149 (+   654us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:40.485209 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:19:40.485222 (+    13us) tablet.cc:662] Decoding operations
0709 13:19:40.853580 (+368358us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:40.853660 (+    80us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:40.853703 (+    43us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:19:40.854621 (+   918us) tablet.cc:701] Row locks acquired
0709 13:19:40.854673 (+    52us) write_op.cc:260] PREPARE: finished
0709 13:19:40.854867 (+   194us) write_op.cc:270] Start()
0709 13:19:40.855026 (+   159us) write_op.cc:276] Timestamp: P: 1783603180854798 usec, L: 0
0709 13:19:40.855056 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:19:40.879067 (+ 24011us) log.cc:844] Serialized 3574 byte log entry
0709 13:19:41.725571 (+846504us) op_driver.cc:464] REPLICATION: finished
0709 13:19:42.023784 (+298213us) write_op.cc:301] APPLY: starting
0709 13:19:42.023925 (+   141us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:42.024151 (+   226us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:42.024298 (+   147us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:42.025536 (+  1238us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:42.025851 (+   315us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 13:19:42.025896 (+    45us) write_op.cc:312] APPLY: finished
0709 13:19:42.026243 (+   347us) log.cc:844] Serialized 71 byte log entry
0709 13:19:42.026443 (+   200us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:42.026560 (+   117us) write_op.cc:454] Released schema lock
0709 13:19:42.026614 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:42.029873 (+  3259us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:42.029904 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1846786,"reported_tablets":44,"spinlock_wait_cycles":292956416,"child_traces":[["op",{"apply.queue_time_us":244058,"apply.run_cpu_time_us":3301,"apply.run_wall_time_us":25153,"num_ops":9,"prepare.queue_time_us":315717,"prepare.run_cpu_time_us":30918,"prepare.run_wall_time_us":507691,"raft.queue_time_us":251522,"raft.run_cpu_time_us":16928,"raft.run_wall_time_us":222543,"replication_time_us":851151,"thread_start_us":324669,"threads_started":4,"wal-append.queue_time_us":176916,"wal-append.run_cpu_time_us":12816,"wal-append.run_wall_time_us":1976116}]]}
I20260709 13:19:43.962643 19087 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:43.981086 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:43.992923 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:43.982241 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:43.999435 15523 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
W20260709 13:19:43.971266 15641 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.077s (SENT) [suppressed 9 similar messages]
I20260709 13:19:44.000645 15523 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:19:43.967070 18981 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:43.912803 19031 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:43.966509 18842 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:44.007218 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 99735.9us
I20260709 13:19:43.970978 18849 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:44.018558 19182 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:44.019173 19182 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:44.021283 19182 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:44.023558 19148 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:19:44.027889 19081 consensus_meta.cc:287] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.531s	user 0.009s	sys 0.000s
I20260709 13:19:44.028676 19171 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:44.030174 16203 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:44.030781 19003 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:43.981117 19069 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:43.999112 19108 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:43.956113 18966 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:19:43.983913 19027 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:43.995347 19040 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:43.987273 19167 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:44.003160 18940 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:43.999131 19087 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:44.003618 18698 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:44.032042 19164 log.cc:927] Time spent T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:19:44.000731 19110 raft_consensus.cc:697] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:44.013371 18981 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:44.074818 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:44.042361 19126 consensus_meta.cc:287] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.549s	user 0.009s	sys 0.000s
I20260709 13:19:44.060233 15522 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
W20260709 13:19:44.081871 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 117069us
W20260709 13:19:44.085932 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.052270 19166 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:43.971138 19168 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:44.055337 19171 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:44.055250 19080 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.083719 19087 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:44.103832 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.055962 19069 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:44.106642 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:44.112852 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:44.040903 19081 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:44.084731 19167 raft_consensus.cc:697] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:44.111053 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:44.089761 19097 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:44.081375 19043 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:19:44.096613 18940 raft_consensus.cc:697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:44.103480 19126 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11.
W20260709 13:19:44.121383 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 178675us
W20260709 13:19:44.126225 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 113620us
I20260709 13:19:44.103112 19070 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:44.144670 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:44.152715 15521 raft_consensus.cc:2368] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9.
W20260709 13:19:44.120563 19048 consensus_meta.cc:287] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.725s	user 0.010s	sys 0.000s
W20260709 13:19:44.002828 19098 consensus_queue.cc:1175] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" responder_term: 6 status { last_received { term: 4 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 5. Current term is 6. Ops: [5.6-5.6]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:19:44.102428 18981 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:44.167326 18940 consensus_queue.cc:237] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:44.109746 19166 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:44.114436 19080 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:44.169950 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:44.230104 15566 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:19:44.153206 19161 log.cc:927] Time spent T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.216s	user 0.000s	sys 0.005s
I20260709 13:19:44.174352 16204 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:44.172410 15668 raft_consensus.cc:1240] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:44.166399 19167 consensus_queue.cc:237] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:44.208930 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:44.213408 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:44.196627 16237 raft_consensus.cc:1240] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:44.151762 15527 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:44.150071 19078 raft_consensus.cc:3055] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:44.157368 19168 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:44.154052 19180 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:44.215291 19110 consensus_queue.cc:237] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
W20260709 13:19:44.155478 19160 log.cc:927] Time spent T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:19:44.150735 19117 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:44.176582 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:44.284041 19180 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:44.271205 19195 log.cc:927] Time spent T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:19:44.168877 19185 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:44.241262 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:44.274641 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:19:44.161234 19059 consensus_meta.cc:287] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.662s	user 0.002s	sys 0.006s
W20260709 13:19:44.167711 19072 consensus_meta.cc:287] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.683s	user 0.007s	sys 0.004s
W20260709 13:19:44.267105 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 112709us
I20260709 13:19:44.286703 16231 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
W20260709 13:19:44.166900 19158 log.cc:927] Time spent T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.315s	user 0.004s	sys 0.000s
I20260709 13:19:44.292363 15672 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
W20260709 13:19:44.291397 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 60394.5us
W20260709 13:19:44.293545 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 119285us
W20260709 13:19:44.241804 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 209818us
I20260709 13:19:44.304389 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:44.168138 19031 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:44.208978 19081 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:44.302709 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:44.302810 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:44.319772 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:44.298053 15674 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:44.300108 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 328855us
W20260709 13:19:44.310997 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.323348 15668 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:19:44.303874 16204 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:44.325156 16204 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:44.327548 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:44.277099 19078 raft_consensus.cc:740] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:44.309223 15527 raft_consensus.cc:2955] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:44.354900 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:44.303356 19105 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:44.368211 15493 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:19:44.431329 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 63208.1us
I20260709 13:19:44.320844 19174 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:44.371073 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:44.371268 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:44.374680 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:44.366384 15641 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:44.301028 19059 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11.
I20260709 13:19:44.432119 15493 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
W20260709 13:19:44.435238 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 68943.9us
W20260709 13:19:44.430012 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 118079us
I20260709 13:19:44.358637 19039 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10.
W20260709 13:19:44.464354 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 48422.3us
I20260709 13:19:44.356169 19170 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:44.356922 19046 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:44.352846 19020 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:44.302028 19185 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:44.366616 19128 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.132s
I20260709 13:19:44.427055 19087 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:44.575738 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:44.554690 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 282969us
W20260709 13:19:44.505897 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.572080 15674 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:19:44.462731 19168 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:44.578299 19168 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:44.578783 19168 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:44.581797 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:44.582741 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:44.582659 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:44.583675 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:44.583768 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:44.587240 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:44.526726 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 223891us
I20260709 13:19:44.599756 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:44.486606 19081 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:44.490213 19170 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:44.530294 19105 raft_consensus.cc:697] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:44.489315 19046 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:44.605347 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:44.608716 18966 consensus_queue.cc:1059] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:44.609800 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:44.610806 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.617818 15528 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:19:44.633407 16238 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:19:44.530357 19020 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:44.472353 19174 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:44.760073 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:44.567596 19078 consensus_queue.cc:260] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:44.462944 19147 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:44.863837 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:44.807761 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 379495us
W20260709 13:19:44.842563 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 393541us
I20260709 13:19:44.770499 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:44.868036 15641 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:44.803081 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:44.771425 15493 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:44.754921 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:44.848366 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:44.461959 19069 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:44.871006 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 99701.1us
I20260709 13:19:44.871320 15674 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
W20260709 13:19:44.828317 19193 log.cc:927] Time spent T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:19:44.705613 19170 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:44.875761 15493 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:44.870047 16235 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:44.825410 19087 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:44.866945 19162 log.cc:927] Time spent T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.905s	user 0.002s	sys 0.005s
I20260709 13:19:44.706063 19046 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:44.868916 18698 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.623634 19081 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:44.758661 19080 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:44.787348 18981 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:44.899331 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 139443us
W20260709 13:19:44.870255 19190 log.cc:927] Time spent T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.421s	user 0.001s	sys 0.000s
W20260709 13:19:44.758816 19194 log.cc:927] Time spent T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:19:44.646131 19180 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:19:44.907407 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:44.900734 19061 consensus_meta.cc:287] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 1.211s	user 0.009s	sys 0.003s
W20260709 13:19:44.880869 19187 log.cc:927] Time spent T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.393s	user 0.000s	sys 0.001s
W20260709 13:19:44.917505 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 49334.7us
I20260709 13:19:44.879994 19090 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.148660 19090 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:44.874743 19203 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:45.150254 19203 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:45.152622 19203 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:44.927142 15527 consensus_meta.cc:287] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.569s	user 0.007s	sys 0.002s
I20260709 13:19:44.876428 19031 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:45.153301 19031 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:44.899060 19147 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:44.881007 19166 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:44.960255 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 84585.5us
I20260709 13:19:44.882191 15641 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:44.960294 19110 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:44.963047 19018 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.154613 19018 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:45.155376 15493 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:45.156332 15493 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:45.156783 19018 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:45.154635 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 272688us
I20260709 13:19:45.157545 19059 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:45.157982 15641 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:45.153787 19147 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:45.158984 19147 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:45.159153 15641 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:45.159571 19166 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:45.159934 19165 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:19:44.970850 19188 log.cc:927] Time spent T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.395s	user 0.000s	sys 0.004s
I20260709 13:19:44.939743 18842 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:45.158161 19181 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:44.879948 19175 consensus_queue.cc:1175] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3ef186ec12554a1d87814af0a0ace9a4" responder_term: 9 status { last_received { term: 8 index: 6 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:19:44.877794 18784 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:19:45.161247 18842 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:45.027689 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 127113us
I20260709 13:19:45.163714 18842 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:44.958881 19105 consensus_queue.cc:237] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
W20260709 13:19:45.093283 19198 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.465s	user 0.001s	sys 0.000s
I20260709 13:19:44.979858 18698 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:45.160614 16203 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:44.976914 19108 consensus_meta.cc:287] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.914s	user 0.010s	sys 0.000s
I20260709 13:19:45.167685 18698 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:45.159994 19166 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:45.169786 18440 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:45.170195 19166 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:45.172060 19080 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:45.172600 19080 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:45.174779 19080 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:45.177279 18854 raft_consensus.cc:3055] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:45.177702 18854 raft_consensus.cc:740] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:45.178754 19090 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:45.157382 15493 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:45.181602 19181 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:45.182417 19214 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:45.182967 19214 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:45.185094 19214 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:45.142735 19213 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:45.190621 19062 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.190480 19213 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:45.191238 19062 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:44.976679 19185 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:45.141750 15567 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:44.979624 19081 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:45.143909 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.176384 19046 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:45.153715 19031 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:45.157696 19148 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:44.965622 19020 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:45.160661 19165 consensus_queue.cc:1048] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:45.225972 16240 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1152) took 1033 ms. Trace:
I20260709 13:19:45.178455 18854 consensus_queue.cc:260] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:45.173086 19033 consensus_meta.cc:287] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 1.312s	user 0.003s	sys 0.007s
W20260709 13:19:45.135749 16205 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.001s	sys 0.009s
W20260709 13:19:45.214939 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:45.216025 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:45.236003 16205 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.002s	sys 0.010s
I20260709 13:19:45.146895 19061 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:45.190482 19173 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:45.195950 15493 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:45.226683 16240 rpcz_store.cc:276] 0709 13:19:44.192535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.193691 (+  1156us) service_pool.cc:224] Handling call
0709 13:19:44.218939 (+ 25248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.290639 (+ 71700us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:44.724777 (+434138us) log.cc:844] Serialized 14 byte log entry
0709 13:19:45.225549 (+500772us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:45.225609 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:45.225675 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.225747 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:45.225877 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26080,"prepare.queue_time_us":588903,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"thread_start_us":866096,"threads_started":2}
I20260709 13:19:45.198756 16235 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
W20260709 13:19:45.204051 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:45.217756 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 55302.6us
W20260709 13:19:45.216087 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 74416us
W20260709 13:19:44.944103 19200 log.cc:927] Time spent T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:19:44.986861 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1849) took 3248 ms. Trace:
I20260709 13:19:45.232121 15491 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:19:45.236446 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 787807us
W20260709 13:19:45.238241 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 54333.4us
I20260709 13:19:45.192826 19213 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:45.244133 15493 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:45.242782 15519 rpcz_store.cc:276] 0709 13:19:41.729786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.910589 (+180803us) service_pool.cc:224] Handling call
0709 13:19:41.926019 (+ 15430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:41.932785 (+  6766us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:41.932825 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:41.943002 (+ 10177us) log.cc:844] Serialized 26 byte log entry
0709 13:19:43.466961 (+1523959us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b18004ae8a0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab0d1 00007f5ce49c6a0f
0709 13:19:43.714715 (+247754us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:43.714761 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.714833 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:44.581291 (+866458us) spinlock_profiling.cc:243] Waited 4.44 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:19:44.978375 (+397084us) raft_consensus.cc:1685] finished
0709 13:19:44.978401 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:44.978551 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6395,"raft.queue_time_us":15680,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":3152,"spinlock_wait_cycles":63343616,"thread_start_us":247587,"threads_started":1,"wal-append.queue_time_us":1941101}
I20260709 13:19:45.221194 18784 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:45.226090 19148 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:45.245095 16205 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:45.307361 16236 raft_consensus.cc:1240] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7. Ops: []
W20260709 13:19:45.308421 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 64399.7us
W20260709 13:19:45.308353 15525 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.015s	sys 0.023s
W20260709 13:19:45.309172 15525 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.015s	sys 0.023s
I20260709 13:19:45.235116 19062 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:45.318470 15493 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:19:45.318933 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 79676.8us
I20260709 13:19:45.319829 15567 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:45.318899 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 80021.9us
I20260709 13:19:45.245394 19072 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:45.320680 15566 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:45.323323 15567 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:45.304852 19018 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:45.306653 19020 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:45.306922 19105 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:45.317061 19168 consensus_meta.cc:287] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.738s	user 0.010s	sys 0.000s
I20260709 13:19:45.308563 19181 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:45.310336 19148 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:45.237488 19033 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:45.340790 19214 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:45.365973 19018 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:45.374241 19214 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:45.374684 19214 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:45.375229 19217 log.cc:927] Time spent T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:19:45.375934 19072 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:45.369288 19090 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:45.379408 19090 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:45.381357 19110 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:45.382647 19061 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:45.382771 19046 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:45.366892 19020 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:45.383420 19214 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:45.385607 19020 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:45.369555 19105 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:45.387547 19215 log.cc:927] Time spent T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:19:45.370471 19168 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:45.388031 19147 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:45.333571 19059 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:45.390254 19203 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:45.437067 19213 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:45.437242 19203 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:45.437685 19213 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:45.439412 19203 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:45.439706 19213 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:45.440317 19147 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:45.392607 19166 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:45.393231 19224 log.cc:927] Time spent T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.060s	user 0.002s	sys 0.000s
W20260709 13:19:45.442284 19223 log.cc:927] Time spent T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 13:19:45.383358 15521 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1913) took 1850 ms (client timeout 2099 ms). Trace:
W20260709 13:19:45.442906 15521 rpcz_store.cc:269] 0709 13:19:43.532899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.533001 (+   102us) service_pool.cc:224] Handling call
0709 13:19:45.383110 (+1850109us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b540029c7d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:45.383264 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1398680320}
I20260709 13:19:45.443435 19166 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:45.448632 19174 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:45.449808 19170 consensus_meta.cc:287] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.543s	user 0.004s	sys 0.006s
I20260709 13:19:45.450392 19170 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:45.452670 19170 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:45.340678 19218 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:19:45.342994 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 134998us
W20260709 13:19:45.454838 15528 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56840 (request call id 1027) took 1886 ms (client timeout 1928 ms). Trace:
I20260709 13:19:45.455389 19148 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:45.455767 15596 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53324 (request call id 1867) took 1153 ms (client timeout 1505 ms). Trace:
I20260709 13:19:45.421096 19214 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:45.456856 16238 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 805) took 1551 ms (client timeout 1611 ms). Trace:
I20260709 13:19:45.457576 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1921) took 1722 ms. Trace:
W20260709 13:19:45.458153 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 236269us
I20260709 13:19:45.576370 15526 rpcz_store.cc:276] 0709 13:19:43.735508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.849566 (+114058us) service_pool.cc:224] Handling call
0709 13:19:43.849810 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:43.868331 (+ 18521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:43.868362 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:43.868402 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:43.868432 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.457357 (+1588925us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b54003cdcd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:45.457400 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:45.457517 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42142464}
W20260709 13:19:45.575654 16238 rpcz_store.cc:269] 0709 13:19:43.905269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.906851 (+  1582us) service_pool.cc:224] Handling call
0709 13:19:45.456623 (+1549772us) spinlock_profiling.cc:243] Waited 525 ms on lock 0x7b54001cfcd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:45.456756 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1207150848}
I20260709 13:19:45.458456 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1868) took 1456 ms. Trace:
I20260709 13:19:45.578822 15527 rpcz_store.cc:276] 0709 13:19:44.002503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.308581 (+306078us) service_pool.cc:224] Handling call
0709 13:19:44.308892 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.309101 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:45.153282 (+844181us) log.cc:844] Serialized 14 byte log entry
0709 13:19:45.458101 (+304819us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:45.458155 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:45.458221 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.458285 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:45.458400 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56103,"raft.queue_time_us":93689,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":4581,"thread_start_us":247260,"threads_started":2,"wal-append.queue_time_us":172221}
W20260709 13:19:45.456636 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 119510us
W20260709 13:19:45.463032 19221 log.cc:927] Time spent T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.223s	user 0.000s	sys 0.000s
W20260709 13:19:45.464300 19199 log.cc:927] Time spent T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.876s	user 0.000s	sys 0.001s
I20260709 13:19:45.467361 18854 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:45.467622 19061 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:19:45.467430 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 222405us
I20260709 13:19:45.470081 19078 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:45.485462 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:45.598841 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:45.599383 16237 raft_consensus.cc:2376] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10. Re-sending same reply.
I20260709 13:19:45.600268 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:45.485877 16231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 802) took 2009 ms (client timeout 1801 ms). Trace:
W20260709 13:19:45.601854 16231 rpcz_store.cc:269] 0709 13:19:43.476824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.544003 (+ 67179us) service_pool.cc:224] Handling call
0709 13:19:45.485683 (+1941680us) spinlock_profiling.cc:243] Waited 625 ms on lock 0x7b54004cf4d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:45.485797 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1438230784}
I20260709 13:19:45.602891 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:45.603281 15601 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:19:45.603709 15601 pending_rounds.cc:85] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting all ops after (but not including) 7
I20260709 13:19:45.603897 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:45.604532 16231 raft_consensus.cc:2376] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11. Re-sending same reply.
I20260709 13:19:45.605337 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:45.606029 15597 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:45.606449 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:45.488014 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:45.607026 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:45.488760 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:45.437470 19072 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:45.372969 19181 raft_consensus.cc:697] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:45.496493 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:45.499356 19105 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:45.501480 15675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 982) took 1195 ms. Trace:
I20260709 13:19:45.371986 18784 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:45.469910 19171 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:45.503840 19168 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:45.374923 19033 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:45.381518 19090 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:45.467868 19018 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:45.506417 15520 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1861) took 1728 ms (client timeout 2270 ms). Trace:
I20260709 13:19:45.506847 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:45.632180 15520 rpcz_store.cc:269] 0709 13:19:43.778066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.798885 (+ 20819us) service_pool.cc:224] Handling call
0709 13:19:45.506187 (+1707302us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540029bdd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:45.506340 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2532614528}
I20260709 13:19:45.632913 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:45.632218 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:45.509825 18955 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:45.510282 19038 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
W20260709 13:19:45.510938 19212 log.cc:927] Time spent T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.201s	user 0.003s	sys 0.000s
I20260709 13:19:45.514799 16236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1135) took 2215 ms. Trace:
W20260709 13:19:45.411401 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 179327us
I20260709 13:19:45.521797 15524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1855) took 3649 ms. Trace:
I20260709 13:19:45.634984 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:45.634821 15524 rpcz_store.cc:276] 0709 13:19:41.872584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.128060 (+255476us) service_pool.cc:224] Handling call
0709 13:19:42.128346 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.316586 (+2188240us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:44.316618 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:44.316667 (+    49us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:44.316710 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.521501 (+1204791us) spinlock_profiling.cc:243] Waited 2.19 s on lock 0x7b54003cf5d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:45.521554 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:45.521720 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6883,"raft.queue_time_us":456,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":32533,"spinlock_wait_cycles":5031721216}
I20260709 13:19:45.636482 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:45.637266 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:45.517870 19081 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:45.534788 19031 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:45.519985 19219 log.cc:927] Time spent T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
W20260709 13:19:45.550431 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 232038us
W20260709 13:19:45.570170 15528 rpcz_store.cc:269] 0709 13:19:43.568474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.568607 (+   133us) service_pool.cc:224] Handling call
0709 13:19:45.454641 (+1886034us) spinlock_profiling.cc:243] Waited 904 ms on lock 0x7b54000c03d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:45.454772 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2078252800}
W20260709 13:19:45.572187 15596 rpcz_store.cc:269] 0709 13:19:44.302759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.303710 (+   951us) service_pool.cc:224] Handling call
0709 13:19:45.455543 (+1151833us) spinlock_profiling.cc:243] Waited 871 ms on lock 0x7b540030d254. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:45.455704 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2002638592}
I20260709 13:19:45.588003 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:45.590905 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:45.587211 19205 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:45.590482 19214 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:45.595486 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:45.667516 15602 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:45.595997 19218 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:45.608417 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:45.608562 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:45.608909 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:45.606194 15676 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:19:45.603943 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:45.671617 15676 raft_consensus.cc:606] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Received request to transfer leadership to TS 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:45.672039 15676 raft_consensus.cc:614] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:45.603924 15601 pending_rounds.cc:107] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:19:45.672569 15601 raft_consensus.cc:2889] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:45.606789 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:45.613132 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.616214 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:45.673604 15672 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:45.618784 15525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1040) took 1102 ms. Trace:
I20260709 13:19:45.623680 16239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1153) took 1422 ms. Trace:
I20260709 13:19:45.618026 15675 rpcz_store.cc:276] 0709 13:19:44.306173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.306307 (+   134us) service_pool.cc:224] Handling call
0709 13:19:44.960369 (+654062us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.960609 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:44.960650 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:44.960691 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:44.960742 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.960799 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:45.501297 (+540498us) spinlock_profiling.cc:243] Waited 654 ms on lock 0x7b540042a750. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:45.501412 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1503623936}
W20260709 13:19:45.629832 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 309211us
I20260709 13:19:45.588747 19209 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:45.677186 19205 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:45.678802 19181 consensus_queue.cc:237] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:45.679373 19205 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:45.638863 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:45.684945 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:45.685613 15602 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:45.686882 15673 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:45.689639 15602 raft_consensus.cc:1240] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:45.690073 19166 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.690779 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:45.691427 15596 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:19:45.647044 15642 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:45.692440 19081 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:45.651834 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:45.653262 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:45.655006 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:45.661252 15493 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:45.695286 15527 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:45.637542 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:45.662372 15671 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:19:45.662796 16238 raft_consensus.cc:2393] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 11: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:45.664256 15491 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:45.665692 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:45.624007 18784 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:45.674072 15525 rpcz_store.cc:276] 0709 13:19:44.516353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.588644 (+ 72291us) service_pool.cc:224] Handling call
0709 13:19:44.588925 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.589147 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:44.610398 (+ 21251us) log.cc:844] Serialized 14 byte log entry
0709 13:19:45.618461 (+1008063us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:45.618508 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:45.618567 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.618630 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:45.618731 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":21825,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":64272,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"thread_start_us":998381,"threads_started":2,"wal-append.queue_time_us":783312}
I20260709 13:19:45.661476 18440 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:45.652724 19038 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:45.638135 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:45.690598 19166 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:45.634482 16236 rpcz_store.cc:276] 0709 13:19:43.298912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.299943 (+  1031us) service_pool.cc:224] Handling call
0709 13:19:43.300232 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:45.307896 (+2007664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.514511 (+206615us) spinlock_profiling.cc:243] Waited 2.01 s on lock 0x7b54003c9454. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:45.514708 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4615593728}
I20260709 13:19:45.705499 15526 raft_consensus.cc:2376] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11. Re-sending same reply.
I20260709 13:19:45.652515 18955 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:45.692564 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 45594.7us
I20260709 13:19:45.712566 19118 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:45.712566 19185 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:45.712950 15673 raft_consensus.cc:1240] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:45.732698 19185 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:45.674378 16239 rpcz_store.cc:276] 0709 13:19:44.194043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.194888 (+   845us) service_pool.cc:224] Handling call
0709 13:19:44.218410 (+ 23522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:44.282139 (+ 63729us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:44.282170 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:45.389546 (+1107376us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b2c00047828. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce3304a11 00007f5ce3314372 00007f5ce4a46a01 00007f5ce4a0b43a 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2
0709 13:19:45.389987 (+   441us) log.cc:844] Serialized 26 byte log entry
0709 13:19:45.434112 (+ 44125us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:45.434151 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.434230 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:45.592941 (+158711us) raft_consensus.cc:1685] finished
0709 13:19:45.592958 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:45.616726 (+ 23768us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63289,"raft.queue_time_us":63715,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"spinlock_wait_cycles":29700864,"thread_start_us":43735,"threads_started":1,"wal-append.queue_time_us":155338}
I20260709 13:19:45.606261 15420 catalog_manager.cc:5697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 3 to 8, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 8 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:45.736733 19171 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:45.737311 19171 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:45.738819 19208 log.cc:927] Time spent T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:19:45.722539 19033 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:45.741582 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:45.723081 19209 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:45.651587 19090 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:19:45.713117 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 50766.1us
I20260709 13:19:45.721366 15527 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:19:45.712648 18784 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:45.712807 19214 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:45.731710 19118 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:45.704823 19165 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:45.729481 19038 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:45.775127 15672 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:45.618062 19072 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:45.794588 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 69651us
I20260709 13:19:45.797128 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:46.035799 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:46.036782 15493 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:45.861785 15669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 988) took 1530 ms. Trace:
W20260709 13:19:45.888571 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 139149us
I20260709 13:19:46.038566 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:46.033396 16205 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:46.042196 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:45.960214 15595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 834) took 1892 ms. Trace:
I20260709 13:19:46.051097 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:46.032428 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:46.065412 15596 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:46.033831 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:45.827059 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.797959 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:46.047644 15597 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
W20260709 13:19:46.031687 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.071838 15493 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:45.918478 15594 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:19:45.746059 19081 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:46.071552 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:46.069253 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:46.052907 16203 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:45.862282 18955 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:46.074066 15603 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:46.046603 19098 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.073908 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:46.037868 15669 rpcz_store.cc:276] 0709 13:19:44.331766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.331891 (+   125us) service_pool.cc:224] Handling call
0709 13:19:44.332221 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:44.332594 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:44.332622 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:44.561589 (+228967us) log.cc:844] Serialized 26 byte log entry
0709 13:19:44.975380 (+413791us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:44.975424 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.975499 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:45.510209 (+534710us) spinlock_profiling.cc:243] Waited 2.85 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:19:45.861501 (+351292us) raft_consensus.cc:1685] finished
0709 13:19:45.861523 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:45.861718 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45769,"raft.queue_time_us":351,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":11045,"spinlock_wait_cycles":6549248,"thread_start_us":355185,"threads_started":1,"wal-append.queue_time_us":634929}
I20260709 13:19:46.081985 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:46.051287 19148 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:46.047998 19151 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:46.098183 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 45333.6us
I20260709 13:19:46.106914 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:46.099658 18440 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:46.108204 18440 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:46.052264 19231 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:46.064064 19147 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:46.089458 16204 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:46.084295 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:46.058663 15595 rpcz_store.cc:276] 0709 13:19:44.067377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.068230 (+   853us) service_pool.cc:224] Handling call
0709 13:19:44.068525 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.151461 (+ 82936us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:44.266254 (+114793us) log.cc:844] Serialized 14 byte log entry
0709 13:19:45.453169 (+1186915us) spinlock_profiling.cc:243] Waited 5.07 ms on lock 0x7b1800472300. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab23a 00007f5ce4a15a2a
0709 13:19:45.726500 (+273331us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:45.726559 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:45.726620 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.959902 (+233282us) spinlock_profiling.cc:243] Waited 82.7 ms on lock 0x7b540030dc54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:45.959974 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:45.960125 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":175996,"prepare.queue_time_us":114466,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":28829,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":28828,"spinlock_wait_cycles":201759744,"thread_start_us":272636,"threads_started":1}
I20260709 13:19:46.109529 18440 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:46.109946 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:46.099700 16205 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:46.097119 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 306589us
I20260709 13:19:45.826170 19090 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:45.723186 19218 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.086982 19167 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:46.118628 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:46.107612 16203 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:46.092312 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:46.120445 15493 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:46.112527 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:46.114705 16232 raft_consensus.cc:1240] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:46.083467 19081 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:46.121621 19081 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:46.122810 19181 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:46.125020 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:46.126026 19018 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:46.083665 19117 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:19:46.126372 19232 log.cc:927] Time spent T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:19:46.126737 19033 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:46.095423 18955 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.129256 18955 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.111716 16237 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:46.120249 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:46.067692 19189 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:19:46.130290 19185 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:46.130944 19147 raft_consensus.cc:697] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:46.103039 19205 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.132340 18784 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:46.135079 19078 consensus_meta.cc:287] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.539s	user 0.002s	sys 0.006s
I20260709 13:19:46.135422 19209 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:46.176923 19237 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.177268 19205 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.098335 19098 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:45.751308 15420 catalog_manager.cc:5697] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 3 to 4, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 4 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:46.178628 18854 consensus_queue.cc:1059] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:46.179603 19205 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:46.138411 19214 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:46.139087 19218 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:46.179862 19235 consensus_queue.cc:1059] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:19:46.113888 19151 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.141539 19090 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:46.141793 19118 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:46.150429 15525 raft_consensus.cc:1240] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:46.151764 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:46.152441 15493 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:46.276753 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 124387us
I20260709 13:19:46.277467 15493 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:46.277599 15598 raft_consensus.cc:1240] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 6. Current term is 7. Ops: []
W20260709 13:19:46.153345 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 53895.8us
I20260709 13:19:46.150993 19018 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.157902 15668 raft_consensus.cc:1240] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:46.177047 19185 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.162632 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:46.167481 18440 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.160094 18676 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:19:46.170171 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 62600us
I20260709 13:19:46.171900 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:46.296204 15493 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:46.173401 19147 consensus_queue.cc:237] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.298368 18676 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.298838 19167 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:46.174103 18955 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:46.299690 19090 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.300191 19090 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.300840 18676 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:46.301180 19098 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:46.302798 19090 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:46.304747 19072 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:46.306412 19018 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:46.306644 19218 raft_consensus.cc:3055] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:46.307085 19218 raft_consensus.cc:740] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:46.307860 19218 consensus_queue.cc:260] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.309007 19218 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:46.310132 18440 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:46.313987 19185 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:46.316275 19185 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:46.174619 19180 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:46.175422 19236 consensus_queue.cc:1059] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:46.317693 19180 raft_consensus.cc:697] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:46.151311 19046 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:46.318203 19018 raft_consensus.cc:3055] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:46.318454 19046 raft_consensus.cc:697] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:46.180054 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:46.318717 19018 raft_consensus.cc:740] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:46.319416 19043 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:46.319612 19018 consensus_queue.cc:260] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.319929 19046 consensus_queue.cc:237] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.320417 16237 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:19:46.320902 19018 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:46.180645 15677 raft_consensus.cc:1240] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:46.323695 15671 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:46.184687 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:46.186720 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:46.324921 16234 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:19:46.192358 19171 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.195539 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:46.332837 15526 raft_consensus.cc:2393] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 7: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:46.249105 15528 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:19:46.251358 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:46.266424 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 74861.9us
I20260709 13:19:46.336395 15568 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:46.334522 15522 raft_consensus.cc:2410] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:19:46.163419 19031 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:46.163193 19062 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:19:46.339401 19241 log.cc:927] Time spent T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:19:46.339522 19062 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.341351 18784 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:46.158313 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:46.169636 19117 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:46.292636 15520 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:46.342464 15420 catalog_manager.cc:5697] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
W20260709 13:19:46.343430 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 89077.5us
I20260709 13:19:46.346163 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:46.347458 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:46.189718 19166 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:46.181452 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:46.318681 19180 consensus_queue.cc:237] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.338653 19236 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:46.348383 16237 raft_consensus.cc:606] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:46.292354 19231 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:46.361260 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:46.351688 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:46.325994 19033 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:46.362665 16204 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:46.362021 16237 raft_consensus.cc:614] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:46.391714 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:46.196951 19237 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:46.368139 15669 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:19:46.407126 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:46.408039 15668 raft_consensus.cc:2393] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 9: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:46.409686 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:46.412247 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:46.413805 15671 raft_consensus.cc:2410] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:19:46.393891 19033 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:46.418084 19166 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:46.390894 19211 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:19:46.418673 19166 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:46.390362 19210 consensus_queue.cc:1059] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:19:46.385429 18601 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:46.420920 19166 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:46.421047 18601 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.422914 16236 raft_consensus.cc:1240] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 12. Current term is 13. Ops: []
W20260709 13:19:46.388187 19244 log.cc:927] Time spent T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:19:46.341786 19062 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:19:46.427983 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 65382.6us
I20260709 13:19:46.418438 19033 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:46.430001 15671 raft_consensus.cc:1240] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:46.430470 19189 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:46.434232 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:46.391371 19236 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:46.361650 19165 raft_consensus.cc:697] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:46.427453 19105 raft_consensus.cc:3055] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:46.442229 15520 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:19:46.345072 19081 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.445873 16204 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:46.429651 19233 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:46.441711 19189 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:46.434886 19210 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:46.443276 19236 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:46.439635 19018 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:46.459085 15566 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:46.459384 15527 raft_consensus.cc:1240] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:46.444092 19105 raft_consensus.cc:740] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:46.450443 19050 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:46.465480 19050 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:19:46.468752 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 366902us
I20260709 13:19:46.452335 19189 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:46.467897 18440 raft_consensus.cc:3055] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:46.488296 18440 raft_consensus.cc:740] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:46.492144 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:46.493070 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:46.493422 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:46.493799 16234 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:46.467995 19050 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:46.502465 19243 log.cc:927] Time spent T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:19:46.469398 19205 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:46.503352 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:46.504057 15600 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:46.460578 19210 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.506269 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:46.507022 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:46.507594 16237 raft_consensus.cc:1813] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Handling vote request from an unknown peer 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:46.508038 16237 raft_consensus.cc:2393] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 5: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:46.465693 19151 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:46.461355 19018 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.509634 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:46.509634 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:46.511221 19050 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.513430 19050 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:46.514021 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:46.514568 15520 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:46.514819 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:46.515029 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1046) took 1294 ms. Trace:
I20260709 13:19:46.515435 15523 rpcz_store.cc:276] 0709 13:19:45.220607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.368344 (+147737us) service_pool.cc:224] Handling call
0709 13:19:45.368711 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:45.369050 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:45.369078 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:45.430801 (+ 61723us) log.cc:844] Serialized 26 byte log entry
0709 13:19:46.036587 (+605786us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:46.036627 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.036689 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:46.387603 (+350914us) spinlock_profiling.cc:243] Waited 3.67 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:19:46.514740 (+127137us) raft_consensus.cc:1685] finished
0709 13:19:46.514761 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:46.514972 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7646,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":78,"spinlock_wait_cycles":8451456,"thread_start_us":274874,"threads_started":1,"wal-append.queue_time_us":647871}
I20260709 13:19:46.516959 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:46.452245 19233 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.463619 19214 consensus_queue.cc:1059] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 13:19:46.519042 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:46.519642 15600 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:46.520556 19105 consensus_queue.cc:260] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.521273 19148 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:46.521836 19148 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:46.521734 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:46.522984 19237 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:46.523926 19148 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:46.525101 15520 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:19:46.521843 19105 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:46.514461 18601 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:46.531105 19081 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:46.532212 19171 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:46.533428 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:46.489221 18440 consensus_queue.cc:260] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.535359 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:46.536238 15671 raft_consensus.cc:1275] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:19:46.536749 16204 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:46.538419 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:46.488034 19165 consensus_queue.cc:237] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.538990 15526 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:46.540781 15671 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:19:46.543642 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:46.519742 19214 raft_consensus.cc:3055] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:46.465757 15566 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:46.553979 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:46.562091 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 96404.9us
W20260709 13:19:46.565701 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.532866 19148 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:46.565196 15642 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:46.508129 19210 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:46.535846 19151 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:46.591626 15526 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:19:46.552994 19233 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:46.592883 16204 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:46.593458 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.552284 19118 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:46.539772 18955 consensus_queue.cc:1059] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:19:46.560868 19214 raft_consensus.cc:740] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:46.581007 19090 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:46.567467 19237 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:46.621878 19062 consensus_queue.cc:1059] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:46.623041 19062 raft_consensus.cc:3055] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:46.623373 19062 raft_consensus.cc:740] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:46.624158 19062 consensus_queue.cc:260] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.625276 19062 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:46.626070 19118 raft_consensus.cc:697] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:46.626978 19118 consensus_queue.cc:237] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.601827 19210 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:46.592374 19151 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:46.633178 15642 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:46.633996 15668 raft_consensus.cc:1240] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:46.581818 19148 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:46.613705 19018 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:46.617111 19090 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:46.640833 19090 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:46.687130 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:46.639391 19210 raft_consensus.cc:697] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:46.687851 15672 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:46.688354 19210 consensus_queue.cc:237] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.689705 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:46.694554 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:46.695726 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 399573us
I20260709 13:19:46.697784 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:46.652884 19185 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:46.632483 19166 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:46.699918 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:46.700544 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:46.745862 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:46.702049 15673 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:19:46.703970 19151 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:46.704576 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:46.704855 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:46.698815 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:46.708457 15523 raft_consensus.cc:1240] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:46.699033 19098 consensus_queue.cc:1059] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 13:19:46.752526 15594 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:46.754639 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:46.756560 19148 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:46.732754 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:46.745175 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:46.734269 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 130618us
W20260709 13:19:46.749061 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 361308us
I20260709 13:19:46.756240 19033 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.650411 19214 consensus_queue.cc:260] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:46.724671 19185 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:46.758170 19078 consensus_queue.cc:1059] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:46.758594 19185 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:46.759024 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:46.759244 19033 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:46.759970 19251 raft_consensus.cc:3055] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:46.760340 19251 raft_consensus.cc:740] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:46.761137 19251 consensus_queue.cc:260] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.763168 19251 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:46.757344 15601 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:46.698374 15493 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:46.769627 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:46.769958 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 72221.5us
I20260709 13:19:46.770457 15596 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:46.770673 15642 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:19:46.770673 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 72367us
I20260709 13:19:46.772300 15493 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:46.768257 15566 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:46.780354 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:46.782644 15642 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:46.737908 19147 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.793748 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:46.811645 15601 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:19:46.815156 15566 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:46.795260 15594 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:19:46.816527 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:46.816906 15594 pending_rounds.cc:85] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting all ops after (but not including) 9
I20260709 13:19:46.817222 15594 pending_rounds.cc:107] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:19:46.817467 15594 raft_consensus.cc:2889] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:46.822391 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 272141us
I20260709 13:19:46.824746 15493 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:46.801011 16205 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:46.831241 15566 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:46.779333 19033 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:46.765756 19236 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.753696 18955 consensus_queue.cc:1059] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:19:46.783835 19148 raft_consensus.cc:3055] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:46.840843 15493 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:46.781395 19253 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:46.861284 15566 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:46.795110 19098 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:19:46.862234 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 53606.5us
I20260709 13:19:46.794210 19063 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.789620 19151 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:46.751452 19166 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:46.781070 19255 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:19:46.863507 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 67931.1us
W20260709 13:19:46.863845 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 62903.1us
I20260709 13:19:46.794565 19235 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:46.863752 19081 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:46.869470 19248 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:46.870360 19098 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:46.870939 19205 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:46.875525 19236 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:46.878823 19249 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:46.865084 19148 raft_consensus.cc:740] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:46.834976 19018 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:46.905897 19214 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:46.861891 19105 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:46.805122 19090 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:46.914464 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 73711.1us
I20260709 13:19:46.945640 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:46.915076 18440 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:46.916301 19260 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:46.919549 19168 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:46.934939 19098 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:46.813033 19218 consensus_queue.cc:1059] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.113s
I20260709 13:19:46.866839 19250 consensus_queue.cc:1059] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:19:46.813534 19185 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:46.929563 16205 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:46.930943 16203 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:46.795398 19147 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:46.868535 15566 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:46.929268 19151 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:46.950774 15493 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:46.933127 19063 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:46.962698 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 94242.9us
I20260709 13:19:46.970125 15493 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:46.939395 19018 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:46.972044 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:19:46.976779 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 45921.4us
W20260709 13:19:47.019510 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 45761.2us
W20260709 13:19:46.976336 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 46886us
I20260709 13:19:47.023041 19168 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:46.954958 19081 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:46.961680 19209 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.032920 15674 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
W20260709 13:19:46.988965 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:47.037459 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:47.038337 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 64175.1us
I20260709 13:19:46.937155 19189 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:47.034286 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.043707 16205 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:47.059701 15526 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:19:46.975965 19255 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:19:47.103334 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 286878us
I20260709 13:19:47.061545 16204 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:47.104285 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:47.092725 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.105757 15642 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:47.028278 19043 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:46.975747 19205 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:19:47.118040 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 147984us
I20260709 13:19:47.027568 19265 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:47.120736 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.125244 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.126111 15525 raft_consensus.cc:2393] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 8: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:47.120733 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.121371 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.122366 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:19:47.123281 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 80257.1us
I20260709 13:19:46.968644 19151 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:47.127746 16203 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:47.123622 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 79997.8us
I20260709 13:19:47.128724 16205 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:47.124163 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:47.127209 15527 raft_consensus.cc:2368] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9.
I20260709 13:19:47.130092 15519 raft_consensus.cc:1240] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:47.019021 19236 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.089s
I20260709 13:19:46.976804 19018 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:47.132400 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 70974.5us
I20260709 13:19:46.972806 18601 raft_consensus.cc:3055] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:47.132447 15524 raft_consensus.cc:1240] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:47.139232 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:47.134907 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.135176 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:47.119179 19244 log.cc:927] Time spent T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:19:47.140635 15525 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:19:47.137526 15523 raft_consensus.cc:1240] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:47.141991 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.142275 16205 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:47.131695 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.143419 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.143962 15520 raft_consensus.cc:1240] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:47.149837 15522 raft_consensus.cc:1813] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Handling vote request from an unknown peer 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:47.150491 15522 raft_consensus.cc:2410] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:19:47.151837 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:47.157572 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:47.158034 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 52370.9us
I20260709 13:19:47.158535 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.157886 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:47.158963 15642 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:47.159497 15524 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:47.159873 15642 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:47.162726 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.162786 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:47.162887 15520 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:47.164640 16234 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:47.163365 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.166253 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:47.166674 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.167249 15523 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:47.170151 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:47.170722 16236 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:47.172725 15524 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:47.173786 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.174427 15520 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:47.175215 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.175761 15520 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:47.176508 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.177026 15520 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:47.130628 19265 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:47.177728 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:47.177757 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:47.192618 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.029052 19148 consensus_queue.cc:260] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:47.140264 18676 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:47.241232 19270 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 13:19:47.148643 19050 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:47.242365 19050 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:47.242766 19050 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:47.179966 16236 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
W20260709 13:19:47.211201 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 69262.2us
I20260709 13:19:47.103648 19209 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:47.132179 19255 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:47.251718 19166 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:47.252374 19166 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:47.252864 19209 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:47.211434 19062 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:47.255262 18676 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:19:47.255841 19273 log.cc:927] Time spent T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:19:47.257658 19185 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:47.258467 19185 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:47.258826 19185 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:47.259146 19185 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:47.230592 19211 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
W20260709 13:19:47.266331 19262 log.cc:927] Time spent T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
I20260709 13:19:47.271102 19189 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:47.148900 19220 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:47.271956 19220 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:47.272379 19220 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:47.306299 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 151191us
I20260709 13:19:47.138286 18601 raft_consensus.cc:740] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:47.150898 19063 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:47.152359 19165 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:19:47.308846 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 149058us
W20260709 13:19:47.313586 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 62183.5us
I20260709 13:19:47.153223 19275 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:47.314592 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:47.303982 19265 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:47.326427 16203 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:19:47.333662 19271 log.cc:927] Time spent T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:19:47.311084 19280 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:47.250617 19255 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:47.347456 15521 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:47.349030 19105 raft_consensus.cc:3055] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:47.349891 19105 raft_consensus.cc:740] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:47.351006 19105 consensus_queue.cc:260] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:47.352108 19105 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:47.318126 19275 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:47.363116 15641 leader_election.cc:336] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.077s (SENT)
W20260709 13:19:47.384964 15641 leader_election.cc:336] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.099s (SENT)
I20260709 13:19:47.385509 15641 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:47.384819 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 58486us
I20260709 13:19:47.313019 19128 consensus_queue.cc:1059] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:19:47.318322 19209 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:19:47.338896 19147 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:47.388509 19171 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:47.418238 15596 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:47.401988 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:47.402275 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:47.414841 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:47.388798 19275 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:47.431318 19043 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:19:47.410408 19268 log.cc:927] Time spent T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
I20260709 13:19:47.413084 19018 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:47.451066 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.411365 19248 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:47.451745 19018 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:47.441622 18676 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:47.452575 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:47.397305 19283 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:47.454282 19283 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:47.454674 19283 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:47.460867 19275 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:47.463052 19275 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:47.463178 15566 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:47.416966 19235 raft_consensus.cc:3055] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:47.466390 19235 raft_consensus.cc:740] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:47.467445 19050 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:47.467381 19235 consensus_queue.cc:260] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:47.468850 19235 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:47.469611 19050 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:47.399078 19282 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:47.412526 19090 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:47.414426 19280 raft_consensus.cc:697] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:47.490348 19118 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:47.497301 15668 raft_consensus.cc:2410] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 10 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:19:47.497200 19118 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:47.451871 19248 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:47.488726 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 174196us
I20260709 13:19:47.470341 18784 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:47.498987 18784 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:47.499198 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:47.497460 19280 consensus_queue.cc:237] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:47.500159 19282 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:19:47.502652 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.489329 15603 raft_consensus.cc:1240] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:47.502656 15641 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:47.398012 15521 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
W20260709 13:19:47.442377 19253 consensus_meta.cc:287] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.520s	user 0.007s	sys 0.005s
I20260709 13:19:47.477234 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:47.470628 19255 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:47.485966 19278 consensus_queue.cc:1059] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:19:47.508684 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.508206 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:47.511312 15568 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:47.487257 19147 consensus_queue.cc:1059] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:47.487915 19165 consensus_queue.cc:1059] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:47.488507 19274 consensus_queue.cc:1059] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:47.394994 18601 consensus_queue.cc:260] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:47.431485 19098 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:47.507287 16236 raft_consensus.cc:1240] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 12. Current term is 13. Ops: []
W20260709 13:19:47.507862 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.477703 15599 raft_consensus.cc:1240] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:47.512573 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:19:47.498703 19254 log.cc:927] Time spent T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.732s	user 0.010s	sys 0.001s
I20260709 13:19:47.514092 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:47.536988 15568 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:47.507215 15641 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:47.556529 15525 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:19:47.506407 18784 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:47.566161 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:47.566885 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:47.566740 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.568029 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:47.566794 15597 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:47.569326 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:47.569983 16232 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:47.570004 16234 raft_consensus.cc:1240] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12. Ops: []
W20260709 13:19:47.518733 19214 consensus_meta.cc:287] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.574s	user 0.011s	sys 0.001s
I20260709 13:19:47.512169 19279 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:47.574828 16234 raft_consensus.cc:1240] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:47.582247 16236 raft_consensus.cc:1240] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:47.582369 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:47.583011 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:47.515460 19033 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:47.514046 19236 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:47.517791 19253 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:47.533318 19251 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:47.590097 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:47.502712 19118 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:47.592772 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:47.594277 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:47.604628 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.519351 19018 consensus_queue.cc:1059] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:19:47.607693 15672 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 7: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:47.645396 16234 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:19:47.647130 15674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:19:47.608449 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:47.572765 19233 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:47.610843 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:47.610384 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:47.651655 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:47.652544 15597 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:19:47.653738 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:47.654881 19279 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:47.554767 19081 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:19:47.608743 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:47.660995 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:47.664355 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:47.667438 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:47.668077 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:47.668881 16238 raft_consensus.cc:2393] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 9: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:47.671855 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:47.679401 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:47.678951 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 96728.4us
I20260709 13:19:47.595381 19275 raft_consensus.cc:3055] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:47.689563 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:47.565279 19276 consensus_queue.cc:1048] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.130s
I20260709 13:19:47.692780 15523 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:47.608393 19251 raft_consensus.cc:697] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:47.698789 15676 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:47.699908 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:47.702917 15525 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:19:47.703998 15596 raft_consensus.cc:1275] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:19:47.704803 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:47.686956 19270 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:19:47.596657 19033 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:47.672811 19279 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:47.713694 15596 raft_consensus.cc:1240] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:47.713444 16236 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:19:47.714136 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.714958 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.715699 15526 raft_consensus.cc:1240] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:19:47.598784 19236 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:47.720400 15528 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:47.721297 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.722045 15526 raft_consensus.cc:2393] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 6: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:47.716639 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.690281 19018 consensus_queue.cc:1243] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer dd1a14c3173d49dc9b55bb116e3783c9 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:19:47.722882 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:47.723068 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.723938 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.724195 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.721323 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.725271 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.726402 15526 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:47.693104 19275 raft_consensus.cc:740] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:47.608556 19288 consensus_queue.cc:1059] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.108s
I20260709 13:19:47.731769 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.733947 15520 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:47.734926 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.737130 15519 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:47.740515 15603 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:47.721692 19279 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:47.745330 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:47.746497 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.748224 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.749222 15526 raft_consensus.cc:2393] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 6: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:19:47.754590 19284 log.cc:927] Time spent T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
W20260709 13:19:47.756361 19264 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.709s	user 0.001s	sys 0.009s
I20260709 13:19:47.757746 16232 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:47.767598 19061 raft_consensus.cc:3055] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:47.768075 19061 raft_consensus.cc:740] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:47.768872 19061 consensus_queue.cc:260] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:47.771569 19236 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:47.772228 19275 consensus_queue.cc:260] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:47.773258 19061 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:47.773344 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:47.774171 19251 consensus_queue.cc:237] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:47.775246 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.777031 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.774080 19275 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:47.780148 16237 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:19:47.788283 15595 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:19:47.796552 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.790413 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.791193 19283 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:47.737414 19260 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.059s
W20260709 13:19:47.794615 19151 consensus_meta.cc:287] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.647s	user 0.008s	sys 0.003s
I20260709 13:19:47.788492 15677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 1019) took 1351 ms. Trace:
I20260709 13:19:47.769312 19220 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:47.801867 15522 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:47.807277 15603 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:19:47.808132 19098 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:47.807024 15677 rpcz_store.cc:276] 0709 13:19:46.437362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.438164 (+   802us) service_pool.cc:224] Handling call
0709 13:19:46.438470 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:46.438810 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:46.438835 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:46.443108 (+  4273us) log.cc:844] Serialized 26 byte log entry
0709 13:19:46.866241 (+423133us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:46.866992 (+   751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.867075 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:47.755822 (+888747us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:19:47.762587 (+  6765us) raft_consensus.cc:1685] finished
0709 13:19:47.762607 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:47.788417 (+ 25810us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":514,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":3391,"spinlock_wait_cycles":279452928,"thread_start_us":413319,"threads_started":1,"wal-append.queue_time_us":508615}
I20260709 13:19:47.807693 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.751742 19168 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.804479 15526 raft_consensus.cc:2393] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 8: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:19:47.805212 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 155266us
I20260709 13:19:47.814636 15524 raft_consensus.cc:3055] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:47.815141 15524 raft_consensus.cc:740] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
W20260709 13:19:47.817184 19166 consensus_meta.cc:287] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.564s	user 0.007s	sys 0.000s
I20260709 13:19:47.817402 15524 consensus_queue.cc:260] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:47.818583 15524 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:47.820878 15642 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:47.805536 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.823776 15522 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:19:47.824482 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:47.826073 15523 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:19:47.828601 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:47.828994 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.829770 15520 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:19:47.829241 16203 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:47.834627 19274 consensus_queue.cc:1059] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:47.835929 19296 consensus_queue.cc:1059] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:47.836755 19285 consensus_queue.cc:1059] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:47.837881 19285 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:47.837882 19165 raft_consensus.cc:3055] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:47.839181 19165 raft_consensus.cc:740] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:47.840041 19165 consensus_queue.cc:260] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:47.841401 19165 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:47.842010 15598 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:19:47.843845 15524 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 13:19:47.845485 15524 pending_rounds.cc:85] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 7
I20260709 13:19:47.757689 19185 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:47.846263 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:47.846849 15524 pending_rounds.cc:107] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:19:47.850807 15524 raft_consensus.cc:2889] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:47.841303 19282 raft_consensus.cc:3055] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:47.843274 19166 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:47.773813 19265 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:47.855113 19282 raft_consensus.cc:740] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:47.848417 19098 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:47.812659 19151 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:47.874884 15670 raft_consensus.cc:1240] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:47.901437 15567 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:47.830413 19168 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:47.909030 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:47.911271 16236 raft_consensus.cc:1240] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 7. Current term is 8. Ops: []
W20260709 13:19:47.913024 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 405862us
I20260709 13:19:47.914081 15641 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:47.867846 19301 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:47.916760 15641 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:47.917028 16231 raft_consensus.cc:1240] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:47.922199 15641 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:47.925985 15594 raft_consensus.cc:1240] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:47.925673 15641 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:47.849429 19248 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:47.929067 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 198036us
W20260709 13:19:47.931107 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.923260 15567 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:47.878193 19285 consensus_queue.cc:1059] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:19:47.918067 19098 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:47.907764 19211 consensus_queue.cc:1059] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:19:47.939146 19168 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:47.940200 19302 log.cc:927] Time spent T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:19:47.921190 19305 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:47.929354 19296 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:19:48.002758 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 69366us
W20260709 13:19:48.000787 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 128300us
W20260709 13:19:48.009179 19299 log.cc:927] Time spent T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:19:48.010614 18784 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:48.012036 15599 raft_consensus.cc:2393] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 10: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:48.018595 19283 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:48.022888 19185 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:47.936864 19255 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:48.023813 19255 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:47.934226 19098 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:48.026151 19098 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:48.040669 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:48.041448 15673 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:48.046092 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 175554us
I20260709 13:19:48.048661 15673 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:19:48.053787 16203 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:48.006417 19267 log.cc:927] Time spent T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.876s	user 0.010s	sys 0.014s
I20260709 13:19:48.047484 15493 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:48.068495 15673 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:48.000496 19297 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:48.008183 19296 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:19:48.066388 19307 log.cc:927] Time spent T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.101s	user 0.004s	sys 0.005s
I20260709 13:19:48.078143 19171 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:48.079197 19061 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:19:48.072670 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 147045us
W20260709 13:19:48.080937 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 72628.8us
I20260709 13:19:48.087199 15641 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:48.066977 19285 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:48.083303 19296 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:48.071135 19304 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:48.074024 15493 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:48.084479 15672 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 4.
I20260709 13:19:48.053761 19255 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:48.067185 19168 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:48.070158 19220 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:48.089629 19211 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:48.093055 19265 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:48.071488 19315 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:48.155490 15676 raft_consensus.cc:1240] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:48.176486 19296 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:48.185489 19282 consensus_queue.cc:260] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:48.240451 15671 raft_consensus.cc:1240] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:48.173249 19148 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:48.178189 15675 raft_consensus.cc:1240] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:19:48.242187 19210 consensus_queue.cc:1059] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:48.242522 15671 raft_consensus.cc:1240] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:48.243166 15675 raft_consensus.cc:1240] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:48.245150 19295 consensus_queue.cc:1059] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:48.245441 19281 consensus_queue.cc:1059] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:48.193385 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:48.246287 19276 consensus_queue.cc:1059] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:48.248087 15595 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:48.248435 19276 raft_consensus.cc:3055] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:48.247668 19295 raft_consensus.cc:3055] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:48.121738 19098 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:48.189219 18940 raft_consensus.cc:3055] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:48.251303 18940 raft_consensus.cc:740] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:48.164148 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1936) took 1026 ms. Trace:
I20260709 13:19:48.202299 19166 consensus_queue.cc:1059] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:19:48.143904 19279 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:48.254235 15677 raft_consensus.cc:1240] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:48.153146 19090 raft_consensus.cc:3055] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:48.254976 19090 raft_consensus.cc:740] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:48.247011 19281 raft_consensus.cc:3055] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:48.247503 19315 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:48.151969 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.249841 19295 raft_consensus.cc:740] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:48.256804 19295 consensus_queue.cc:260] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:48.258095 19295 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:48.089039 19285 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:48.258888 19285 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:48.259459 19281 raft_consensus.cc:740] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:48.180491 19046 raft_consensus.cc:3055] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:48.260404 19046 raft_consensus.cc:740] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:48.192348 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:48.260355 19281 consensus_queue.cc:260] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:48.160632 19033 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:48.261631 19046 consensus_queue.cc:260] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:48.186278 19253 raft_consensus.cc:3055] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:48.152704 19147 raft_consensus.cc:3055] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:48.261965 19281 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:48.263164 19147 raft_consensus.cc:740] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:48.263360 19165 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:48.263486 19046 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:48.173054 19128 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 13:19:48.097657 16204 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:48.094157 19309 raft_consensus.cc:3055] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:48.264420 19253 raft_consensus.cc:740] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:48.184569 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:48.207633 18601 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:48.265741 19282 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:48.186643 19237 raft_consensus.cc:3055] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:48.266957 19237 raft_consensus.cc:740] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:48.253508 19166 consensus_queue.cc:1243] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer 5498ef3e89a941e39c25f68cf64e6b0f log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:19:48.269331 19128 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:48.134335 19151 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:48.183166 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:48.268783 19237 consensus_queue.cc:260] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:48.252693 19276 raft_consensus.cc:740] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:48.128885 19248 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:48.270119 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:48.193454 15597 raft_consensus.cc:1240] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:48.271395 19276 consensus_queue.cc:260] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:48.252313 18940 consensus_queue.cc:260] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:48.151731 19061 consensus_queue.cc:1059] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.121s
I20260709 13:19:48.273056 19237 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:48.252252 15521 rpcz_store.cc:276] 0709 13:19:47.137227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.137354 (+   127us) service_pool.cc:224] Handling call
0709 13:19:48.163935 (+1026581us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b54003ce1d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:48.164077 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":448368384}
I20260709 13:19:48.201486 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:48.264528 19309 raft_consensus.cc:740] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:48.277628 18940 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:48.203318 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:48.181906 19305 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.227211 19297 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:48.253995 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:48.254892 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:48.273411 19038 raft_consensus.cc:3055] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:48.277956 16231 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10. Ops: []
W20260709 13:19:48.269024 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 171458us
I20260709 13:19:48.278438 19279 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:48.246975 19296 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:48.279385 16237 raft_consensus.cc:1275] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:19:48.169772 15528 raft_consensus.cc:1240] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:48.240345 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:48.270735 15598 raft_consensus.cc:2393] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 7: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
W20260709 13:19:48.247479 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.272639 19276 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:48.263994 19165 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:48.252833 19098 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:48.265990 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:48.294878 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 226670us
I20260709 13:19:48.295922 19309 consensus_queue.cc:260] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:48.297819 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 210693us
I20260709 13:19:48.135366 15566 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:48.131148 18784 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:48.098510 19274 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:48.269690 19090 consensus_queue.cc:260] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:48.318701 19220 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.319264 19220 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:48.321367 19220 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:48.321209 19285 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:48.297392 19038 raft_consensus.cc:740] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:48.316150 19309 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:48.317075 19165 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:48.313658 15641 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:48.299903 19305 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:48.322006 19061 consensus_queue.cc:1175] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" responder_term: 12 status { last_received { term: 12 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 12 index: 10 } }
I20260709 13:19:48.299466 19033 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:48.326589 19090 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:48.359874 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:48.360641 15602 raft_consensus.cc:2393] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 7: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:48.314563 15493 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
W20260709 13:19:48.333439 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 198146us
I20260709 13:19:48.353667 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:48.258349 19315 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:48.365025 15672 raft_consensus.cc:1240] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:48.355613 15676 raft_consensus.cc:1240] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
W20260709 13:19:48.357594 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 44032.4us
I20260709 13:19:48.366825 15641 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:48.361573 15598 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:48.358110 16204 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:48.376274 15677 raft_consensus.cc:1240] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:48.372471 16205 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:48.377417 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 62924.8us
I20260709 13:19:48.377784 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:48.378886 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:48.379158 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:48.379647 15527 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:48.380511 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:48.355613 18784 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:48.384698 15566 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:48.383111 15642 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:48.390905 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:48.396370 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:48.366894 19165 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:48.400677 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:48.395292 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:48.404978 15528 raft_consensus.cc:2393] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 8: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:48.405332 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:48.406477 15567 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:48.355844 19105 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:19:48.376814 19098 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:48.264155 19147 consensus_queue.cc:260] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:48.357126 19211 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:19:48.438681 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:48.413584 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:48.421404 15527 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:48.437799 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:48.439383 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:48.387459 19062 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:48.395099 19255 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:48.437425 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:48.424103 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:48.382735 19039 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.413482 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:48.434262 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:48.453783 15528 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:48.414359 16232 raft_consensus.cc:1240] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:19:48.439309 15526 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:48.453497 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:48.481191 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:48.481840 15527 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:19:48.416169 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:48.455526 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:48.393618 19078 raft_consensus.cc:2764] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:19:48.456100 15493 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:48.496985 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:48.457772 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:48.411271 18955 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:48.382490 19033 raft_consensus.cc:697] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:48.466827 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:48.468509 15641 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:48.395839 19236 consensus_queue.cc:1059] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:19:48.479197 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:48.485322 15527 raft_consensus.cc:1240] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:48.451917 19211 consensus_queue.cc:1243] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer d3c4f5bd76e0445795bad5f6dd93ff69 log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:19:48.502885 15520 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:19:48.475497 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:48.398157 19279 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:48.508766 18955 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:48.542603 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.518452 16232 raft_consensus.cc:1240] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9. Ops: []
W20260709 13:19:48.518981 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 50561.3us
I20260709 13:19:48.519263 15521 raft_consensus.cc:1240] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:48.519703 19098 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:48.525198 16237 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:48.527052 15528 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:48.527139 15595 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
W20260709 13:19:48.531219 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 55918.9us
I20260709 13:19:48.563326 15526 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:48.404238 19214 consensus_queue.cc:1059] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.099s
W20260709 13:19:48.476619 19287 log.cc:927] Time spent T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:19:48.400624 19251 consensus_queue.cc:1059] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.110s
W20260709 13:19:48.505460 19306 log.cc:927] Time spent T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:19:48.539100 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 132698us
W20260709 13:19:48.584290 15567 leader_election.cc:336] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.082s (SENT)
I20260709 13:19:48.509287 19165 raft_consensus.cc:697] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:48.584717 15567 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:48.543707 15641 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:48.555712 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.563490 16204 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:48.509333 18854 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:48.582831 15642 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:48.586615 16204 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:48.587494 19165 consensus_queue.cc:237] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:48.592047 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 48415.4us
I20260709 13:19:48.403709 19253 consensus_queue.cc:260] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:48.593390 15641 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:48.591113 15567 leader_election.cc:336] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.096s (SENT)
I20260709 13:19:48.597456 15567 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:48.595003 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 138970us
I20260709 13:19:48.562332 18955 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:48.540972 19168 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:19:48.598503 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:48.603545 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.503535 19039 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:48.588406 15426 catalog_manager.cc:5697] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 6 to 7, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 7 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:48.565060 19279 raft_consensus.cc:697] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:48.436982 19038 consensus_queue.cc:260] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:48.617729 15567 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:48.518451 19033 consensus_queue.cc:237] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:48.598965 19296 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:48.608275 19275 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:48.624100 19328 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:48.622606 16240 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
W20260709 13:19:48.619298 19050 consensus_queue.cc:1175] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3ef186ec12554a1d87814af0a0ace9a4" responder_term: 8 status { last_received { term: 6 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: [7.8-7.8]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:19:48.618738 19038 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:48.620100 19253 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:48.616758 19043 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:48.621872 19326 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:48.605072 19248 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:48.643633 15567 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
W20260709 13:19:48.625723 19320 log.cc:927] Time spent T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:19:48.657627 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 51974.3us
I20260709 13:19:48.619432 19281 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:48.622876 19251 raft_consensus.cc:3055] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:48.674424 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:48.676544 15567 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:48.682042 15567 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:48.642367 18784 raft_consensus.cc:697] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:48.647949 19128 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:48.651934 19275 raft_consensus.cc:697] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:48.651114 19214 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:48.694082 15522 raft_consensus.cc:2410] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:19:48.677891 19251 raft_consensus.cc:740] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:48.705425 16233 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:19:48.710125 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:48.696421 19326 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:48.693522 19062 raft_consensus.cc:697] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:48.778980 16231 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:48.704457 19105 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:19:48.785472 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 183997us
I20260709 13:19:48.708199 19078 consensus_queue.cc:1059] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 13:19:48.754405 19250 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:48.794827 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 119146us
I20260709 13:19:48.798224 15642 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:48.801373 15642 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:48.687711 19279 consensus_queue.cc:237] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:48.804385 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 205955us
I20260709 13:19:48.807998 15493 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:48.710336 19285 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:48.784919 19326 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:48.805641 15426 catalog_manager.cc:5697] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 5 to 8, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 8 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:48.696158 19332 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:48.812114 15527 raft_consensus.cc:1240] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:48.814260 15493 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:48.807009 19296 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:48.803532 19248 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:19:48.816532 19282 consensus_meta.cc:287] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.550s	user 0.010s	sys 0.000s
W20260709 13:19:48.826258 19046 consensus_meta.cc:287] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.534s	user 0.009s	sys 0.001s
I20260709 13:19:48.815739 19326 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:48.828006 19039 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:48.850061 19105 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:48.852903 19237 consensus_meta.cc:287] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.558s	user 0.007s	sys 0.001s
W20260709 13:19:48.849382 19148 consensus_meta.cc:287] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.608s	user 0.004s	sys 0.004s
I20260709 13:19:48.859337 16234 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:19:48.815606 19337 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:48.837836 19081 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:48.854074 19296 raft_consensus.cc:697] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:48.811637 19275 consensus_queue.cc:237] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:48.809731 19251 consensus_queue.cc:260] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:48.862871 19250 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.922528 15594 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:48.870760 15674 raft_consensus.cc:606] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:48.826220 19062 consensus_queue.cc:237] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:48.900480 19334 log.cc:927] Time spent T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:19:48.936460 15674 raft_consensus.cc:614] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:48.816923 18784 consensus_queue.cc:237] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:48.938642 19338 log.cc:927] Time spent T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:19:48.943348 19236 raft_consensus.cc:3055] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:19:48.944525 19090 consensus_meta.cc:287] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.573s	user 0.009s	sys 0.001s
W20260709 13:19:48.948596 19276 consensus_meta.cc:287] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.613s	user 0.006s	sys 0.002s
I20260709 13:19:48.949036 19148 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:48.950650 16236 raft_consensus.cc:2410] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:19:48.957753 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.961383 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:48.962857 19305 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:48.966148 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.970304 19235 raft_consensus.cc:3055] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:19:48.973322 16233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57258 (request call id 810) took 1363 ms (client timeout 1707 ms). Trace:
I20260709 13:19:48.978446 15677 raft_consensus.cc:1240] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:48.996417 19251 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:48.935859 19250 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:48.871102 19105 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.010411 19296 consensus_queue.cc:237] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:48.934652 18955 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.022724 19301 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:49.026181 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:49.029933 15600 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:19:49.033380 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:49.034375 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:49.043577 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.187376 19337 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:49.151295 19168 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:49.188272 19337 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:49.188658 16233 rpcz_store.cc:269] 0709 13:19:47.610100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.610220 (+   120us) service_pool.cc:224] Handling call
0709 13:19:48.973136 (+1362916us) spinlock_profiling.cc:243] Waited 980 ms on lock 0x7b54007201d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:48.973256 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2254679808}
I20260709 13:19:49.187592 19235 raft_consensus.cc:740] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:49.051723 19330 raft_consensus.cc:3055] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:19:49.191243 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.193826 15523 pending_rounds.cc:85] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 8
I20260709 13:19:49.194149 15523 pending_rounds.cc:107] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:19:49.063659 15673 raft_consensus.cc:1275] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:19:49.195024 15673 pending_rounds.cc:85] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4: Aborting all ops after (but not including) 8
I20260709 13:19:49.194409 15523 raft_consensus.cc:2889] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:49.195258 15673 pending_rounds.cc:107] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:19:49.195964 15673 raft_consensus.cc:2889] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:49.188417 19168 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.058760 15426 catalog_manager.cc:5697] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 6 to 8, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 8 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:19:49.243116 15426 catalog_manager.cc:5697] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 5 to 8, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 8 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:49.244903 15426 catalog_manager.cc:5697] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 4 to 6, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 6 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:19:49.201586 15567 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.000s	sys 0.014s
W20260709 13:19:49.246606 15567 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.000s	sys 0.014s
W20260709 13:19:49.247237 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 565279us
W20260709 13:19:49.148034 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 333827us
I20260709 13:19:49.249439 15426 catalog_manager.cc:5697] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 4 to 6, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 6 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:19:49.254326 16238 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.019s	sys 0.051s
W20260709 13:19:49.254669 16238 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.019s	sys 0.051s
I20260709 13:19:49.251037 15426 catalog_manager.cc:5697] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:49.257766 15426 catalog_manager.cc:5697] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:19:49.259835 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 298550us
I20260709 13:19:49.265515 15669 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:49.266503 15493 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:49.276587 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:49.278554 15674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:36980 (request call id 18) took 1662 ms. Trace:
I20260709 13:19:49.281805 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:49.283020 15642 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:49.063575 19147 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:49.177798 19285 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:49.341163 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 58225us
I20260709 13:19:49.190788 19330 raft_consensus.cc:740] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:49.277298 15668 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:49.197461 19151 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.315661 15602 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:49.175218 19081 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:49.253693 19298 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:49.357877 15642 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:49.136588 19314 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:49.278998 15674 rpcz_store.cc:276] 0709 13:19:47.591697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.593202 (+  1505us) service_pool.cc:224] Handling call
0709 13:19:49.254305 (+1661103us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b540021d754. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a05853 00007f5ce99d360a 00007f5ce2f597d4 00007f5ce2f59762 00007f5ce2f59691 00007f5ce2f5960c 00007f5ce2f588c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:49.254429 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2786299648}
I20260709 13:19:49.191835 19236 raft_consensus.cc:740] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:49.190615 19337 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:49.337656 19274 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:49.312355 15426 catalog_manager.cc:5697] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:49.239301 19253 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:49.353554 15524 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:19:49.285274 19248 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:19:49.338510 19255 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.007s	sys 0.036s
W20260709 13:19:49.335774 19295 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.009s	sys 0.014s
I20260709 13:19:49.276091 19282 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.261793 19305 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:49.251996 19301 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.282359 19315 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:49.049778 19326 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:49.336290 19339 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:49.253075 19033 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:49.339993 19117 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.316728 19078 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 13:19:49.354521 19090 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:49.314054 18676 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:19:49.314265 19209 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:19:49.258721 19148 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.359324 19151 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:49.406013 19274 raft_consensus.cc:697] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:49.385635 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:49.285017 19276 consensus_queue.cc:1059] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 13:19:49.410102 19305 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.365386 19347 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:49.407421 19295 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.009s	sys 0.016s
I20260709 13:19:49.348564 19285 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.587419 19326 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:49.372452 19314 raft_consensus.cc:697] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:49.603649 19314 consensus_queue.cc:237] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:49.408210 19255 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.885s	user 0.008s	sys 0.037s
I20260709 13:19:49.606201 19255 raft_consensus.cc:697] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:49.607095 19255 consensus_queue.cc:237] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:49.347661 19235 consensus_queue.cc:260] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:49.608325 19235 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:49.432811 19210 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:49.610229 19210 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.612368 19210 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:49.346381 19327 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.076s
W20260709 13:19:49.506335 19349 log.cc:927] Time spent T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:19:49.615056 19344 consensus_queue.cc:1175] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3ef186ec12554a1d87814af0a0ace9a4" responder_term: 5 status { last_received { term: 4 index: 3 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 4. Current term is 5. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:19:49.507359 19305 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:49.405445 19253 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.410425 19282 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.364701 19298 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.621008 19298 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:49.414156 19315 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.637248 19347 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.671653 15426 catalog_manager.cc:5697] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:49.675793 15426 catalog_manager.cc:5697] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 5 to 8, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 8 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:49.678377 15426 catalog_manager.cc:5697] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 5 to 6, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 6 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:19:49.681953 15420 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49144 (request call id 350) took 28900 ms (client timeout 15000 ms). Trace:
W20260709 13:19:49.690483 15597 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.024s	sys 0.016s
I20260709 13:19:49.690934 15596 raft_consensus.cc:1240] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: []
W20260709 13:19:49.691151 15597 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.024s	sys 0.016s
W20260709 13:19:49.691601 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.405993 15669 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:19:49.545601 19062 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.020s	sys 0.045s
W20260709 13:19:49.716774 19062 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.021s	sys 0.046s
I20260709 13:19:49.512183 18955 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:49.413131 19033 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:49.718686 16235 raft_consensus.cc:1240] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 10. Current term is 11. Ops: []
W20260709 13:19:49.718508 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 358751us
W20260709 13:19:49.682649 15420 rpcz_store.cc:269] 0709 13:19:20.775060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:20.788843 (+ 13783us) service_pool.cc:224] Handling call
0709 13:19:49.666889 (+28878046us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:47.037181 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:47.334559 (+297378us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:47.334599 (+    40us) write_op.cc:435] Acquired schema lock
0709 13:19:47.334617 (+    18us) tablet.cc:662] Decoding operations
0709 13:19:47.335164 (+   547us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:47.339178 (+  4014us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:47.355025 (+ 15847us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:19:47.355417 (+   392us) tablet.cc:701] Row locks acquired
0709 13:19:47.355445 (+    28us) write_op.cc:260] PREPARE: finished
0709 13:19:47.355591 (+   146us) write_op.cc:270] Start()
0709 13:19:47.355727 (+   136us) write_op.cc:276] Timestamp: P: 1783603187355544 usec, L: 0
0709 13:19:47.355749 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:19:47.432385 (+ 76636us) log.cc:844] Serialized 1298 byte log entry
0709 13:19:47.822040 (+389655us) op_driver.cc:464] REPLICATION: finished
0709 13:19:48.009979 (+187939us) write_op.cc:301] APPLY: starting
0709 13:19:48.010079 (+   100us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:48.010218 (+   139us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:48.010355 (+   137us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:48.010817 (+   462us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:48.011049 (+   232us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:19:48.011096 (+    47us) write_op.cc:312] APPLY: finished
0709 13:19:48.011314 (+   218us) log.cc:844] Serialized 35 byte log entry
0709 13:19:48.011484 (+   170us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:48.011581 (+    97us) write_op.cc:454] Released schema lock
0709 13:19:48.011619 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:48.012074 (+   455us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:48.012104 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":170977,"apply.run_cpu_time_us":2116,"apply.run_wall_time_us":13370,"num_ops":3,"prepare.queue_time_us":535186,"prepare.run_cpu_time_us":22427,"prepare.run_wall_time_us":659642,"raft.queue_time_us":94566,"raft.run_cpu_time_us":49004,"raft.run_wall_time_us":125170,"replication_time_us":456356,"spinlock_wait_cycles":2816,"thread_start_us":707115,"threads_started":4,"wal-append.queue_time_us":248444,"wal-append.run_cpu_time_us":7164,"wal-append.run_wall_time_us":1330114}]]}
I20260709 13:19:49.373741 19081 consensus_queue.cc:1059] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:19:49.562235 19105 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:49.431375 19117 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.777120 19117 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:49.504669 19165 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:49.569898 19168 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:49.500379 19236 consensus_queue.cc:260] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:49.782146 19304 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.783653 19236 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:49.783309 19304 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.582805 19248 raft_consensus.cc:697] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:49.786137 19327 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.787361 19327 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.787662 19248 consensus_queue.cc:237] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.576350 19274 consensus_queue.cc:237] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.783529 19347 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:49.592573 19301 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:49.588940 19350 log.cc:927] Time spent T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 13:19:49.809207 15524 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1955) took 1430 ms (client timeout 1565 ms). Trace:
W20260709 13:19:49.809762 15524 rpcz_store.cc:269] 0709 13:19:48.379050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.379897 (+   847us) service_pool.cc:224] Handling call
0709 13:19:49.808733 (+1428836us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b54002de1d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:49.808940 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1922999040}
W20260709 13:19:49.811733 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.812198 19372 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:49.813246 19372 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.816478 19372 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:49.817180 19372 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.818488 19305 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.819439 19305 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.823123 19367 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:49.823725 15641 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.074s (SENT) [suppressed 28 similar messages]
I20260709 13:19:49.770468 19033 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:49.772594 19332 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:49.783890 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.428990 19330 consensus_queue.cc:260] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:49.823997 19367 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:49.799142 19351 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:19:49.826056 15493 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.000s	sys 0.016s
I20260709 13:19:49.680908 15426 catalog_manager.cc:5697] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:49.834177 19327 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:49.836481 19332 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:49.839200 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.847944 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:49.837149 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:49.849085 16233 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:49.849455 19358 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.850463 19358 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.849349 16238 raft_consensus.cc:2393] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 7: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:49.855477 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:49.839447 19165 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.856210 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:49.857211 16231 raft_consensus.cc:2393] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 8: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:49.857070 19165 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:49.856334 16240 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:19:49.858618 16235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53976 (request call id 840) took 1420 ms. Trace:
I20260709 13:19:49.859006 19368 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:49.862576 19368 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.865245 19368 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:49.859038 16235 rpcz_store.cc:276] 0709 13:19:48.437932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.439530 (+  1598us) service_pool.cc:224] Handling call
0709 13:19:48.498055 (+ 58525us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:49.719411 (+1221356us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:49.858332 (+138921us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54001cfcd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:49.858528 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2805649152}
I20260709 13:19:49.867368 19339 raft_consensus.cc:3055] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:49.853796 15426 catalog_manager.cc:5697] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 2 to 5, term changed from 2 to 5, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) changed from NON_VOTER to VOTER. New cstate: current_term: 5 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:19:49.869269 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 601413us
I20260709 13:19:49.841672 19367 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:49.858628 19332 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:49.881597 19298 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:49.882478 19298 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.883312 19148 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:49.885079 19298 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:49.885396 19282 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:49.890242 15426 catalog_manager.cc:5697] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 6 to 7, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 7 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:49.891006 19090 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:49.869474 19339 raft_consensus.cc:740] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:49.856611 19351 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:49.878903 19372 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:49.910406 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:49.910012 15426 catalog_manager.cc:5697] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:49.911316 19165 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:49.900053 19090 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:49.912892 15568 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:49.901620 19305 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:49.913983 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.917820 19372 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:49.911788 15426 catalog_manager.cc:5697] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:19:49.923642 15426 catalog_manager.cc:5697] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 5 to 6, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 6 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:49.925130 19295 raft_consensus.cc:3055] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:49.925659 19295 raft_consensus.cc:740] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:49.926828 19295 consensus_queue.cc:260] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.928105 19295 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:49.911260 15594 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:49.938448 15568 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:49.939045 18854 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:49.939525 18854 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:49.939832 18854 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:49.940387 19237 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:49.940594 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:49.940917 19237 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:49.941334 19237 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:49.941427 16233 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:49.943420 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:49.944015 19151 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:49.944262 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:49.944881 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:49.945112 15601 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:49.944139 15600 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:49.947467 19250 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:49.950116 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:49.950881 19330 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:49.951658 19237 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.952109 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:49.952808 16238 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:19:49.907505 19351 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:49.950942 16233 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
W20260709 13:19:49.958329 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 491176us
I20260709 13:19:49.953979 19237 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:49.975798 15594 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:49.982652 19253 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:49.986719 19359 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.987479 19359 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:49.989578 15567 leader_election.cc:336] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.943s (SENT)
I20260709 13:19:49.990079 15567 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:49.993374 19359 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:49.994122 15493 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:49.995215 19351 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.995992 19351 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:49.996891 15568 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:49.997284 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:49.997390 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:49.997896 16240 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:49.925400 15426 catalog_manager.cc:5697] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 5 to 7, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 7 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:49.998117 16238 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:19:49.998657 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:50.009140 15568 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:50.004208 19351 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:50.048256 19368 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:50.048629 15568 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:50.049260 19368 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.050192 18784 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:50.050628 18784 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:50.051023 18784 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:50.051102 19105 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:50.061913 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:50.080166 15594 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:50.093178 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:50.006728 19359 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.051616 19105 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:50.129170 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 139763us
W20260709 13:19:50.132513 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 308990us
I20260709 13:19:50.127737 19105 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:50.136299 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:50.132879 15426 catalog_manager.cc:5697] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 4 to 5, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 5 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:50.012528 19339 consensus_queue.cc:260] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.148697 15524 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:19:50.154419 19358 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:50.156405 19235 consensus_meta.cc:287] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.548s	user 0.013s	sys 0.003s
I20260709 13:19:50.156738 19344 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:50.157106 19251 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:19:50.162427 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 196334us
I20260709 13:19:50.163440 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:50.154508 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:50.146200 15567 leader_election.cc:336] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.957s (SENT)
I20260709 13:19:50.171589 16237 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:19:50.154508 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:50.178154 15567 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:50.178818 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:50.181074 15600 raft_consensus.cc:1275] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:19:50.170370 19344 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:50.191206 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.191663 15668 raft_consensus.cc:1240] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:50.183900 19339 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:50.193746 15525 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 7: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:50.141885 19359 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:50.276823 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.374876 15524 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
W20260709 13:19:50.277096 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 88082.3us
W20260709 13:19:50.276965 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:50.267849 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 67569.5us
I20260709 13:19:50.170853 19151 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:50.421857 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 157669us
W20260709 13:19:50.423066 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 282059us
I20260709 13:19:50.388427 16240 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:19:50.180819 19358 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.301430 16203 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:50.299247 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:50.353797 15602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34102 (request call id 911) took 1327 ms. Trace:
I20260709 13:19:50.220325 16231 raft_consensus.cc:1275] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:50.170814 19253 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:50.425760 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.184901 19344 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:50.199115 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:50.245319 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.219455 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:50.304626 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.173545 19148 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.219419 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.188889 19337 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:50.448064 16236 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:50.443022 15602 rpcz_store.cc:276] 0709 13:19:49.025771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.025890 (+   119us) service_pool.cc:224] Handling call
0709 13:19:50.353433 (+1327543us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b540025ba54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:50.353618 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":286075648}
I20260709 13:19:50.219143 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:50.455128 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 153824us
W20260709 13:19:50.455385 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.267560 15426 catalog_manager.cc:5697] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 3 to 6, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 6 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:50.200414 19282 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:50.456053 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.199293 19371 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.462029 16234 raft_consensus.cc:2393] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 6: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:50.352339 19363 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:50.467257 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.304098 19078 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:50.435845 19117 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:50.461414 15677 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:19:50.474833 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.476197 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:50.476938 15527 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:50.478309 19210 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:50.474824 19128 raft_consensus.cc:2764] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:19:50.474965 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:50.480655 15599 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:50.449227 19062 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.481796 19148 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:50.482841 19236 consensus_meta.cc:287] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.698s	user 0.010s	sys 0.005s
I20260709 13:19:50.482609 19062 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.483031 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:50.483700 15597 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:50.281375 19090 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.484995 15670 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:50.487738 19090 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:50.489102 15521 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:19:50.379951 18854 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.476850 19304 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:50.216427 19279 consensus_queue.cc:1059] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.300s
I20260709 13:19:50.490263 15598 raft_consensus.cc:2393] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 10: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:50.490818 19279 consensus_queue.cc:1243] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer dd1a14c3173d49dc9b55bb116e3783c9 log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:19:50.493943 19359 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:50.494625 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:50.494797 19368 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:50.494694 19358 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.497778 19358 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:50.494621 19282 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:50.500411 19363 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.501129 19253 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.502720 19363 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:50.464967 16205 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:50.500697 19315 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:50.474221 15567 leader_election.cc:336] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.058s (SENT)
I20260709 13:19:50.510847 15568 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:50.513581 15567 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:50.443497 19274 consensus_queue.cc:1059] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.332s
I20260709 13:19:50.474869 15426 catalog_manager.cc:5697] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 5 to 9, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 9 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:50.485034 19062 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:50.495642 19285 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:50.523335 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 96713.1us
I20260709 13:19:50.515796 19236 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:50.517843 19117 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.526932 19237 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:50.519752 19081 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.528148 15426 catalog_manager.cc:5697] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 2 to 6, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 6 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:19:50.532338 18854 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:50.532212 19081 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:50.471684 19151 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.529002 15493 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:50.533145 15426 catalog_manager.cc:5697] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 4 to 5, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 5 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:50.536213 19236 raft_consensus.cc:697] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:50.544176 16205 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:50.514556 19371 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.545256 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.520155 19105 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.544400 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:19:50.546797 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 76755.9us
I20260709 13:19:50.537681 19253 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:50.548135 16236 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:50.539923 19117 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:50.544952 19062 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:50.543785 19128 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.556205 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.548612 19351 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:50.556900 15670 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:50.541982 19359 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:50.552156 19377 log.cc:927] Time spent T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
W20260709 13:19:50.555694 18784 consensus_meta.cc:287] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.504s	user 0.014s	sys 0.006s
I20260709 13:19:50.562332 19081 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:50.551069 19344 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.557801 19210 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:50.564318 19210 raft_consensus.cc:697] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:50.564424 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:50.564566 19344 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:50.564996 15594 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:50.566156 19128 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.562522 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.568490 19128 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:50.568930 15670 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:19:50.571072 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:50.571677 16238 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:50.568742 19210 consensus_queue.cc:237] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:50.577696 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.580039 16238 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:19:50.581285 15568 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:50.553702 19282 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:50.586093 19351 raft_consensus.cc:697] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:50.586642 18854 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:50.587185 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:50.555250 19062 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
W20260709 13:19:50.587406 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 79209.2us
W20260709 13:19:50.592232 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.594364 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.594816 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:50.596376 15670 raft_consensus.cc:2393] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 9: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:50.595427 15602 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:50.571650 19371 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:50.585958 18784 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.593117 19062 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:50.602341 15641 leader_election.cc:336] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.565s (SENT)
I20260709 13:19:50.594530 18854 raft_consensus.cc:697] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:50.611063 15641 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:50.596764 19081 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:50.613487 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.593825 19359 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:50.616740 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.617908 18854 consensus_queue.cc:237] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.619022 15527 raft_consensus.cc:1240] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:50.620544 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:50.621354 15527 raft_consensus.cc:2393] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 9: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:50.621614 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:50.622236 15669 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:50.606441 19344 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.627316 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:50.627501 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:50.624713 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:50.634186 15670 raft_consensus.cc:2393] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 8: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:50.642338 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:50.643146 15524 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:50.635468 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.593422 19236 consensus_queue.cc:237] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.619218 19148 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:50.627563 19371 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:50.663786 15521 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:50.664256 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:50.652807 19117 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.665053 19105 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:50.673074 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 112297us
I20260709 13:19:50.671715 15519 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:50.667344 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:50.672097 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.676036 15670 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 7: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:50.675421 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:50.677392 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 75855.6us
I20260709 13:19:50.666428 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:50.677831 15668 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:19:50.680186 15672 raft_consensus.cc:1275] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:19:50.682966 15527 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:50.688205 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:50.688865 15677 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:50.690019 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:50.690652 15522 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:50.692454 15521 raft_consensus.cc:1240] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:50.650436 19344 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:50.675066 19151 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:50.652418 19081 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.694192 19351 consensus_queue.cc:237] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.682751 19117 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.697487 19304 raft_consensus.cc:3055] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:50.709396 19304 raft_consensus.cc:740] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:50.703233 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:50.710410 19304 consensus_queue.cc:260] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.711779 19304 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:50.712124 19081 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:50.715672 19359 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:50.717911 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:50.719161 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:50.719794 15527 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:50.720567 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:50.721114 15527 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:50.722290 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:50.723153 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.723825 15672 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:19:50.735761 19117 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:50.710357 15525 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:50.708310 15669 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:19:50.745486 18784 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:50.747582 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:50.795801 15594 raft_consensus.cc:1275] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:19:50.750013 15676 raft_consensus.cc:1275] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:19:50.798219 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.798731 15594 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:50.798892 15676 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:19:50.799791 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:50.801090 15410 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 582343936346420b87ecf2eebd00dc25 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3ef186ec12554a1d87814af0a0ace9a4 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:50.750703 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.802855 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:50.803571 15673 raft_consensus.cc:2393] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 8: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:50.752040 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.804412 15671 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:50.805294 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.806450 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.807061 15671 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:19:50.807885 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.808408 15671 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:19:50.809110 15671 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:19:50.753753 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:50.810125 15598 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:50.811007 15676 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:50.813479 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:50.814145 15523 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:50.815099 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:50.755599 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.756774 15675 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:50.759042 15566 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:19:50.826709 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 67739.4us
I20260709 13:19:50.762044 19315 raft_consensus.cc:3055] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:50.828406 19315 raft_consensus.cc:740] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:50.829583 15598 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:19:50.829357 19315 consensus_queue.cc:260] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:50.830622 19315 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:50.762394 15641 leader_election.cc:336] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.847s (SENT)
I20260709 13:19:50.831818 15641 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:50.832140 16234 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:19:50.833173 15566 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:50.771029 16236 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:19:50.772467 19344 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:50.773188 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.841060 15672 raft_consensus.cc:2393] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 9: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:50.776124 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:50.777652 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:50.779232 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:50.783154 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:50.784226 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:50.842324 15519 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:50.790970 19368 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:50.754774 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:50.854784 15600 raft_consensus.cc:1813] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Handling vote request from an unknown peer 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:50.817682 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:50.855021 15677 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:19:50.831305 15522 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:50.833415 15568 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:50.766433 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 61134.8us
I20260709 13:19:50.839880 15594 raft_consensus.cc:1275] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:50.856950 16204 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:50.809746 15671 raft_consensus.cc:606] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:50.858140 15671 raft_consensus.cc:614] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:50.859243 16231 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:19:50.859962 15600 raft_consensus.cc:2393] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 6: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:50.844956 15525 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
W20260709 13:19:50.846030 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.865567 15595 raft_consensus.cc:1240] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:50.866254 19050 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:50.868213 19117 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:50.868677 19117 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:50.869017 19117 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:50.872207 19368 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.874579 19368 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:50.878032 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 44698.1us
I20260709 13:19:50.880414 15568 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:50.881654 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 48525.5us
I20260709 13:19:50.882611 15566 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:50.892422 15568 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:50.892674 16204 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:50.898233 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:50.898962 15594 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
W20260709 13:19:50.908293 15567 leader_election.cc:336] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.086s (SENT)
I20260709 13:19:50.909373 15668 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 11. (term mismatch)
I20260709 13:19:50.909808 15668 pending_rounds.cc:85] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting all ops after (but not including) 10
I20260709 13:19:50.911391 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:50.912724 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:50.917455 15566 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:50.859516 19330 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:50.920765 16204 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:50.910312 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:50.884670 19344 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.891676 19237 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:50.891903 19105 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:50.910058 15668 pending_rounds.cc:107] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting uncommitted NO_OP operation due to leader change: 10.11
I20260709 13:19:50.932736 15668 raft_consensus.cc:2889] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:50.925256 15601 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:19:50.929001 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.891901 18784 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:50.933967 15669 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:50.908023 19235 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:50.899322 19368 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:50.910467 19151 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:50.935348 19344 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:50.934698 19128 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:50.946216 19128 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:50.946601 19128 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:50.950053 19280 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:50.934651 19237 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:50.942062 18676 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.942209 15520 raft_consensus.cc:1275] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:19:50.949968 16203 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:50.951738 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.952961 16203 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:50.946239 18784 raft_consensus.cc:697] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:50.954824 18784 consensus_queue.cc:237] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.955438 16203 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:50.955760 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:50.956019 19210 raft_consensus.cc:2764] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:19:50.956517 15672 raft_consensus.cc:2393] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 9: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:50.956734 19210 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:50.942654 19105 raft_consensus.cc:697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:50.942883 19235 raft_consensus.cc:697] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:50.958613 19105 consensus_queue.cc:237] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.959381 19235 consensus_queue.cc:237] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.951125 18676 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.931692 19078 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:50.945623 19388 consensus_queue.cc:1048] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:19:50.931532 19251 consensus_queue.cc:1048] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:19:50.963668 19078 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.964195 19078 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:50.950203 16231 raft_consensus.cc:2393] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 8: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:50.950959 19237 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:50.950709 15642 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:50.950577 19280 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:50.969236 19280 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:50.972200 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 209860us
I20260709 13:19:50.972867 19368 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:50.973461 19368 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:50.929179 16204 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:50.941568 15523 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:50.958323 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.949275 19062 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:50.980194 15675 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
W20260709 13:19:50.986025 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 93933.6us
I20260709 13:19:50.984134 15642 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:50.985622 19078 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:50.945770 19344 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:50.995683 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 66592us
W20260709 13:19:50.995309 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.996490 16203 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:51.005128 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.010457 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:51.011183 15672 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:51.013422 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:50.994652 19282 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:51.015893 15566 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:51.016347 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:51.020951 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:51.022037 19260 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:51.022480 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:51.023193 15599 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
W20260709 13:19:51.025000 15641 leader_election.cc:336] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.107s (SENT)
W20260709 13:19:51.026108 15567 leader_election.cc:336] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.119s (SENT)
I20260709 13:19:51.019991 15566 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:51.017894 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:51.029052 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.027707 15596 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:19:51.031450 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:51.032301 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:51.003046 19330 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.037209 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:51.034626 15567 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:51.032965 15601 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:19:51.037288 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:51.034046 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.036562 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:51.041314 18955 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:51.014894 19090 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:19:51.043292 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.026928 19282 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:51.045058 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 48637.3us
I20260709 13:19:51.028995 18676 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:51.033470 19251 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:51.022480 19260 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:19:51.064390 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.049528 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:51.055197 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.063874 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:51.065974 15523 raft_consensus.cc:2376] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7. Re-sending same reply.
I20260709 13:19:51.044386 19090 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:51.067104 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.028673 19371 consensus_queue.cc:1048] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:19:51.047785 19274 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:51.070626 15642 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:51.044237 19235 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:51.043037 19330 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:51.042266 19304 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:19:51.061461 19251 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.067229 19260 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:51.078886 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:51.069566 19050 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:51.079859 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:51.136929 15522 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:51.130005 15519 raft_consensus.cc:2376] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9. Re-sending same reply.
I20260709 13:19:51.138000 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:51.081041 15527 raft_consensus.cc:1275] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 4. (term mismatch)
I20260709 13:19:51.082892 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:51.083535 15525 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:19:51.082158 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:51.083869 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:51.048504 19282 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:51.070060 19090 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:51.094002 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:19:51.159775 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 91506.7us
I20260709 13:19:51.061400 19285 consensus_queue.cc:1059] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 13:19:51.038549 19388 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:19:51.098428 15526 raft_consensus.cc:1240] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:51.078033 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:51.124784 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:51.126219 19330 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:51.127218 15567 leader_election.cc:336] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.818s (SENT)
I20260709 13:19:51.078716 19398 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:19:51.127794 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 59580us
I20260709 13:19:51.105973 19274 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.136608 19050 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.165458 15527 pending_rounds.cc:85] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 3
I20260709 13:19:51.129832 19251 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:51.157801 19337 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:51.165192 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.162822 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:51.138576 15522 raft_consensus.cc:2376] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5. Re-sending same reply.
I20260709 13:19:51.175062 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:51.175781 15522 raft_consensus.cc:2393] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 10: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:51.168409 15521 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:51.166399 15524 raft_consensus.cc:2393] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 6: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:51.169646 15527 pending_rounds.cc:107] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 13:19:51.169569 15567 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:51.177359 15527 raft_consensus.cc:2889] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:51.170672 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:51.178459 15528 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:19:51.126207 19128 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:51.184015 19210 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:51.183895 19304 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:51.164790 19398 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.212914 19398 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:51.161886 19280 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:51.220014 19280 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:51.169175 19274 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:51.167037 19237 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:51.190205 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:19:51.212407 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 215151us
I20260709 13:19:51.221088 15526 raft_consensus.cc:2376] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7. Re-sending same reply.
I20260709 13:19:51.220152 19090 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:51.191141 19330 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:51.199070 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:51.173213 19050 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:51.170540 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:51.142553 19151 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:51.206636 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:51.229992 15523 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:19:51.230798 19151 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:51.232312 19151 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:51.234778 19117 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:51.236986 19117 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:51.231081 15522 raft_consensus.cc:1813] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Handling vote request from an unknown peer 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:51.238618 15522 raft_consensus.cc:2393] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 6: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:51.088263 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:51.240203 19351 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:19:51.241475 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 153277us
I20260709 13:19:51.094516 19368 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:51.212124 15641 leader_election.cc:336] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.067s (SENT)
I20260709 13:19:51.222740 19237 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:51.190601 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 78182us
I20260709 13:19:51.225346 19128 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:51.226992 15521 raft_consensus.cc:1275] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:19:51.228335 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:51.252893 15526 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:19:51.169238 19285 consensus_queue.cc:1243] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer 5498ef3e89a941e39c25f68cf64e6b0f log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:19:51.245906 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:51.254127 15525 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
W20260709 13:19:51.255085 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 213206us
W20260709 13:19:51.258908 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.258203 15493 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:51.182307 19062 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:51.271040 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:51.271744 15672 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:51.274137 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:51.274318 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:51.274789 16233 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:51.279618 19117 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.280287 19117 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:51.280879 19351 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:51.276731 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 111542us
I20260709 13:19:51.181767 19337 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:51.284951 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:51.267640 19235 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.294999 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:51.298291 16236 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:19:51.282493 19117 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:51.308326 15642 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:51.312386 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 146712us
W20260709 13:19:51.315212 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 57111.1us
I20260709 13:19:51.282526 19090 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:51.317621 15493 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:51.320148 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:51.321285 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:51.316668 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:51.326684 16237 raft_consensus.cc:2368] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8.
I20260709 13:19:51.326869 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:51.325477 15493 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:51.327689 16235 raft_consensus.cc:2393] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 10: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:51.328353 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:51.328986 15521 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:51.337057 16203 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:51.338267 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:51.338929 15526 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:51.339128 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:51.339779 16232 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:51.340605 19410 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:51.341579 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:19:51.343202 19368 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:51.344281 19062 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:51.346551 15526 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
W20260709 13:19:51.348227 19396 log.cc:927] Time spent T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.327s	user 0.000s	sys 0.003s
I20260709 13:19:51.352094 16234 raft_consensus.cc:1275] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:19:51.355755 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 157935us
I20260709 13:19:51.359217 16233 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:19:51.371481 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:51.372313 16233 raft_consensus.cc:2393] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 7: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:51.344506 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:51.369606 16203 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:51.374562 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:51.377285 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.378612 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:51.379216 15677 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:51.383003 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:51.403900 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:51.404668 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:51.430943 15669 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:51.431332 15676 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:51.384625 19062 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:19:51.402645 19415 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:51.450203 19415 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.450594 19415 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:51.423081 19151 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:51.425477 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:51.426244 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:51.426548 16235 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:19:51.427367 16236 raft_consensus.cc:1240] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8. Ops: []
W20260709 13:19:51.458271 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.422960 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:51.430997 15671 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:19:51.458817 15597 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:51.436060 15677 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:19:51.376621 19315 raft_consensus.cc:3055] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:51.307433 19235 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:51.460999 15669 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:51.465381 15676 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:51.438558 16204 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:19:51.467239 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 119995us
I20260709 13:19:51.468572 15672 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:19:51.469769 15521 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:51.471602 15642 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:51.461587 16233 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:19:51.474368 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:51.468562 15491 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:51.478013 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:51.476285 15568 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:51.487701 15567 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:51.470377 19295 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:51.464223 19315 raft_consensus.cc:740] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:51.464370 19413 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:19:51.493081 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.494202 15642 leader_election.cc:400] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8.
I20260709 13:19:51.478439 19285 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:51.497937 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.499797 19105 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:51.500483 19148 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:51.500893 19418 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:51.475878 18676 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:51.503242 15642 leader_election.cc:403] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:19:51.515089 18676 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.513392 15597 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:51.515509 18676 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:51.515920 15642 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:51.515425 15671 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:51.483379 19388 raft_consensus.cc:2764] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:19:51.485252 19090 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:51.474182 19351 consensus_queue.cc:1048] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:19:51.504165 19295 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:51.519028 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 49451.2us
W20260709 13:19:51.523864 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 198478us
I20260709 13:19:51.520169 19371 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:51.555112 19371 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.555490 19371 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:51.557600 15493 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:51.558749 15493 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:51.517640 19418 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.565385 19418 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:51.570525 19235 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:51.571967 19237 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:51.573889 19237 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:51.574462 19237 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:51.518689 19410 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8.
I20260709 13:19:51.578339 19408 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:51.560781 19315 consensus_queue.cc:260] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:51.580231 19274 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:51.581420 19315 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:51.503925 19251 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:51.582648 15603 raft_consensus.cc:2393] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 8: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
W20260709 13:19:51.587857 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 72011.7us
I20260709 13:19:51.588976 15642 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:51.554183 19295 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:51.504479 19415 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:51.577597 18676 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:51.519615 19413 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.092s
W20260709 13:19:51.622247 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 110527us
I20260709 13:19:51.624670 16232 raft_consensus.cc:2393] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 10: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:51.578507 19260 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:51.613308 19410 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:51.631469 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:51.617314 19251 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:51.653203 16231 raft_consensus.cc:1240] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:51.654170 15642 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:51.614636 19398 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:51.657236 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:51.635015 19274 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:51.632386 19416 raft_consensus.cc:2764] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:19:51.661243 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:51.663345 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:51.668109 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 44359.1us
I20260709 13:19:51.680063 16237 raft_consensus.cc:1240] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:51.662643 19410 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:51.657944 19426 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:51.649479 19050 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:51.655185 19090 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:19:51.681704 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:51.796638 16238 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:51.793778 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:51.805243 16240 raft_consensus.cc:2393] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 7: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:51.814750 19410 raft_consensus.cc:697] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:51.667165 19151 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:19:51.800956 19431 log.cc:927] Time spent T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:19:51.683712 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:51.824026 16236 raft_consensus.cc:2368] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10.
I20260709 13:19:51.825021 19260 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:19:51.827852 19423 log.cc:927] Time spent T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 13:19:51.803259 18676 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:51.834196 19432 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:51.834947 19432 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:51.802069 19425 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:51.834280 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:51.868057 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 173197us
W20260709 13:19:51.837867 19424 log.cc:927] Time spent T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
W20260709 13:19:52.035180 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:52.006846 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 66274.5us
I20260709 13:19:52.023253 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:52.040544 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:52.040453 15519 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:52.046985 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:52.047300 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:52.047678 15522 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:19:52.047814 15527 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:51.820045 19413 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.848057 19425 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:51.848465 19415 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:52.058353 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:52.059005 15519 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:51.845799 19398 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:52.053223 15526 raft_consensus.cc:1275] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:19:52.057426 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 498759us
I20260709 13:19:52.060730 15493 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:52.063695 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:51.844762 19371 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:51.840992 19418 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:52.060258 19432 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:52.023795 19237 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:52.077103 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 275289us
I20260709 13:19:52.059315 19425 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:52.079643 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 265578us
I20260709 13:19:52.085795 15522 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:19:52.074694 19416 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:52.092796 15672 raft_consensus.cc:1240] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:19:52.093667 15602 raft_consensus.cc:1275] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:19:52.078716 19428 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:52.098428 16204 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:52.098901 19237 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:52.099924 19050 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:52.100968 19336 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:52.101485 19336 raft_consensus.cc:697] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:19:52.102062 19282 consensus_meta.cc:287] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.899s	user 0.012s	sys 0.000s
I20260709 13:19:52.102527 19282 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:52.102442 19336 consensus_queue.cc:237] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:52.102062 19434 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:52.110810 19285 consensus_meta.cc:287] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.601s	user 0.007s	sys 0.003s
I20260709 13:19:52.110630 19434 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:52.084553 19432 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:52.111913 19426 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.111707 19432 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:52.112455 19426 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:52.064904 19410 consensus_queue.cc:237] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:52.114660 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:52.091897 19420 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 13:19:52.128003 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:52.128798 15676 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:19:52.115232 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:52.115499 15671 raft_consensus.cc:1240] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:19:52.119096 15527 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:52.134824 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:52.135462 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 48146.7us
I20260709 13:19:52.137212 15673 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:52.136269 16203 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:52.092015 19398 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.137331 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:52.065860 19413 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:52.135499 15601 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:19:52.191881 15676 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:52.142621 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:52.142742 16238 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:52.111322 19285 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:52.209182 15668 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:19:52.130769 19295 consensus_meta.cc:287] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.518s	user 0.006s	sys 0.003s
I20260709 13:19:52.117674 19432 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:52.125140 19090 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:52.134424 19237 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:52.206341 19062 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:19:52.232013 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 133658us
I20260709 13:19:52.239297 15603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53324 (request call id 1899) took 1221 ms. Trace:
I20260709 13:19:52.239892 16204 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:52.239738 15603 rpcz_store.cc:276] 0709 13:19:51.017841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.018000 (+   159us) service_pool.cc:224] Handling call
0709 13:19:52.239074 (+1221074us) spinlock_profiling.cc:243] Waited 419 ms on lock 0x7b540030eb54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:52.239231 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":963358080}
I20260709 13:19:52.240921 19090 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:52.242954 19447 consensus_queue.cc:1059] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:52.244746 19090 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:52.244946 19337 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:52.245625 19337 raft_consensus.cc:3055] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:52.134413 19151 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.100s
I20260709 13:19:52.206722 19251 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:52.258538 15641 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:52.111733 19439 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.127s
I20260709 13:19:52.264159 16204 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:52.269220 15676 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:19:52.210610 19398 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:52.226257 19295 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:52.275074 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:52.293666 15672 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:52.275811 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:52.277004 19406 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:52.280527 19434 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:52.271459 18784 consensus_queue.cc:1048] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:19:52.270977 19443 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:52.260914 19148 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.235417 19237 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:52.287916 19282 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:52.289227 19425 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:52.290678 19418 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:52.138669 19428 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:52.295132 15669 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:52.259224 19337 raft_consensus.cc:740] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:52.318318 15672 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
W20260709 13:19:52.273090 19315 consensus_meta.cc:287] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.691s	user 0.011s	sys 0.001s
I20260709 13:19:52.261821 19062 raft_consensus.cc:697] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:19:52.304361 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 61954.3us
I20260709 13:19:52.301774 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:52.320227 16203 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:52.292114 19090 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:52.311173 19416 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:52.314688 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 179567us
I20260709 13:19:52.346995 15493 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:52.291920 19236 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:52.298353 19406 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:52.314162 15675 raft_consensus.cc:1275] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:52.274328 19251 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:52.319847 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:52.341858 15641 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:52.315013 19443 raft_consensus.cc:697] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:52.319401 19432 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.356791 15519 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
W20260709 13:19:52.350366 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 73728.6us
I20260709 13:19:52.345566 15669 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:19:52.358919 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:52.346410 15677 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 13:19:52.318898 19078 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:52.359802 15669 raft_consensus.cc:2393] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 8: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:52.359548 15677 pending_rounds.cc:85] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting all ops after (but not including) 8
I20260709 13:19:52.360517 15677 pending_rounds.cc:107] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 13:19:52.360793 15677 raft_consensus.cc:2889] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:52.360733 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:52.361868 16203 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:52.367228 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:52.343271 19344 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:52.312095 19148 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:52.351754 19406 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:52.368940 15673 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:19:52.355918 19401 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:52.372361 15675 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:19:52.374081 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:52.375386 15669 raft_consensus.cc:1240] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:19:52.372916 15676 raft_consensus.cc:1240] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:19:52.355718 19151 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:52.355612 19090 raft_consensus.cc:697] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:52.378217 15641 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:52.370899 19415 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.369392 19439 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:52.365528 19117 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:19:52.380110 15491 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:52.359367 19236 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:52.382905 16204 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:52.368829 19078 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:52.384531 15493 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:52.376251 19128 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:52.365320 19432 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:52.388355 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:19:52.372944 15410 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 582343936346420b87ecf2eebd00dc25 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3ef186ec12554a1d87814af0a0ace9a4 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:52.389156 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:52.402405 16234 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:52.381595 15566 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:52.359570 19416 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:52.394163 15493 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:52.394606 19090 consensus_queue.cc:237] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:52.398698 19442 log.cc:927] Time spent T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:19:52.399206 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:52.417595 16237 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:52.399909 16231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1224) took 1173 ms. Trace:
I20260709 13:19:52.401111 16238 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:52.379462 19451 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:52.380465 19448 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:52.383852 19282 raft_consensus.cc:2764] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:19:52.381558 19401 raft_consensus.cc:697] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:52.408326 19078 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:52.408932 19416 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:52.376504 19344 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:52.415951 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:52.387840 19435 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:52.418787 16231 rpcz_store.cc:276] 0709 13:19:51.226478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.228139 (+  1661us) service_pool.cc:224] Handling call
0709 13:19:51.228418 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:51.653696 (+425278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.399670 (+745974us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b54004efa54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:52.399853 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":976407936}
I20260709 13:19:52.383505 19415 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:52.425907 15566 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:52.400220 19418 consensus_queue.cc:1059] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 13:19:52.386200 19062 consensus_queue.cc:237] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:52.430529 19435 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:52.479598 16234 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:52.390058 19337 consensus_queue.cc:260] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:52.483304 16238 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:19:52.431342 19295 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:52.432842 19285 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:52.492259 19456 raft_consensus.cc:3055] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:52.435971 19416 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:52.435828 19451 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:19:52.494221 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 100125us
W20260709 13:19:52.494776 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 109004us
I20260709 13:19:52.471374 19398 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:52.498461 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:52.435406 19434 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.487273 19435 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:52.502710 16238 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:19:52.502164 16237 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:52.480444 19427 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:52.480898 19237 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:52.512075 15493 leader_election.cc:400] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10.
I20260709 13:19:52.500123 18784 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:52.513885 15493 leader_election.cc:403] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:52.510306 19330 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:52.500605 19425 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:52.516652 15493 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:52.502072 19456 raft_consensus.cc:740] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:52.507895 19337 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:52.527581 15601 raft_consensus.cc:1275] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:19:52.520207 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:52.527753 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:52.520936 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:52.521271 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:52.521557 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:52.528622 15527 raft_consensus.cc:2393] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 8: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:52.522962 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:52.528805 15519 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:19:52.518494 15642 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:52.529294 15522 raft_consensus.cc:2393] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 10: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:52.530721 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:52.531314 15601 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:52.531194 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:52.531665 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:52.519101 19428 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:52.534019 16238 raft_consensus.cc:1240] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:52.537798 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:52.538360 15523 raft_consensus.cc:2376] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10. Re-sending same reply.
I20260709 13:19:52.538792 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:52.539306 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:52.531796 19416 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:52.517452 19427 raft_consensus.cc:697] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:52.539881 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:52.535084 16239 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:52.540927 15600 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:19:52.515386 18784 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:52.514851 19251 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:52.536489 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:52.536715 19453 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:52.537000 19344 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:52.533495 19237 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:52.526365 19432 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:52.504498 19285 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:52.535781 19148 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:52.583007 19148 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:52.583465 19148 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:52.583841 19148 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:52.453339 19448 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:52.539227 19454 consensus_queue.cc:1059] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 13:19:52.539876 15520 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:52.537298 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:52.587049 15603 raft_consensus.cc:2393] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 8: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:52.549434 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:52.551481 15526 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:52.551476 18784 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:52.404238 19439 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:52.401775 19443 consensus_queue.cc:237] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:52.555044 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 176899us
I20260709 13:19:52.510434 19434 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:52.538720 19415 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:52.515861 19401 consensus_queue.cc:237] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:52.586484 19237 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:52.532107 19451 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10.
I20260709 13:19:52.549754 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:52.519157 19330 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:52.535722 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:19:52.629472 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 111047us
I20260709 13:19:52.630075 15600 raft_consensus.cc:2393] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 6: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:52.632576 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:52.631618 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 197059us
I20260709 13:19:52.541242 19427 consensus_queue.cc:237] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:54.120855 15677 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
W20260709 13:19:54.122450 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 1.47289e+06us
I20260709 13:19:52.688827 16235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57258 (request call id 819) took 1048 ms. Trace:
I20260709 13:19:54.111243 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:53.055593 16240 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.017s	sys 0.032s
W20260709 13:19:54.132831 16240 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.619s	user 0.031s	sys 0.049s
W20260709 13:19:54.133266 15677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57710 (request call id 1093) took 1579 ms (client timeout 1538 ms). Trace:
I20260709 13:19:54.134709 16240 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53976 (request call id 865) took 1746 ms. Trace:
I20260709 13:19:54.129315 15641 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:52.588205 19453 raft_consensus.cc:697] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:19:54.443511 19148 consensus_meta.cc:287] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 1.859s	user 0.009s	sys 0.002s
I20260709 13:19:54.439152 19210 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:54.011350 19304 consensus_queue.cc:1048] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.460s
I20260709 13:19:54.470950 19210 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:54.470306 19148 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:54.474027 19148 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:54.435474 19151 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.132s	user 0.005s	sys 0.012s
W20260709 13:19:54.475226 19151 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.297s	user 0.008s	sys 0.014s
I20260709 13:19:54.475809 19151 raft_consensus.cc:2764] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 13:19:54.337881 16204 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.328s	user 0.005s	sys 0.014s
W20260709 13:19:54.477233 16204 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.468s	user 0.010s	sys 0.014s
W20260709 13:19:54.338289 15643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.127s	user 0.000s	sys 0.019s
W20260709 13:19:54.480723 15643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.270s	user 0.000s	sys 0.024s
I20260709 13:19:54.484843 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:54.485625 15671 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:54.474730 19148 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:54.490650 19148 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:54.163004 19404 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:54.497256 19404 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:54.499887 19404 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:54.219947 18784 consensus_meta.cc:287] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 1.571s	user 0.006s	sys 0.002s
I20260709 13:19:54.520443 18784 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:54.522774 18784 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:54.134433 19090 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.384s	user 0.001s	sys 0.008s
W20260709 13:19:54.523555 19090 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.774s	user 0.001s	sys 0.013s
I20260709 13:19:54.524185 19090 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:54.524653 19482 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.524767 19090 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:54.525249 19482 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:54.124191 16235 rpcz_store.cc:276] 0709 13:19:51.635370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.635586 (+   216us) service_pool.cc:224] Handling call
0709 13:19:51.655058 (+ 19472us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:51.671341 (+ 16283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:51.671360 (+    19us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:51.682615 (+ 11255us) log.cc:844] Serialized 52 byte log entry
0709 13:19:52.087312 (+404697us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:52.236698 (+149386us) log.cc:844] Serialized 14 byte log entry
0709 13:19:52.256057 (+ 19359us) log.cc:844] Serialized 14 byte log entry
0709 13:19:52.256307 (+   250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.256410 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:52.669113 (+412703us) raft_consensus.cc:1685] finished
0709 13:19:52.669132 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:52.669282 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50855,"prepare.queue_time_us":311185,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":4043,"raft.queue_time_us":182301,"raft.run_cpu_time_us":373,"raft.run_wall_time_us":39228,"thread_start_us":408267,"threads_started":2,"wal-append.queue_time_us":494563,"wal-append.run_cpu_time_us":4835,"wal-append.run_wall_time_us":1369279}
I20260709 13:19:52.851168 19432 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:54.530445 19432 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:54.532668 19432 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:54.501410 19478 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.555662 19478 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:54.557298 19151 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:54.557950 19151 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:54.133736 15677 rpcz_store.cc:269] 0709 13:19:52.542159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.542302 (+   143us) service_pool.cc:224] Handling call
0709 13:19:54.121502 (+1579200us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:54.134267 19435 consensus_meta.cc:287] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 1.615s	user 0.010s	sys 0.000s
I20260709 13:19:54.559487 19435 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:54.559962 19151 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:54.561897 19435 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:54.473604 19210 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:54.765759 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 329181us
I20260709 13:19:54.768491 16203 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:54.218305 19237 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:54.769110 19210 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.770332 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:54.770936 19401 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.004s	sys 0.011s
I20260709 13:19:54.772039 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:54.897908 15677 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:54.898051 15675 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:54.772310 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:54.772671 19427 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.000s	sys 0.015s
W20260709 13:19:54.899232 19401 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.004s	sys 0.011s
W20260709 13:19:54.899415 19427 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.000s	sys 0.016s
W20260709 13:19:54.780704 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 252584us
I20260709 13:19:54.527236 19090 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:54.527221 19482 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:54.899107 19210 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:54.903715 19210 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:54.903937 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:54.905547 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:54.907049 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:54.907781 15675 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 7: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:54.908957 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:54.909441 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 141030us
I20260709 13:19:54.910173 16203 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:54.910758 19090 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:54.910925 19427 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:54.911427 19427 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:54.911317 19090 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:54.911849 19427 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:54.912109 19151 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:54.912623 19151 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:54.912407 19210 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:54.914147 19210 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:54.914587 19210 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:54.916215 19090 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:54.846021 19478 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:54.493618 19148 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:54.943696 15642 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.66:41593 timed out after 1.789s (SENT) [suppressed 17 similar messages]
I20260709 13:19:54.914718 19151 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:19:53.061810 19315 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.007s	sys 0.023s
W20260709 13:19:54.973237 19315 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.442s	user 0.015s	sys 0.052s
I20260709 13:19:52.720567 19459 raft_consensus.cc:3055] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:54.974685 19459 raft_consensus.cc:740] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:54.975584 19459 consensus_queue.cc:260] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:54.976701 19459 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:52.697722 19351 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.978375 19351 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:54.980623 19351 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:54.981345 16233 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.000s	sys 0.000s
W20260709 13:19:54.981730 16233 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.000s	sys 0.000s
I20260709 13:19:54.982549 16233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53976 (request call id 862) took 2595 ms. Trace:
I20260709 13:19:54.982985 16233 rpcz_store.cc:276] 0709 13:19:52.387225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.387353 (+   128us) service_pool.cc:224] Handling call
0709 13:19:52.387648 (+   295us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:52.399789 (+ 12141us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:52.399819 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:52.410571 (+ 10752us) log.cc:844] Serialized 52 byte log entry
0709 13:19:54.108775 (+1698204us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:54.445759 (+336984us) log.cc:844] Serialized 14 byte log entry
0709 13:19:54.982163 (+536404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:54.982246 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:54.982293 (+    47us) raft_consensus.cc:1685] finished
0709 13:19:54.982305 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:54.982451 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1442,"prepare.queue_time_us":359952,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":101716,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":5034,"thread_start_us":1061148,"threads_started":3,"wal-append.queue_time_us":1702043}
I20260709 13:19:54.986284 19453 consensus_queue.cc:237] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:55.000108 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 89998.9us
I20260709 13:19:53.972098 19465 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:19:55.012549 19474 log.cc:927] Time spent T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.568s	user 0.000s	sys 0.001s
I20260709 13:19:55.014267 16231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1232) took 2544 ms. Trace:
I20260709 13:19:55.014781 16231 rpcz_store.cc:276] 0709 13:19:52.470368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.481724 (+ 11356us) service_pool.cc:224] Handling call
0709 13:19:52.503458 (+ 21734us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:52.507052 (+  3594us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:52.507083 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:52.514600 (+  7517us) log.cc:844] Serialized 26 byte log entry
0709 13:19:52.679893 (+165293us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:52.683467 (+  3574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.683552 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:55.013997 (+2330445us) raft_consensus.cc:1685] finished
0709 13:19:55.014017 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:55.014178 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":201,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":6263,"thread_start_us":121098,"threads_started":1,"wal-append.queue_time_us":1911912}
I20260709 13:19:55.016582 19475 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:55.017326 19475 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:54.470263 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 341004us
I20260709 13:19:54.135104 16240 rpcz_store.cc:276] 0709 13:19:52.388553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.388808 (+   255us) service_pool.cc:224] Handling call
0709 13:19:52.402348 (+ 13540us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:52.433094 (+ 30746us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:52.433133 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:52.453645 (+ 20512us) log.cc:844] Serialized 26 byte log entry
0709 13:19:54.134376 (+1680731us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:54.134414 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:54.134478 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:54.134521 (+    43us) raft_consensus.cc:1685] finished
0709 13:19:54.134531 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:54.134654 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15654,"raft.queue_time_us":25395,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":31563,"thread_start_us":1620578,"threads_started":1,"wal-append.queue_time_us":1643540}
I20260709 13:19:55.001317 16203 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:55.122324 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 121068us
W20260709 13:19:54.076678 19337 consensus_meta.cc:287] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 1.543s	user 0.009s	sys 0.000s
I20260709 13:19:55.283452 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:55.123711 19401 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:55.012696 19465 raft_consensus.cc:697] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
W20260709 13:19:54.082600 19441 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.685s	user 0.004s	sys 0.006s
I20260709 13:19:54.469257 19330 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:52.810461 19425 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:52.720306 19464 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:52.669551 19128 raft_consensus.cc:697] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:52.748390 19458 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:54.177455 19415 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.297762 19401 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:54.158527 19285 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:54.439517 19473 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:54.158157 19414 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:54.173606 19434 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:55.273103 19315 consensus_queue.cc:1048] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:19:52.809330 19406 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:55.337358 19304 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.341902 19401 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:55.371173 16204 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.859s	user 0.009s	sys 0.023s
W20260709 13:19:55.351090 16205 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.908s	user 0.006s	sys 0.009s
I20260709 13:19:55.342482 19151 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:55.394580 16205 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.951s	user 0.006s	sys 0.010s
W20260709 13:19:55.373370 16204 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.009s	sys 0.023s
I20260709 13:19:55.348491 19285 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:55.372107 15641 connection.cc:256] RPC call timeout handler was delayed by -2.2762907930000438s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1885s for a response
W20260709 13:19:55.400764 15641 connection.cc:256] RPC call timeout handler was delayed by -1.7177583119998872s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20030000000000003s for a response
W20260709 13:19:55.401146 15641 connection.cc:256] RPC call timeout handler was delayed by -1.711458311999877s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20099999999999998s for a response
W20260709 13:19:55.401438 15641 connection.cc:256] RPC call timeout handler was delayed by -1.6682583119998071s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20579999999999998s for a response
W20260709 13:19:55.401712 15641 connection.cc:256] RPC call timeout handler was delayed by -1.5737583119998817s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2163s for a response
W20260709 13:19:55.402099 15641 connection.cc:256] RPC call timeout handler was delayed by -1.4303318019999551s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2217s for a response
I20260709 13:19:55.402506 19285 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:55.403149 15641 leader_election.cc:336] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.885s (SENT)
I20260709 13:19:55.398056 19151 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:55.403673 15641 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:55.405112 19285 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:55.405529 19151 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:55.407217 15668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 1084) took 3133 ms. Trace:
I20260709 13:19:55.407639 15668 rpcz_store.cc:276] 0709 13:19:52.273726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.274343 (+   617us) service_pool.cc:224] Handling call
0709 13:19:52.279585 (+  5242us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:52.379455 (+ 99870us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:52.379483 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:52.381015 (+  1532us) log.cc:844] Serialized 26 byte log entry
0709 13:19:52.512587 (+131572us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:52.512623 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:55.406799 (+2894176us) spinlock_profiling.cc:243] Waited 99.5 ms on lock 0x7b54004600d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:55.406847 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:55.406900 (+    53us) raft_consensus.cc:1685] finished
0709 13:19:55.406911 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:55.407120 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":637,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":5360,"spinlock_wait_cycles":228894976,"thread_start_us":116106,"threads_started":1,"wal-append.queue_time_us":149140,"wal-append.run_cpu_time_us":1592,"wal-append.run_wall_time_us":1423075}
I20260709 13:19:55.403034 19148 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.409618 19148 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:55.409891 19128 consensus_queue.cc:237] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:55.411810 19148 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:55.374567 19478 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:55.412873 19478 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:55.374213 19304 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:52.691578 19456 consensus_queue.cc:260] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:55.342463 19415 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:55.416869 19478 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:55.417465 19406 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:55.351043 19414 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:55.420212 19439 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:55.399266 19441 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.002s	user 0.010s	sys 0.016s
I20260709 13:19:55.342331 19425 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:55.423014 19465 consensus_queue.cc:237] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:55.379583 19473 raft_consensus.cc:697] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:55.337785 19464 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:55.419616 19210 consensus_meta.cc:287] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.505s	user 0.008s	sys 0.000s
I20260709 13:19:55.499652 19210 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:55.461589 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 58478.2us
I20260709 13:19:55.344003 19458 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:55.835451 19458 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:55.425078 19090 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:19:55.983093 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 139605us
W20260709 13:19:55.856280 19410 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.454s	user 0.028s	sys 0.074s
W20260709 13:19:56.498247 19410 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 4.096s	user 0.029s	sys 0.076s
I20260709 13:19:56.213557 19210 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:56.362581 19368 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.941s	user 0.001s	sys 0.023s
W20260709 13:19:56.610968 19427 consensus_meta.cc:287] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 1.698s	user 0.010s	sys 0.000s
I20260709 13:19:56.687633 19427 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:56.679172 19425 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:56.680051 19304 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:56.730249 19427 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:56.691365 19368 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.269s	user 0.003s	sys 0.024s
W20260709 13:19:56.733060 19401 consensus_meta.cc:287] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 1.335s	user 0.006s	sys 0.001s
I20260709 13:19:56.733681 19401 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:56.736701 19459 consensus_meta.cc:287] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 1.760s	user 0.012s	sys 0.000s
I20260709 13:19:56.721935 19464 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:56.767720 16205 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.916s	user 0.000s	sys 0.008s
I20260709 13:19:56.217439 19473 consensus_queue.cc:237] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:56.775923 19415 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:56.780332 16232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 857) took 5376 ms (client timeout 1875 ms). Trace:
W20260709 13:19:56.780871 16232 rpcz_store.cc:269] 0709 13:19:51.404188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.425217 (+ 21029us) service_pool.cc:224] Handling call
0709 13:19:56.780022 (+5354805us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54007b02d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:56.780210 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":374239616}
W20260709 13:19:56.768200 16205 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.992s	user 0.000s	sys 0.008s
W20260709 13:19:56.692533 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 1.09679e+06us
W20260709 13:19:57.946008 15641 leader_election.cc:336] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.010s (SENT)
W20260709 13:19:57.946942 15641 leader_election.cc:336] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.058s (SENT)
I20260709 13:19:57.947362 15641 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:19:57.948480 15641 leader_election.cc:336] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.163s (SENT)
I20260709 13:19:57.948875 15641 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:57.014935 19448 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:57.020107 15524 raft_consensus.cc:2393] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 8: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:19:57.951718 15524 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1934) took 5428 ms (client timeout 1753 ms). Trace:
W20260709 13:19:57.952087 15524 rpcz_store.cc:269] 0709 13:19:52.523215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.523316 (+   101us) service_pool.cc:224] Handling call
0709 13:19:57.951616 (+5428300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31616}
W20260709 13:19:57.953126 15524 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1938) took 5429 ms (client timeout 1952 ms). Trace:
W20260709 13:19:57.953429 15524 rpcz_store.cc:269] 0709 13:19:52.523903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.952957 (+5429054us) service_pool.cc:211] Skipping call since client already timed out
0709 13:19:57.953073 (+   116us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:19:57.023626 15602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 952) took 4424 ms. Trace:
I20260709 13:19:57.954263 15602 rpcz_store.cc:276] 0709 13:19:52.599239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.599366 (+   127us) service_pool.cc:224] Handling call
0709 13:19:57.023024 (+4423658us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:57.023242 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:57.023269 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:57.023311 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:57.023343 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.023396 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:57.023521 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.955713 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:57.956522 15602 raft_consensus.cc:2410] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:19:57.957389 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:57.025794 15520 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1937) took 4502 ms (client timeout 1705 ms). Trace:
W20260709 13:19:57.961191 15520 rpcz_store.cc:269] 0709 13:19:52.523758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.025524 (+4501766us) service_pool.cc:211] Skipping call since client already timed out
0709 13:19:57.025694 (+   170us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:19:57.035879 19414 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:57.655532 15519 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1988) took 5131 ms (client timeout 1885 ms). Trace:
W20260709 13:19:57.964994 15519 rpcz_store.cc:269] 0709 13:19:52.524375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.655294 (+5130919us) service_pool.cc:211] Skipping call since client already timed out
0709 13:19:57.655448 (+   154us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:19:57.965890 15519 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56840 (request call id 1175) took 5438 ms (client timeout 2003 ms). Trace:
W20260709 13:19:57.966264 15519 rpcz_store.cc:269] 0709 13:19:52.527574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.965703 (+5438129us) service_pool.cc:211] Skipping call since client already timed out
0709 13:19:57.965826 (+   123us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:19:57.967001 15519 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1989) took 5442 ms (client timeout 2010 ms). Trace:
W20260709 13:19:57.967295 15519 rpcz_store.cc:269] 0709 13:19:52.524529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.966849 (+5442320us) service_pool.cc:211] Skipping call since client already timed out
0709 13:19:57.966951 (+   102us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:19:57.967988 15519 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1991) took 5443 ms (client timeout 2058 ms). Trace:
W20260709 13:19:57.968271 15519 rpcz_store.cc:269] 0709 13:19:52.524845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.967850 (+5443005us) service_pool.cc:211] Skipping call since client already timed out
0709 13:19:57.967944 (+    94us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:19:57.968935 15519 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1990) took 5444 ms (client timeout 2163 ms). Trace:
W20260709 13:19:57.969218 15519 rpcz_store.cc:269] 0709 13:19:52.524697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.968786 (+5444089us) service_pool.cc:211] Skipping call since client already timed out
0709 13:19:57.968889 (+   103us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:19:57.659564 15596 consensus_meta.cc:287] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 5.126s	user 0.020s	sys 0.049s
I20260709 13:19:57.970296 15596 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
W20260709 13:19:57.970839 15596 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53324 (request call id 1909) took 5440 ms (client timeout 1803 ms). Trace:
W20260709 13:19:57.971146 15596 rpcz_store.cc:269] 0709 13:19:52.530837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.530983 (+   146us) service_pool.cc:224] Handling call
0709 13:19:57.970754 (+5439771us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.972152 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:57.973192 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:19:57.728838 15600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53324 (request call id 1910) took 5193 ms (client timeout 2694 ms). Trace:
W20260709 13:19:57.974676 15600 rpcz_store.cc:269] 0709 13:19:52.541703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.544307 (+  2604us) service_pool.cc:224] Handling call
0709 13:19:57.728694 (+5184387us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.975597 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:57.976644 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:57.766927 15566 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:57.767549 19504 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:57.979584 19504 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:57.980057 19504 raft_consensus.cc:697] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:57.980873 19504 consensus_queue.cc:237] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:57.786145 15595 raft_consensus.cc:1275] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:57.983184 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:57.984202 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:57.984694 15595 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:19:57.791702 19505 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.792418 19506 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:57.990480 19506 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:57.804401 19508 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:57.991555 19508 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:57.992877 19506 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:57.794703 19507 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:57.993101 19401 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:57.833683 19416 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:57.814705 19510 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.993541 19507 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:57.994338 19510 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:57.995752 19445 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:57.995808 19507 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:57.996505 19510 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:57.845963 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:19:57.858309 15567 connection.cc:256] RPC call timeout handler was delayed by -4.9333529989999079s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20920000000000002s for a response
W20260709 13:19:57.997758 15567 connection.cc:256] RPC call timeout handler was delayed by -4.9261529989998962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21000000000000002s for a response
W20260709 13:19:57.998112 15567 connection.cc:256] RPC call timeout handler was delayed by -4.89915299900008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.213s for a response
W20260709 13:19:57.998558 15567 connection.cc:256] RPC call timeout handler was delayed by -4.6371154449998357s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1753s for a response
I20260709 13:19:57.998633 19507 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:57.998896 15567 connection.cc:256] RPC call timeout handler was delayed by -4.5847314159998405s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1927s for a response
I20260709 13:19:57.865689 19513 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:57.999167 19507 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:57.999567 19513 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.001420 19507 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:57.872539 15527 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1935) took 5349 ms (client timeout 1930 ms). Trace:
W20260709 13:19:58.002065 15527 rpcz_store.cc:269] 0709 13:19:52.523464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.872285 (+5348821us) service_pool.cc:211] Skipping call since client already timed out
0709 13:19:57.872435 (+   150us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:19:57.994212 19416 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:58.003000 15527 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1993) took 5477 ms (client timeout 2217 ms). Trace:
W20260709 13:19:58.003371 15527 rpcz_store.cc:269] 0709 13:19:52.525157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.002819 (+5477662us) service_pool.cc:211] Skipping call since client already timed out
0709 13:19:58.002935 (+   116us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:19:58.004823 19416 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:57.888823 19514 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.005517 19514 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:57.999202 15567 connection.cc:256] RPC call timeout handler was delayed by -4.48051544499981s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1927s for a response
W20260709 13:19:58.007459 15567 connection.cc:256] RPC call timeout handler was delayed by -4.0371849649998239s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.193s for a response
I20260709 13:19:57.889482 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:58.001762 19513 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:57.894157 15640 connection.cc:256] RPC call timeout handler was delayed by -4.3416374740002084s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 13:19:57.977648 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 210801us
I20260709 13:19:58.007615 19514 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:57.993979 19508 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:58.008315 15567 connection.cc:256] RPC call timeout handler was delayed by -4.0330107170000247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1705s for a response
I20260709 13:19:57.894815 19515 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:57.900315 19304 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:58.009104 15567 connection.cc:256] RPC call timeout handler was delayed by -4.0253119489998426s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1587s for a response
W20260709 13:19:58.009428 15567 connection.cc:256] RPC call timeout handler was delayed by -3.9208529990000898s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32170000000000004s for a response
W20260709 13:19:58.009732 15567 connection.cc:256] RPC call timeout handler was delayed by -3.8107107169998926s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1952s for a response
W20260709 13:19:58.010064 15567 connection.cc:256] RPC call timeout handler was delayed by -3.7278557539998474s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2217s for a response
I20260709 13:19:57.906541 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1172) took 5824 ms. Trace:
I20260709 13:19:58.010138 19515 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.010447 15521 rpcz_store.cc:276] 0709 13:19:52.082410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.082559 (+   149us) service_pool.cc:224] Handling call
0709 13:19:52.090337 (+  7778us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:52.115731 (+ 25394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:52.115761 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:52.115803 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:52.115839 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.906226 (+5790387us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b54003ccdd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:57.906275 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:57.906446 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57903360}
I20260709 13:19:58.011610 19506 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:58.012434 19515 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:58.012598 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1173) took 5485 ms. Trace:
I20260709 13:19:58.012959 15521 rpcz_store.cc:276] 0709 13:19:52.527079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.011839 (+5484760us) service_pool.cc:224] Handling call
0709 13:19:58.012146 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:58.012305 (+   159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:58.012329 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:58.012369 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:58.012398 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.012452 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.012547 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.910470 19463 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.008713 19445 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.014197 19463 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:58.015987 15410 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 582343936346420b87ecf2eebd00dc25 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3ef186ec12554a1d87814af0a0ace9a4 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:19:58.016296 19463 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:58.009447 19304 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:58.018057 15491 connection.cc:256] RPC call timeout handler was delayed by -4.1440002680001271s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
W20260709 13:19:58.018467 15491 connection.cc:256] RPC call timeout handler was delayed by -3.8884002680001686s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18030000000000002s for a response
W20260709 13:19:58.018769 15491 connection.cc:256] RPC call timeout handler was delayed by -3.6760002680000525s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20390000000000003s for a response
I20260709 13:19:58.019030 19304 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:58.019199 19510 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:58.013455 19506 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:19:58.019742 19510 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.020264 19413 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:58.013856 15520 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1992) took 5488 ms (client timeout 2003 ms). Trace:
W20260709 13:19:58.021027 15520 rpcz_store.cc:269] 0709 13:19:52.525009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.961880 (+5436871us) service_pool.cc:211] Skipping call since client already timed out
0709 13:19:58.013777 (+ 51897us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:19:57.913640 15522 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
W20260709 13:19:58.022949 15522 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1940) took 5498 ms (client timeout 1587 ms). Trace:
W20260709 13:19:58.023334 15522 rpcz_store.cc:269] 0709 13:19:52.524204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.566169 (+ 41965us) service_pool.cc:224] Handling call
0709 13:19:58.022871 (+5456702us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:58.023765 15524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1177) took 5495 ms. Trace:
I20260709 13:19:58.024115 15524 rpcz_store.cc:276] 0709 13:19:52.528094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.023000 (+5494906us) service_pool.cc:224] Handling call
0709 13:19:58.023307 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:58.023470 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:58.023495 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:58.023534 (+    39us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:58.023571 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.023621 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.023718 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:58.024708 19446 log.cc:927] Time spent T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 5.782s	user 0.043s	sys 0.061s
I20260709 13:19:58.025105 15600 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
W20260709 13:19:58.019052 15491 connection.cc:256] RPC call timeout handler was delayed by -3.0641158229998382s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26940000000000003s for a response
I20260709 13:19:57.923053 19408 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:19:58.025775 15672 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49100 (request call id 2280) took 5718 ms (client timeout 1930 ms). Trace:
W20260709 13:19:57.932482 16236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57258 (request call id 832) took 5397 ms (client timeout 1519 ms). Trace:
W20260709 13:19:57.917042 16204 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 2.071s	user 0.013s	sys 0.028s
I20260709 13:19:58.026130 19408 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:57.941385 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:57.932641 15624 debug-util.cc:398] Leaking SignalData structure 0x7b0800199760 after lost signal to thread 14432
I20260709 13:19:57.942905 19426 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:57.822958 19511 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.027405 19511 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.028978 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1174) took 5501 ms. Trace:
I20260709 13:19:57.837795 19454 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:58.029359 15521 rpcz_store.cc:276] 0709 13:19:52.527334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.013854 (+5486520us) service_pool.cc:224] Handling call
0709 13:19:58.028486 (+ 14632us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:58.028675 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:58.028699 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:58.028739 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:58.028774 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.028827 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.028925 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:58.026643 16204 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 2.181s	user 0.014s	sys 0.029s
I20260709 13:19:57.877090 15594 raft_consensus.cc:1240] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 12. Current term is 13. Ops: []
W20260709 13:19:57.882131 16205 connection.cc:256] RPC call timeout handler was delayed by -1.4919911880001564s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15380000000000002s for a response
W20260709 13:19:58.030350 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 2.18497e+06us
I20260709 13:19:58.031472 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:58.032061 15594 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:19:58.032363 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.032917 15669 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:19:58.030179 19454 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:57.990407 19505 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:19:58.026183 15672 rpcz_store.cc:269] 0709 13:19:52.307222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.319579 (+ 12357us) service_pool.cc:224] Handling call
0709 13:19:58.025559 (+5705980us) spinlock_profiling.cc:243] Waited 2.17 s on lock 0x7b540023ad54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:58.025682 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4995432704}
I20260709 13:19:58.034228 19413 consensus_queue.cc:1059] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:58.035140 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:58.035969 15669 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:58.036135 19508 raft_consensus.cc:3055] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:58.036513 19511 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:58.030047 15528 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:58.037214 19505 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:58.036556 19508 raft_consensus.cc:740] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:58.039194 19454 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:58.016912 19507 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
W20260709 13:19:58.041152 16236 rpcz_store.cc:269] 0709 13:19:52.534751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.932253 (+5397502us) service_pool.cc:211] Skipping call since client already timed out
0709 13:19:57.932383 (+   130us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:19:58.042095 19416 consensus_queue.cc:1059] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:58.043393 19448 raft_consensus.cc:3055] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:58.043761 19448 raft_consensus.cc:740] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:58.043803 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.044524 19448 consensus_queue.cc:260] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:58.045511 19448 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:58.044536 15671 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:19:58.046253 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.046838 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.047168 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.046839 15672 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:19:58.048257 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:58.048776 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.049885 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:58.030793 16205 connection.cc:256] RPC call timeout handler was delayed by -1.3407911880001393s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1706s for a response
W20260709 13:19:58.050837 16205 connection.cc:256] RPC call timeout handler was delayed by -1.2021911880001426s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18600000000000003s for a response
W20260709 13:19:58.051141 16205 connection.cc:256] RPC call timeout handler was delayed by -1.1481911880000553s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.192s for a response
W20260709 13:19:58.051432 16205 connection.cc:256] RPC call timeout handler was delayed by -1.020391188000076s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2062s for a response
I20260709 13:19:58.052111 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.046095 15669 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:19:58.021986 19510 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:58.054150 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:19:58.054348 15567 leader_election.cc:336] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.705s (SENT)
I20260709 13:19:58.054741 16232 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:58.047899 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.055534 15671 raft_consensus.cc:2393] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:58.048815 15673 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:58.056257 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1939) took 5532 ms. Trace:
I20260709 13:19:58.056978 15527 rpcz_store.cc:276] 0709 13:19:52.524061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.003952 (+5479891us) service_pool.cc:224] Handling call
0709 13:19:58.004293 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:58.004700 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:58.031111 (+ 26411us) log.cc:844] Serialized 14 byte log entry
0709 13:19:58.053255 (+ 22144us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:58.053715 (+   460us) log.cc:844] Serialized 26 byte log entry
0709 13:19:58.053865 (+   150us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:58.053898 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.053981 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:58.056071 (+  2090us) raft_consensus.cc:1685] finished
0709 13:19:58.056088 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.056200 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26336,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":1352,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":153,"spinlock_wait_cycles":114304,"thread_start_us":47668,"threads_started":2,"wal-append.queue_time_us":17627}
I20260709 13:19:58.054817 15567 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:58.059046 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:19:58.060752 15567 leader_election.cc:336] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.952s (SENT)
I20260709 13:19:58.061198 15567 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:58.062743 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:58.063491 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.065073 15528 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:19:58.066210 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.066712 16232 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:19:58.016086 19445 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:58.022467 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1176) took 5494 ms. Trace:
I20260709 13:19:58.067842 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.068341 15520 rpcz_store.cc:276] 0709 13:19:52.527862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.021645 (+5493783us) service_pool.cc:224] Handling call
0709 13:19:58.022005 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:58.022173 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:58.022196 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:58.022238 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:58.022268 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.022319 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.022412 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:58.070343 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:58.071043 15567 leader_election.cc:336] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.100s (SENT)
W20260709 13:19:58.071960 15567 leader_election.cc:336] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.130s (SENT)
I20260709 13:19:58.075764 15491 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:58.076584 19505 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:58.076812 15491 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:58.080832 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.084038 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.085147 15671 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 8: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
W20260709 13:19:58.085709 19090 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.230s	user 0.047s	sys 0.071s
W20260709 13:19:58.086118 19090 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.230s	user 0.048s	sys 0.071s
W20260709 13:19:58.087268 19282 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.313s	user 0.002s	sys 0.002s
W20260709 13:19:58.087661 19282 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.314s	user 0.003s	sys 0.003s
W20260709 13:19:58.090981 19128 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.316s	user 0.011s	sys 0.007s
W20260709 13:19:58.091387 19128 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.316s	user 0.011s	sys 0.007s
W20260709 13:19:58.093078 19368 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.360s	user 0.036s	sys 0.089s
W20260709 13:19:58.093454 19368 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.360s	user 0.036s	sys 0.089s
W20260709 13:19:58.095235 15568 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 5.608s	user 0.021s	sys 0.031s
W20260709 13:19:58.095932 15568 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 5.609s	user 0.022s	sys 0.031s
I20260709 13:19:58.096704 19518 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:58.097414 15568 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:58.097282 19518 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.098170 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:58.098934 19473 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.318s	user 0.003s	sys 0.007s
W20260709 13:19:58.099284 19473 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.319s	user 0.003s	sys 0.007s
I20260709 13:19:58.099830 19518 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:58.100310 19518 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.100744 19518 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:58.100757 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.103238 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.103744 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1179) took 5575 ms. Trace:
W20260709 13:19:58.104355 15675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57710 (request call id 1101) took 2980 ms (client timeout 1558 ms). Trace:
I20260709 13:19:58.103672 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.105302 16240 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:19:58.105734 19518 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:58.106241 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.106419 19518 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:58.107169 16240 raft_consensus.cc:2410] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:19:58.106916 19518 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.107975 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.108901 19518 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:58.109016 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.109452 19518 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:58.109607 16240 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:19:58.109735 19511 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:58.111091 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1178) took 5582 ms. Trace:
I20260709 13:19:58.111550 19448 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:58.112697 15491 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:58.104144 15521 rpcz_store.cc:276] 0709 13:19:52.528552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.030308 (+5501756us) service_pool.cc:224] Handling call
0709 13:19:58.030680 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:58.062204 (+ 31524us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:58.062234 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:58.062725 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 13:19:58.085909 (+ 23184us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:58.085972 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.086050 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:58.103529 (+ 17479us) raft_consensus.cc:1685] finished
0709 13:19:58.103549 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.103685 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31138,"raft.queue_time_us":32517,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"thread_start_us":22825,"threads_started":1,"wal-append.queue_time_us":20766}
I20260709 13:19:52.650362 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:58.114115 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.112694 19518 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:58.114730 15566 connection.cc:256] RPC call timeout handler was delayed by -4.8774869579999631s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:19:58.115372 19513 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:19:58.116148 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 167688us
I20260709 13:19:58.069837 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:58.039217 19508 consensus_queue.cc:260] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.099715 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:19:58.119408 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 63227us
W20260709 13:19:57.969879 15519 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1936) took 5446 ms (client timeout 2217 ms). Trace:
I20260709 13:19:58.098459 19459 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:52.655222 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.190698 15595 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:19:58.195219 15673 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:19:58.268988 19295 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:58.275179 19295 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:58.232511 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 144780us
I20260709 13:19:58.202831 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.278287 15599 raft_consensus.cc:2376] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9. Re-sending same reply.
I20260709 13:19:58.210857 19368 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.279376 19368 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.238247 19511 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:19:58.281232 19336 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 6.175s	user 0.055s	sys 0.048s
I20260709 13:19:58.281505 19511 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:58.281586 19336 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 6.175s	user 0.055s	sys 0.048s
I20260709 13:19:58.282923 19368 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:58.238493 19448 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:58.283807 19448 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:58.239691 19236 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:58.272941 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:58.243944 19518 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:58.288911 15493 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:58.289517 19510 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:58.289225 19511 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.289911 19510 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:58.241153 15603 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:58.290349 19510 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:58.291342 19511 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:58.104736 15675 rpcz_store.cc:269] 0709 13:19:55.124247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.124474 (+   227us) service_pool.cc:224] Handling call
0709 13:19:58.104121 (+2979647us) spinlock_profiling.cc:243] Waited 2.8 s on lock 0x7b54001a8954. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:58.104264 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6449183616}
I20260709 13:19:58.295249 15493 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:58.295997 19418 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:58.296664 19418 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:58.297850 15493 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:58.297755 19510 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.298684 19418 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:58.299150 19418 raft_consensus.cc:697] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:58.247989 15520 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:58.299968 19418 consensus_queue.cc:237] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:58.251255 15670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 1099) took 3479 ms. Trace:
I20260709 13:19:58.302675 15670 rpcz_store.cc:276] 0709 13:19:54.771350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.771503 (+   153us) service_pool.cc:224] Handling call
0709 13:19:54.771831 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:55.836176 (+1064345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:55.836206 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:55.836246 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:55.836290 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.251004 (+2414714us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54001a7f54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:58.251062 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.251191 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2447325952}
I20260709 13:19:58.254333 19459 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.306253 19459 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:58.254774 19475 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:58.307096 15493 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:58.248481 19508 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:58.310020 19511 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:58.255942 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.256441 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:58.255257 15519 rpcz_store.cc:269] 0709 13:19:52.523619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.969731 (+5446112us) service_pool.cc:211] Skipping call since client already timed out
0709 13:19:57.969835 (+   104us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:19:58.260921 15674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 1094) took 3778 ms. Trace:
I20260709 13:19:58.317767 15674 rpcz_store.cc:276] 0709 13:19:54.482302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.482443 (+   141us) service_pool.cc:224] Handling call
0709 13:19:54.482777 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:54.491876 (+  9099us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:54.491904 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:54.491944 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:54.491973 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.260662 (+3768689us) spinlock_profiling.cc:243] Waited 8.9 ms on lock 0x7b540021b954. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:58.260722 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.260858 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20472192}
I20260709 13:19:58.263340 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.265043 19488 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.320438 19488 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.322201 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.322549 19488 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:58.317312 15524 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:58.324155 15520 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:19:58.212764 19371 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:19:58.213016 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 115680us
I20260709 13:19:58.299988 19510 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:58.218861 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.213183 19506 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:58.224143 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:19:58.328665 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 104603us
I20260709 13:19:58.238363 15522 rpcz_store.cc:276] 0709 13:19:52.528324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.023976 (+5495652us) service_pool.cc:224] Handling call
0709 13:19:58.024253 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:58.024436 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:58.034735 (+ 10299us) log.cc:844] Serialized 14 byte log entry
0709 13:19:58.110651 (+ 75916us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:58.110716 (+    65us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:58.110758 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.110822 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.111027 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3576,"prepare.queue_time_us":6950,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":3685,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":107,"thread_start_us":81653,"threads_started":2,"wal-append.queue_time_us":47060}
I20260709 13:19:58.328218 19459 consensus_queue.cc:1059] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:19:58.329967 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:58.272817 19416 consensus_queue.cc:1048] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:58.331573 19282 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:58.332362 19410 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:58.332787 19410 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:58.332382 19282 raft_consensus.cc:697] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:58.333200 19410 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:58.327895 19518 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:19:58.334661 19518 raft_consensus.cc:3055] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:58.334995 19518 raft_consensus.cc:740] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:58.335767 19518 consensus_queue.cc:260] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.336905 19518 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:58.338165 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.225209 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.339236 19282 consensus_queue.cc:237] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.197360 19504 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:58.230973 19427 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.343968 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.343835 19427 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.344640 15668 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:19:58.230700 19434 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:19:58.346195 19434 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.347072 19236 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:58.348363 19434 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:58.349030 16237 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:19:58.330842 19459 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:19:58.351794 19427 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:58.352437 19504 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.331561 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:58.358286 19504 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:58.358768 16237 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:58.359468 19295 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:58.219398 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:58.246634 15566 connection.cc:256] RPC call timeout handler was delayed by -3.99906208099992s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.193s for a response
I20260709 13:19:58.360965 15603 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
W20260709 13:19:58.361759 15603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34102 (request call id 951) took 5762 ms (client timeout 2192 ms). Trace:
W20260709 13:19:58.362206 15603 rpcz_store.cc:269] 0709 13:19:52.598800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.598944 (+   144us) service_pool.cc:224] Handling call
0709 13:19:58.361492 (+5762548us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:58.324682 15524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1180) took 5795 ms. Trace:
W20260709 13:19:58.267033 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 154434us
I20260709 13:19:58.363986 15491 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:58.365118 15491 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:58.267379 19447 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:58.325595 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.104369 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:58.366343 19425 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:58.366389 15597 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:19:58.366953 19425 raft_consensus.cc:697] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:58.367470 19413 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:58.367925 19425 consensus_queue.cc:237] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:58.367898 19413 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:58.369686 19413 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:52.634313 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:58.372134 15601 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:58.366102 19447 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.375289 19447 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:58.375703 15528 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1932) took 5852 ms (client timeout 1927 ms). Trace:
W20260709 13:19:58.376107 15528 rpcz_store.cc:269] 0709 13:19:52.522816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.522904 (+    88us) service_pool.cc:224] Handling call
0709 13:19:57.889304 (+5366400us) spinlock_profiling.cc:243] Waited 1.68 ms on lock 0x7b5800003698. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6eee 0000000000375939 00007f5ce995a1e0 00007f5ce9a230c4 00007f5ce9a23131 00007f5ce99d10a0 00007f5ce99d0984 00007f5ce2f4b6d4 00007f5ce2f4b662
0709 13:19:58.375518 (+486214us) spinlock_profiling.cc:243] Waited 21.9 ms on lock 0x7b54003ce1d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:58.375622 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54322816}
W20260709 13:19:58.383322 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:58.386510 15624 debug-util.cc:398] Leaking SignalData structure 0x7b080014c5c0 after lost signal to thread 15460
I20260709 13:19:58.386981 15601 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:19:58.388350 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:58.388876 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:58.389670 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.390699 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:58.391577 15528 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
W20260709 13:19:58.390884 15601 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34102 (request call id 955) took 5757 ms (client timeout 1706 ms). Trace:
I20260709 13:19:58.391319 15597 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:58.391259 15669 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:19:58.393752 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:58.394445 15668 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:58.392110 15601 rpcz_store.cc:269] 0709 13:19:52.633905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.634059 (+   154us) service_pool.cc:224] Handling call
0709 13:19:58.390810 (+5756751us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:58.260628 16205 connection.cc:256] RPC call timeout handler was delayed by -1.1843659239998487s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15580000000000002s for a response
I20260709 13:19:58.520381 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:58.520812 16205 connection.cc:256] RPC call timeout handler was delayed by -1.169065923999824s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
I20260709 13:19:58.521046 15523 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:58.230516 15594 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:58.422317 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.522303 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:58.410422 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:58.523468 15601 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:58.524722 15676 raft_consensus.cc:1240] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:58.439576 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.525275 16204 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:19:58.488456 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.442224 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:58.527740 15603 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:19:58.446425 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.447083 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.433529 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.411507 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:58.530889 15598 raft_consensus.cc:2410] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:19:58.414768 15522 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:58.419610 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.416033 15597 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:58.459195 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.495757 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.430820 19466 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.469184 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.535914 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.398175 15524 rpcz_store.cc:276] 0709 13:19:52.528787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.116999 (+5588212us) service_pool.cc:224] Handling call
0709 13:19:58.117289 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:58.323904 (+206615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.324457 (+   553us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b5400188154. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:19:58.324616 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":459558144}
I20260709 13:19:58.486564 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.537248 15670 raft_consensus.cc:2368] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10.
I20260709 13:19:58.537317 15672 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:19:58.537901 15674 raft_consensus.cc:2393] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 9: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:19:58.477891 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.536454 15675 raft_consensus.cc:2393] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 7: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:58.538916 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.450362 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.519773 15520 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:58.084038 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.430604 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:58.540467 16232 raft_consensus.cc:2393] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 9: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:58.541066 15519 raft_consensus.cc:2410] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:19:58.536356 15566 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:58.525547 15600 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:19:58.542455 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.534781 15525 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:19:58.543047 15600 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:19:58.543668 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.544837 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.545454 15597 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:19:58.546422 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:58.546797 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.547612 15672 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:19:58.549445 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:58.550333 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.542666 15566 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:58.317348 19511 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:58.554931 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.543020 18854 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:58.525611 19464 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:58.556923 19438 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:58.418720 19090 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.117s
I20260709 13:19:58.526571 19454 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:58.551383 19513 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:58.569231 15601 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:19:58.571424 15668 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:19:58.579559 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.177660 19408 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.595671 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.583073 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:58.585417 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.573096 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.574856 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.599045 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:58.597189 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.600651 15603 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:19:58.591739 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.578647 19416 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:58.602730 19416 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:19:58.604081 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:58.591127 18854 raft_consensus.cc:697] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:58.604962 19416 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:58.584492 19438 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:58.602749 19418 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:58.605821 19438 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:58.605947 19418 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:58.607218 15671 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:58.608004 19418 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:58.567135 19466 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.609035 19428 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:58.610901 19466 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:58.611405 19428 consensus_queue.cc:1059] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:58.612607 19434 raft_consensus.cc:3055] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:58.612982 19434 raft_consensus.cc:740] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:58.602710 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.613827 19434 consensus_queue.cc:260] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.615324 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.605223 18854 consensus_queue.cc:237] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:58.620488 19434 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:58.623390 15602 raft_consensus.cc:2393] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 14: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
W20260709 13:19:58.623764 15624 debug-util.cc:398] Leaking SignalData structure 0x7b08000dc480 after lost signal to thread 15552
I20260709 13:19:58.578990 19295 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.626359 19295 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:58.626649 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:58.626902 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.628384 19295 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:58.578338 19506 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:58.628993 15600 raft_consensus.cc:2393] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 13: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:58.629705 19456 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:58.630064 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:58.628618 15599 raft_consensus.cc:2393] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 10: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:58.632464 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.633587 19506 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:58.633980 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:58.634279 15624 debug-util.cc:398] Leaking SignalData structure 0x7b08002f5ac0 after lost signal to thread 15627
W20260709 13:19:58.634631 15624 debug-util.cc:398] Leaking SignalData structure 0x7b08002a86e0 after lost signal to thread 15628
I20260709 13:19:58.634035 19506 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:58.638005 19512 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.638345 15527 raft_consensus.cc:2410] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:19:58.638576 19512 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:58.639891 15642 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:19:58.640779 19274 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:58.641145 19274 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:58.637997 19465 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.420s	user 0.079s	sys 0.115s
W20260709 13:19:58.641882 19465 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.424s	user 0.080s	sys 0.115s
I20260709 13:19:58.628726 15415 maintenance_manager.cc:419] P 88b9585660f045bda650d7ffa0d7a0b7: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033354
W20260709 13:19:58.641314 15624 debug-util.cc:398] Leaking SignalData structure 0x7b0800144480 after lost signal to thread 15701
W20260709 13:19:58.643718 15624 debug-util.cc:398] Leaking SignalData structure 0x7b08001a11a0 after lost signal to thread 15702
I20260709 13:19:58.643756 19512 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:19:58.645013 15624 debug-util.cc:398] Leaking SignalData structure 0x7b08002798a0 after lost signal to thread 16266
W20260709 13:19:58.645583 15624 debug-util.cc:398] Leaking SignalData structure 0x7b0800238420 after lost signal to thread 19285
W20260709 13:19:58.645859 15624 debug-util.cc:398] Leaking SignalData structure 0x7b080033bc00 after lost signal to thread 19315
W20260709 13:19:58.646201 15624 debug-util.cc:398] Leaking SignalData structure 0x7b0800328800 after lost signal to thread 19330
W20260709 13:19:58.646682 15624 debug-util.cc:398] Leaking SignalData structure 0x7b08001959e0 after lost signal to thread 19421
I20260709 13:19:58.646565 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:19:58.647112 15624 debug-util.cc:398] Leaking SignalData structure 0x7b08003a4660 after lost signal to thread 19439
W20260709 13:19:58.647488 15624 debug-util.cc:398] Leaking SignalData structure 0x7b08003ea200 after lost signal to thread 19453
W20260709 13:19:58.647749 15624 debug-util.cc:398] Leaking SignalData structure 0x7b0800410f80 after lost signal to thread 19458
I20260709 13:19:58.650439 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.651803 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.652884 15595 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:19:58.653090 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.654181 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.654749 15669 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:19:58.643085 15671 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:58.630867 15600 raft_consensus.cc:2410] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:19:58.655272 15595 raft_consensus.cc:1240] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: []
W20260709 13:19:58.627701 16203 connection.cc:256] RPC call timeout handler was delayed by -2.0386801839999862s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15580000000000002s for a response
W20260709 13:19:58.657322 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 46209.6us
I20260709 13:19:58.655572 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.647712 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.660491 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.662910 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.628821 19408 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:58.514824 19448 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:58.601548 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:58.672813 15642 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:58.669515 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.673193 15603 raft_consensus.cc:2376] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5. Re-sending same reply.
I20260709 13:19:58.673736 15600 raft_consensus.cc:1813] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Handling vote request from an unknown peer 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:58.670571 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.671586 15568 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:58.665865 15594 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:58.600845 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.675014 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:58.678057 15568 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:58.678936 16231 raft_consensus.cc:2393] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 9: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:58.681579 15600 raft_consensus.cc:2410] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:19:58.681718 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.682428 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:19:58.683611 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.687034 15491 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:19:58.641518 19274 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:58.696756 15568 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:58.701606 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.724428 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.702167 15603 raft_consensus.cc:1240] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:58.690178 19418 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.700601 16205 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:19:58.711350 19448 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:58.712505 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.594412 19413 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.715809 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.719952 16203 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:58.722227 19447 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:58.722889 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:58.700421 15601 raft_consensus.cc:1240] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:58.725682 15594 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:19:58.726887 15669 raft_consensus.cc:2368] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 7. Current term is 8.
I20260709 13:19:58.734570 16239 raft_consensus.cc:2393] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 8: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:19:58.684885 19509 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:58.679085 19465 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:58.690531 19466 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:58.743719 15642 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:58.685072 19504 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:58.737618 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:19:58.714219 15493 connection.cc:256] RPC call timeout handler was delayed by -4.4158832650000477s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
I20260709 13:19:58.738840 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.740155 15491 leader_election.cc:400] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 7. Current term is 8.
I20260709 13:19:58.741688 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.752079 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:19:58.752482 15493 connection.cc:256] RPC call timeout handler was delayed by -4.05588326499992s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19190000000000002s for a response
I20260709 13:19:58.753746 15491 leader_election.cc:403] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:19:58.747761 15641 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.021s	sys 0.060s
W20260709 13:19:58.754729 15641 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.022s	sys 0.060s
W20260709 13:19:58.754815 15493 connection.cc:256] RPC call timeout handler was delayed by -3.6319832649999171s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23900000000000002s for a response
W20260709 13:19:58.757154 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 510112us
I20260709 13:19:58.730295 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:58.726640 19295 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.726454 19508 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.734282 19456 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:58.734560 19454 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:58.769146 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.698537 19410 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:19:58.769934 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 50042.5us
I20260709 13:19:58.728637 19506 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.770342 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:58.779764 15641 connection.cc:256] RPC call timeout handler was delayed by -2.3636357880002379s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1504s for a response
I20260709 13:19:58.733695 19438 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.771239 16205 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:19:58.746876 19128 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:58.773725 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.789259 16239 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:19:58.774549 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.758441 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:58.790673 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:58.778647 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:58.770498 19445 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:58.735540 19418 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:58.773065 15594 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 13:19:58.783851 16203 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:58.807690 16238 raft_consensus.cc:1240] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:19:58.797600 16232 raft_consensus.cc:1240] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:19:58.809340 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:58.781582 15641 connection.cc:256] RPC call timeout handler was delayed by -2.2349357880002572s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1647s for a response
I20260709 13:19:58.810148 15527 raft_consensus.cc:2393] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 11: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:19:58.810307 15641 connection.cc:256] RPC call timeout handler was delayed by -2.1485357880001175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1743s for a response
W20260709 13:19:58.810716 15641 connection.cc:256] RPC call timeout handler was delayed by -2.1026357880000432s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
W20260709 13:19:58.810986 15641 connection.cc:256] RPC call timeout handler was delayed by -2.0576357880001979s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1844s for a response
W20260709 13:19:58.811249 15641 connection.cc:256] RPC call timeout handler was delayed by -2.054035788000192s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18480000000000002s for a response
W20260709 13:19:58.811522 15641 connection.cc:256] RPC call timeout handler was delayed by -1.9829357880000771s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1927s for a response
W20260709 13:19:58.811779 15641 connection.cc:256] RPC call timeout handler was delayed by -1.9298357880002186s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1986s for a response
W20260709 13:19:58.812044 15641 connection.cc:256] RPC call timeout handler was delayed by -1.9271357880002142s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19890000000000002s for a response
W20260709 13:19:58.812310 15641 connection.cc:256] RPC call timeout handler was delayed by -1.8344357880000643s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20920000000000002s for a response
W20260709 13:19:58.812564 15641 connection.cc:256] RPC call timeout handler was delayed by -1.8245357880000483s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21030000000000004s for a response
W20260709 13:19:58.812827 15641 connection.cc:256] RPC call timeout handler was delayed by -1.7867357880002146s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21450000000000002s for a response
I20260709 13:19:58.787148 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:19:58.813086 15641 connection.cc:256] RPC call timeout handler was delayed by -1.4087357880000582s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2565s for a response
I20260709 13:19:58.792042 15568 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:58.794354 15526 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:19:58.798810 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:58.782078 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:58.779666 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:59.485488 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:59.497680 15519 raft_consensus.cc:2410] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:19:58.813669 15526 pending_rounds.cc:85] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 8
I20260709 13:19:58.814335 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:58.807317 15594 pending_rounds.cc:85] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting all ops after (but not including) 9
I20260709 13:19:59.599707 15594 pending_rounds.cc:107] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:19:59.600023 15594 raft_consensus.cc:2889] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:59.601265 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:58.871698 15595 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 13. Current term is 14. Ops: []
W20260709 13:19:58.908732 15641 leader_election.cc:336] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.504s (SENT)
I20260709 13:19:58.814604 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:19:59.649392 15669 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:19:58.856106 15673 raft_consensus.cc:1240] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:58.820624 15675 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:19:59.652668 15675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 1120) took 1017 ms. Trace:
I20260709 13:19:59.653043 15675 rpcz_store.cc:276] 0709 13:19:58.634992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.820171 (+185179us) service_pool.cc:224] Handling call
0709 13:19:58.820410 (+   239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:59.652407 (+831997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:59.652613 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:58.832432 15676 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:58.816617 16203 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:19:59.656224 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 839643us
I20260709 13:19:59.656162 19414 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:58.903548 16233 raft_consensus.cc:1240] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:58.795032 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:58.819289 16236 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:19:59.667194 16236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1249) took 1053 ms. Trace:
I20260709 13:19:59.667546 16236 rpcz_store.cc:276] 0709 13:19:58.614243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.795527 (+181284us) service_pool.cc:224] Handling call
0709 13:19:58.795830 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:59.666918 (+871088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:59.667132 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:58.806173 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:59.669330 16239 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 8: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:19:58.815405 19128 raft_consensus.cc:697] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:19:59.672334 19128 consensus_queue.cc:237] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.845080 19274 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:59.676502 19274 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:19:58.779277 19282 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.678776 19282 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:59.681021 19282 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:58.777292 19295 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:59.684055 19295 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:58.782794 19304 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:58.818270 19408 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:58.792768 19409 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:59.688673 19409 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:19:59.689121 19409 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:58.839751 19410 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:58.829242 19413 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:58.828095 19418 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:58.825224 19438 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:58.888048 19447 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:58.779924 19448 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:19:58.819015 19459 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:58.888244 19466 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:58.783340 19456 raft_consensus.cc:697] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:19:59.702693 19456 consensus_queue.cc:237] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:58.821269 19484 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:58.857678 19506 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:58.777458 19508 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:59.711645 19508 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:19:58.746624 19509 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:58.972586 15525 raft_consensus.cc:2393] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 8: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:59.714712 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:58.973755 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:59.715694 15521 raft_consensus.cc:2393] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 8: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:19:59.716480 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:58.986733 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:19:59.005493 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.012894 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:59.719853 15527 raft_consensus.cc:2393] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 10: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:19:59.016589 19545 log.cc:927] Time spent T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:19:59.317382 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:59.722941 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:59.723904 16240 raft_consensus.cc:1240] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:59.724292 15598 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:19:59.318200 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:59.728204 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:19:59.319190 15603 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11. Ops: []
W20260709 13:19:59.333072 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.729699 16234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57244 (request call id 1255) took 1113 ms. Trace:
I20260709 13:19:59.720590 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:59.731007 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:59.731875 19484 consensus_queue.cc:1059] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:19:59.353219 15677 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.303s	user 0.009s	sys 0.008s
W20260709 13:19:59.732911 15677 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.683s	user 0.010s	sys 0.009s
I20260709 13:19:59.733625 15677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2284) took 1689 ms. Trace:
I20260709 13:19:59.366645 19513 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:19:59.738617 19513 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:59.739749 19459 consensus_queue.cc:1059] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:59.740640 19459 consensus_queue.cc:1059] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:59.741626 19459 raft_consensus.cc:3055] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:59.376857 15642 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:19:59.742271 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 365484us
I20260709 13:19:59.744915 19429 raft_consensus.cc:3055] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:59.745357 19429 raft_consensus.cc:740] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:19:59.746209 19429 consensus_queue.cc:260] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:59.747411 19429 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:59.747821 19504 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:59.747993 19368 consensus_queue.cc:1059] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:59.403383 15674 raft_consensus.cc:1275] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:19:59.750949 19410 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:59.751475 19410 raft_consensus.cc:697] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:19:59.752316 19410 consensus_queue.cc:237] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:19:59.424909 15566 leader_election.cc:400] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10.
I20260709 13:19:59.754065 15566 leader_election.cc:403] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:19:59.754558 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 329844us
I20260709 13:19:59.427908 19371 consensus_queue.cc:1059] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:59.429066 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:59.430721 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:59.435251 19425 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.447811 16205 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:59.474397 19090 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:59.467950 15414 maintenance_manager.cc:643] P 88b9585660f045bda650d7ffa0d7a0b7: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.821s	user 0.141s	sys 0.021s Metrics: {"bytes_written":29255,"cfile_init":1,"dirs.queue_time_us":7413,"dirs.run_cpu_time_us":395,"dirs.run_wall_time_us":7573,"drs_written":1,"lbm_read_time_us":397,"lbm_reads_lt_1ms":4,"lbm_write_time_us":4149,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"rows_written":63,"thread_start_us":14059,"threads_started":2,"wal-append.queue_time_us":6064}
I20260709 13:19:58.780155 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:59.499827 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:19:59.498584 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:59.515697 19512 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:59.520205 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:59.553973 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:19:59.566974 15493 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:57310 dropped due to backpressure: service queue is full with 50 items [suppressed 5 similar messages]
W20260709 13:19:59.589028 16237 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.019s	sys 0.014s
I20260709 13:19:59.605386 15595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 2155) took 1166 ms. Trace:
I20260709 13:19:59.593672 15526 pending_rounds.cc:107] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:19:59.597623 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:19:59.601835 15594 raft_consensus.cc:2376] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9. Re-sending same reply.
I20260709 13:19:59.606294 19505 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.624130 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:19:59.627703 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:19:59.648608 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 739903us
I20260709 13:19:59.654798 15676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 1123) took 1015 ms. Trace:
I20260709 13:19:59.658897 19427 consensus_queue.cc:1059] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:59.662962 16233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57258 (request call id 834) took 1005 ms. Trace:
I20260709 13:19:59.664306 16203 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:19:59.707237 16231 raft_consensus.cc:1240] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:59.736871 15521 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:59.730154 16234 rpcz_store.cc:276] 0709 13:19:58.615842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.794804 (+178962us) service_pool.cc:224] Handling call
0709 13:19:59.729627 (+934823us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:59.737586 19509 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:19:59.975961 15523 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:19:59.761359 15600 raft_consensus.cc:2376] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8. Re-sending same reply.
I20260709 13:19:59.975241 15599 raft_consensus.cc:2376] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9. Re-sending same reply.
I20260709 13:19:59.749686 19504 raft_consensus.cc:3055] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:59.733973 15677 rpcz_store.cc:276] 0709 13:19:58.044728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.048885 (+  4157us) service_pool.cc:224] Handling call
0709 13:19:58.049167 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:58.049391 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:58.049639 (+   248us) log.cc:844] Serialized 14 byte log entry
0709 13:19:59.733323 (+1683684us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:59.733374 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:59.733422 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:59.733481 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:59.733576 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":764,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":17383,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":81,"thread_start_us":1683369,"threads_started":1,"wal-append.queue_time_us":567219,"wal-append.run_cpu_time_us":5790,"wal-append.run_wall_time_us":1005221}
I20260709 13:19:59.817257 15526 raft_consensus.cc:2889] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:59.774392 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:19:59.983642 15527 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:19:59.984233 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1954) took 1222 ms. Trace:
I20260709 13:19:59.984720 15527 rpcz_store.cc:276] 0709 13:19:58.761436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.774046 (+1012610us) service_pool.cc:224] Handling call
0709 13:19:59.984178 (+210132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:59.818118 15528 raft_consensus.cc:2376] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8. Re-sending same reply.
W20260709 13:19:59.817093 16237 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.021s	sys 0.019s
I20260709 13:19:59.755978 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:19:59.986332 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1960) took 1223 ms. Trace:
I20260709 13:19:59.987864 15528 rpcz_store.cc:276] 0709 13:19:58.763084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.597364 (+834280us) service_pool.cc:224] Handling call
0709 13:19:59.986273 (+388909us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:59.988376 15601 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
W20260709 13:19:59.976034 15641 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.647s (SENT) [suppressed 24 similar messages]
I20260709 13:19:59.977451 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1962) took 1213 ms. Trace:
I20260709 13:19:59.989787 15521 rpcz_store.cc:276] 0709 13:19:58.763525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.716244 (+952719us) service_pool.cc:224] Handling call
0709 13:19:59.977393 (+261149us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:59.827001 15567 leader_election.cc:336] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.587s (SENT)
I20260709 13:19:59.990754 15567 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:19:59.978776 16233 rpcz_store.cc:276] 0709 13:19:58.657289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.901877 (+244588us) service_pool.cc:224] Handling call
0709 13:19:58.903362 (+  1485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:59.662673 (+759311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:59.662898 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:59.991698 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 164753us
I20260709 13:19:59.979357 15676 rpcz_store.cc:276] 0709 13:19:58.639286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.820956 (+181670us) service_pool.cc:224] Handling call
0709 13:19:58.821244 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:59.654537 (+833293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:59.654735 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:59.985445 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1956) took 1223 ms. Trace:
I20260709 13:19:59.769907 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1207) took 1242 ms. Trace:
I20260709 13:19:59.817596 15595 rpcz_store.cc:276] 0709 13:19:58.439242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.656867 (+217625us) service_pool.cc:224] Handling call
0709 13:19:58.871478 (+214611us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:59.604037 (+732559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:59.605182 (+  1145us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b540032c3d0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:19:59.605325 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":492853504}
I20260709 13:19:59.993049 15523 rpcz_store.cc:276] 0709 13:19:58.761888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.519956 (+758068us) service_pool.cc:224] Handling call
0709 13:19:59.985379 (+465423us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:59.993294 15522 rpcz_store.cc:276] 0709 13:19:58.520852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.571712 (+ 50860us) service_pool.cc:224] Handling call
0709 13:19:58.591402 (+ 19690us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:58.591633 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:58.591656 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:58.622421 (+ 30765us) log.cc:844] Serialized 26 byte log entry
0709 13:19:58.786447 (+164026us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:58.786489 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.786554 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:59.016087 (+229533us) spinlock_profiling.cc:243] Waited 3.2 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:19:59.729832 (+713745us) raft_consensus.cc:1685] finished
0709 13:19:59.729852 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:59.763421 (+ 33569us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7349632,"thread_start_us":163672,"threads_started":1,"wal-append.queue_time_us":260239}
I20260709 13:19:59.987200 16237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1258) took 1318 ms. Trace:
I20260709 13:20:00.001502 15520 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
W20260709 13:20:00.001673 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 553925us
I20260709 13:20:00.054800 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1943) took 1298 ms. Trace:
I20260709 13:20:00.055217 15520 rpcz_store.cc:276] 0709 13:19:58.756191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.758915 (+  2724us) service_pool.cc:224] Handling call
0709 13:20:00.054677 (+1295762us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:59.975399 15519 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:00.058509 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1951) took 1298 ms. Trace:
I20260709 13:20:00.058974 15519 rpcz_store.cc:276] 0709 13:19:58.760638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.498338 (+737700us) service_pool.cc:224] Handling call
0709 13:20:00.058446 (+560108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:59.764510 19460 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.216s
I20260709 13:20:00.062647 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:00.063236 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:00.063236 15415 maintenance_manager.cc:419] P 88b9585660f045bda650d7ffa0d7a0b7: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10842 bytes on disk
I20260709 13:20:00.063416 15527 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:00.064533 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1968) took 1271 ms. Trace:
I20260709 13:20:00.064255 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:00.061911 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 317416us
I20260709 13:20:00.065807 15527 rpcz_store.cc:276] 0709 13:19:58.793542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.062311 (+1268769us) service_pool.cc:224] Handling call
0709 13:20:00.064468 (+  2157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.071175 15523 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:20:00.072331 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1953) took 1311 ms. Trace:
I20260709 13:20:00.072655 15523 rpcz_store.cc:276] 0709 13:19:58.761174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.064008 (+1302834us) service_pool.cc:224] Handling call
0709 13:20:00.072265 (+  8257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.063897 15521 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:59.755781 19459 raft_consensus.cc:740] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:00.097839 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1205) took 1577 ms. Trace:
I20260709 13:20:00.099874 16231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1257) took 1468 ms. Trace:
I20260709 13:20:00.102583 16231 rpcz_store.cc:276] 0709 13:19:58.631653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.818273 (+186620us) service_pool.cc:224] Handling call
0709 13:19:58.834692 (+ 16419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:59.977803 (+1143111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.099584 (+121781us) spinlock_profiling.cc:243] Waited 872 ms on lock 0x7b54005f0354. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:00.099798 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2006193920}
I20260709 13:19:59.765367 19408 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10.
W20260709 13:20:00.107080 15549 debug-util.cc:398] Leaking SignalData structure 0x7b08000dbb80 after lost signal to thread 14432
W20260709 13:20:00.108647 15549 debug-util.cc:398] Leaking SignalData structure 0x7b080038e580 after lost signal to thread 15460
W20260709 13:20:00.110107 15549 debug-util.cc:398] Leaking SignalData structure 0x7b0800376080 after lost signal to thread 15552
I20260709 13:20:00.101219 15526 rpcz_store.cc:276] 0709 13:19:58.520354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.522920 (+  2566us) service_pool.cc:224] Handling call
0709 13:19:58.791345 (+268425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:59.983042 (+1191697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.097578 (+114536us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b54003ce6d0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:00.097757 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":572846848}
I20260709 13:20:00.107452 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:00.112563 15528 raft_consensus.cc:2376] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6. Re-sending same reply.
W20260709 13:20:00.112792 15549 debug-util.cc:398] Leaking SignalData structure 0x7b08002d9420 after lost signal to thread 15627
I20260709 13:20:00.113067 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1963) took 1349 ms. Trace:
W20260709 13:20:00.113770 15549 debug-util.cc:398] Leaking SignalData structure 0x7b08002be520 after lost signal to thread 15628
I20260709 13:20:00.113729 15528 rpcz_store.cc:276] 0709 13:19:58.763740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.063615 (+1299875us) service_pool.cc:224] Handling call
0709 13:20:00.113007 (+ 49392us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:00.115085 15549 debug-util.cc:398] Leaking SignalData structure 0x7b080003adc0 after lost signal to thread 15701
W20260709 13:20:00.115409 15549 debug-util.cc:398] Leaking SignalData structure 0x7b0800134b80 after lost signal to thread 15702
W20260709 13:20:00.116777 15549 debug-util.cc:398] Leaking SignalData structure 0x7b08003a5620 after lost signal to thread 16266
I20260709 13:19:59.816879 19447 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:59.839097 19462 raft_consensus.cc:3055] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:59.772720 19508 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:00.070794 19428 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:59.986323 19456 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:00.001185 19504 raft_consensus.cc:740] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:00.000192 19418 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.768991 19210 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.338s
I20260709 13:19:59.816867 19425 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:00.064495 19509 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:00.070473 19413 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:19:59.981057 19512 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:19:59.768826 19506 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.362s
I20260709 13:19:59.764758 19274 consensus_queue.cc:1059] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.366s
I20260709 13:20:00.137095 19462 raft_consensus.cc:740] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:00.057065 19505 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.000662 19090 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:00.174585 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:00.168390 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 504140us
I20260709 13:20:00.142413 19448 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 7. Current term is 8.
I20260709 13:20:00.197945 16203 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:00.156833 19128 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:00.203577 16203 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:00.139591 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 147130us
I20260709 13:20:00.132661 15596 raft_consensus.cc:1240] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 8. Current term is 9. Ops: []
W20260709 13:20:00.502518 15567 leader_election.cc:336] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.623s (SENT)
I20260709 13:20:00.502995 15596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 998) took 1047 ms. Trace:
I20260709 13:20:00.503571 15596 rpcz_store.cc:276] 0709 13:19:59.455781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.066266 (+610485us) service_pool.cc:224] Handling call
0709 13:20:00.132465 (+ 66199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.502166 (+369701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.502700 (+   534us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.270516 19371 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:20:00.455386 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 479219us
W20260709 13:20:00.506636 15641 leader_election.cc:336] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.743s (SENT)
I20260709 13:20:00.507287 15641 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:00.435173 15524 raft_consensus.cc:2410] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:20:00.266726 19304 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.006s	sys 0.000s
W20260709 13:20:00.453722 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:00.450382 15566 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.006s	sys 0.005s
I20260709 13:20:00.507539 19090 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:00.141459 19508 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:00.515105 15566 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.008s	sys 0.007s
W20260709 13:20:00.515725 19304 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.007s	sys 0.001s
I20260709 13:20:00.193996 19409 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.194442 19413 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.161115 19418 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:00.286192 19462 consensus_queue.cc:260] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.254266 19504 consensus_queue.cc:260] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.524154 19504 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:00.327028 19505 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:00.525761 19506 consensus_queue.cc:1059] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:20:00.518620 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 755817us
W20260709 13:20:00.527278 15412 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 582343936346420b87ecf2eebd00dc25 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:20:00.528040 15566 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:00.263492 19560 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.031s
W20260709 13:20:00.354993 19513 consensus_meta.cc:287] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.616s	user 0.005s	sys 0.004s
I20260709 13:20:00.529527 19506 raft_consensus.cc:3055] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:00.440913 15525 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
W20260709 13:20:00.404572 15426 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49140 (request call id 361) took 20738 ms (client timeout 15000 ms). Trace:
I20260709 13:20:00.530089 19506 raft_consensus.cc:740] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:00.405517 15424 catalog_manager.cc:5697] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:00.533879 15424 catalog_manager.cc:5697] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 6 to 7, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 7 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:00.536541 19413 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:00.267159 19438 consensus_queue.cc:1059] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:20:00.517881 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:00.175671 19456 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.539110 15526 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:00.540608 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:00.541172 19456 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:00.193776 19459 consensus_queue.cc:260] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.530102 19513 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.544270 19418 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:00.546006 19408 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:00.531761 19506 consensus_queue.cc:260] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.547959 19428 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:20:00.549029 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 345405us
I20260709 13:20:00.550796 16203 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:00.549427 19506 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:00.553751 19460 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:00.555269 19505 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:00.555909 19579 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.556761 19579 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:00.524140 15672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2308) took 1697 ms. Trace:
I20260709 13:20:00.558821 19090 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:00.559144 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:00.561226 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:00.549208 19428 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:00.562314 19428 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:00.562847 19428 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:00.563679 19409 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:00.545545 19513 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:00.531003 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:00.530390 15426 rpcz_store.cc:269] 0709 13:19:39.665953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.667542 (+  1589us) service_pool.cc:224] Handling call
0709 13:20:00.404085 (+20736543us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:51.415903 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:51.429308 (+ 13405us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:51.429357 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:19:51.429376 (+    19us) tablet.cc:662] Decoding operations
0709 13:19:51.443005 (+ 13629us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:51.443087 (+    82us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:51.443115 (+    28us) tablet.cc:685] Acquiring locks for 24 operations
0709 13:19:51.445184 (+  2069us) tablet.cc:701] Row locks acquired
0709 13:19:51.445231 (+    47us) write_op.cc:260] PREPARE: finished
0709 13:19:51.445416 (+   185us) write_op.cc:270] Start()
0709 13:19:51.445565 (+   149us) write_op.cc:276] Timestamp: P: 1783603191445352 usec, L: 0
0709 13:19:51.445587 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:19:51.501518 (+ 55931us) log.cc:844] Serialized 9481 byte log entry
0709 13:19:52.373835 (+872317us) op_driver.cc:464] REPLICATION: finished
0709 13:19:52.497032 (+123197us) write_op.cc:301] APPLY: starting
0709 13:19:52.497149 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:52.558553 (+ 61404us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:52.558708 (+   155us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:52.583040 (+ 24332us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:52.583480 (+   440us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=24
0709 13:19:52.583527 (+    47us) write_op.cc:312] APPLY: finished
0709 13:19:52.584053 (+   526us) log.cc:844] Serialized 164 byte log entry
0709 13:19:52.669755 (+ 85702us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:52.754341 (+ 84586us) write_op.cc:454] Released schema lock
0709 13:19:53.056746 (+302405us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:53.961329 (+904583us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:53.964145 (+  2816us) write_op.cc:454] Released schema lock
0709 13:19:58.824180 (+4860035us) spinlock_profiling.cc:243] Waited 2.72 ms on lock 0x7b180011e780. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce191e522 00007f5ce19207c1 00007f5ce1920779 00007f5ce1920709 00007f5ce19206d1 00007f5ce191f9cd 000000000038f354 000000000038f199 00007f5ce190183a
Metrics: {"mutex_wait_us":839932,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":119541,"apply.run_cpu_time_us":6171,"apply.run_wall_time_us":847575,"num_ops":24,"prepare.queue_time_us":213223,"prepare.run_cpu_time_us":28883,"prepare.run_wall_time_us":251229,"raft.queue_time_us":205998,"raft.run_cpu_time_us":10282,"raft.run_wall_time_us":153884,"replication_time_us":907777,"spinlock_wait_cycles":6325248,"thread_start_us":377053,"threads_started":4,"wal-append.queue_time_us":165964,"wal-append.run_cpu_time_us":16029,"wal-append.run_wall_time_us":2289669}]]}
I20260709 13:20:00.194865 19473 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:19:59.817862 15493 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:57310 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1999)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2012)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2013)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2010)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56840 (request call id 1214)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1996)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1997)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2005)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2006)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1994)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2007)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1995)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2009)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2002)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2015)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2001)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1969)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1967)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2008)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2014)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1970)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1946)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1947)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1208)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1209)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1210)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1211)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1212)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1213)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1215)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1972)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1973)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1974)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1975)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1998)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2000)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2003)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2004)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2011)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1216)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1217)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1218)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1219)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1220)
I20260709 13:20:00.191306 19509 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.575229 19509 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:00.476723 19580 log.cc:927] Time spent T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:20:00.575958 15602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 995) took 1254 ms. Trace:
I20260709 13:20:00.477779 19425 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:00.255450 19274 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.433528 15521 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:20:00.577394 15566 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:00.493105 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:00.495265 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:00.509598 15641 leader_election.cc:336] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.794s (SENT)
I20260709 13:20:00.142390 19447 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.252046 19566 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.491986 19210 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.526569 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:00.350651 19512 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:00.321391 19576 raft_consensus.cc:3055] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:00.550639 19128 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:00.515259 19371 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.267004 19572 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:00.558101 15672 rpcz_store.cc:276] 0709 13:19:58.826660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.831290 (+  4630us) service_pool.cc:224] Handling call
0709 13:19:58.831635 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:59.687378 (+855743us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:59.816940 (+129562us) log.cc:844] Serialized 14 byte log entry
0709 13:20:00.139080 (+322140us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:00.139148 (+    68us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:00.139208 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.523486 (+384278us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b540046df54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:00.523567 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.523851 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":543841,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":5162,"raft.queue_time_us":1876,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":11351,"spinlock_wait_cycles":1967541504,"thread_start_us":450897,"threads_started":2,"wal-append.queue_time_us":343496}
I20260709 13:20:00.559607 19090 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.565200 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1996) took 1027 ms. Trace:
I20260709 13:20:00.541577 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1997) took 1003 ms. Trace:
I20260709 13:20:00.560655 19560 raft_consensus.cc:3055] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:20:00.571209 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 56328.1us
I20260709 13:20:00.537425 15424 catalog_manager.cc:5697] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:00.591099 19410 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.604269 19560 raft_consensus.cc:740] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:00.604449 19576 raft_consensus.cc:740] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:00.605175 15594 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:00.581122 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:00.606323 16239 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:00.606814 15523 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
W20260709 13:20:00.607391 15523 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1950) took 1849 ms (client timeout 2332 ms). Trace:
W20260709 13:20:00.607785 15523 rpcz_store.cc:269] 0709 13:19:58.759951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.494662 (+1734711us) service_pool.cc:224] Handling call
0709 13:20:00.607317 (+112655us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.609622 15522 raft_consensus.cc:2410] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:20:00.584422 19425 consensus_queue.cc:1059] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:20:00.612304 19090 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:00.579782 19571 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:00.612691 19410 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:00.548127 19508 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:00.611173 19505 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:00.581684 15527 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:00.614646 19566 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:00.577329 19274 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:00.610317 19210 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.606365 15424 catalog_manager.cc:5697] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:00.616984 19566 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:00.617417 19274 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:00.617594 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 67023.9us
I20260709 13:20:00.610140 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2013) took 1068 ms. Trace:
I20260709 13:20:00.613670 19402 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:20:00.578552 19581 consensus_queue.cc:1059] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:00.577415 19282 raft_consensus.cc:3055] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:00.619089 19282 raft_consensus.cc:740] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:00.614136 15526 rpcz_store.cc:276] 0709 13:19:59.537798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.540162 (+1002364us) service_pool.cc:224] Handling call
0709 13:20:00.541499 (+  1337us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.620321 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:00.620464 19282 consensus_queue.cc:260] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:00.608959 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:00.604166 15528 rpcz_store.cc:276] 0709 13:19:59.537556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.530460 (+992904us) service_pool.cc:224] Handling call
0709 13:20:00.565148 (+ 34688us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:00.621860 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 53218.2us
I20260709 13:20:00.622902 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:00.623308 16205 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:00.583989 15493 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:00.404902 15414 maintenance_manager.cc:643] P 88b9585660f045bda650d7ffa0d7a0b7: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.340s	user 0.003s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":811,"lbm_reads_lt_1ms":4}
I20260709 13:20:00.618534 19409 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
W20260709 13:20:00.603961 15521 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1964) took 1840 ms (client timeout 2217 ms). Trace:
I20260709 13:20:00.601476 19579 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:00.621762 15523 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:20:00.626902 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2005) took 1087 ms. Trace:
I20260709 13:20:00.627287 15523 rpcz_store.cc:276] 0709 13:19:59.539782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.608501 (+1068719us) service_pool.cc:224] Handling call
0709 13:20:00.626821 (+ 18320us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.617973 19210 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:00.617106 15424 catalog_manager.cc:5697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 8 to 9, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 9 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:20:00.625748 15521 rpcz_store.cc:269] 0709 13:19:58.763962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.062962 (+1299000us) service_pool.cc:224] Handling call
0709 13:20:00.603854 (+540892us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.628925 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:00.615228 19410 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:00.630255 15521 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:20:00.631048 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2007) took 1090 ms. Trace:
I20260709 13:20:00.631407 15521 rpcz_store.cc:276] 0709 13:19:59.540254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.628589 (+1088335us) service_pool.cc:224] Handling call
0709 13:20:00.630871 (+  2282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.632229 19445 raft_consensus.cc:3055] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:00.639835 19445 raft_consensus.cc:740] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:00.641088 19445 consensus_queue.cc:260] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.642235 19445 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:00.642993 19447 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:00.643591 19447 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.644212 19447 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:00.054059 16237 rpcz_store.cc:276] 0709 13:19:58.668949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.917487 (+248538us) service_pool.cc:224] Handling call
0709 13:19:58.917985 (+   498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:58.918231 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:58.935453 (+ 17222us) log.cc:844] Serialized 14 byte log entry
0709 13:19:59.986845 (+1051392us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:59.986902 (+    57us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:59.986951 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:59.987016 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:59.987130 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":31945,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":14021,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":20707,"spinlock_wait_cycles":142381952,"thread_start_us":1067866,"threads_started":2,"wal-append.queue_time_us":47289,"wal-append.run_wall_time_us":1017906}
I20260709 13:20:00.633776 19402 raft_consensus.cc:3055] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:00.641104 19428 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.632444 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:00.634424 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:00.629245 15424 catalog_manager.cc:5697] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:00.638056 15493 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:00.639117 19460 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:00.649034 19418 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:00.645725 19402 raft_consensus.cc:740] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:00.650619 16236 raft_consensus.cc:1240] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:00.611492 15641 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:00.618673 16203 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:00.652195 19304 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:20:00.652521 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 143025us
I20260709 13:20:00.653297 19128 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:00.576447 15602 rpcz_store.cc:276] 0709 13:19:59.321441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.726015 (+404574us) service_pool.cc:224] Handling call
0709 13:19:59.726315 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:59.757594 (+ 31279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:59.757621 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:59.765653 (+  8032us) log.cc:844] Serialized 26 byte log entry
0709 13:20:00.544661 (+779008us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:00.545233 (+   572us) log.cc:844] Serialized 14 byte log entry
0709 13:20:00.545566 (+   333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.545757 (+   191us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:00.575560 (+ 29803us) raft_consensus.cc:1685] finished
0709 13:20:00.575576 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.575760 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29071,"prepare.queue_time_us":1089,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":36248,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":52550,"spinlock_wait_cycles":177408,"thread_start_us":434836,"threads_started":1,"wal-append.queue_time_us":590201}
I20260709 13:20:00.612938 19425 raft_consensus.cc:3055] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:00.609002 19371 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:00.651057 19402 consensus_queue.cc:260] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.620983 15526 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:20:00.656924 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2006) took 1116 ms. Trace:
I20260709 13:20:00.656893 19402 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:00.605839 19560 consensus_queue.cc:260] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.654865 19425 raft_consensus.cc:740] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:00.647037 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1995) took 1109 ms. Trace:
I20260709 13:20:00.605839 19576 consensus_queue.cc:260] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:00.623868 15528 raft_consensus.cc:2410] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:20:00.683022 19409 consensus_queue.cc:1175] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" responder_term: 11 status { last_received { term: 10 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:20:00.682960 19425 consensus_queue.cc:260] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.684546 19576 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:00.685443 19425 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:00.686517 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:00.687388 15520 raft_consensus.cc:2410] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:20:00.683563 15521 rpcz_store.cc:276] 0709 13:19:59.537322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.632087 (+1094765us) service_pool.cc:224] Handling call
0709 13:20:00.646961 (+ 14874us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:00.687800 15520 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1999) took 1149 ms (client timeout 1504 ms). Trace:
W20260709 13:20:00.688175 15520 rpcz_store.cc:269] 0709 13:19:59.538332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.686076 (+1147744us) service_pool.cc:224] Handling call
0709 13:20:00.687747 (+  1671us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.688576 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:00.688989 15491 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:00.689231 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:00.684132 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 1994) took 1147 ms. Trace:
I20260709 13:20:00.690415 15528 rpcz_store.cc:276] 0709 13:19:59.537072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.622486 (+1085414us) service_pool.cc:224] Handling call
0709 13:20:00.684066 (+ 61580us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.691399 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:00.692214 15528 raft_consensus.cc:2393] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 12: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:00.692647 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2001) took 1153 ms. Trace:
I20260709 13:20:00.692961 15528 rpcz_store.cc:276] 0709 13:19:59.538826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.691061 (+1152235us) service_pool.cc:224] Handling call
0709 13:20:00.692593 (+  1532us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.660575 16237 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:20:00.689358 15521 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:20:00.661768 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:00.694867 19462 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:00.647025 15523 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:00.695673 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2009) took 1154 ms. Trace:
I20260709 13:20:00.696056 15523 rpcz_store.cc:276] 0709 13:19:59.540738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.634056 (+1093318us) service_pool.cc:224] Handling call
0709 13:20:00.695600 (+ 61544us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.557866 15642 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:00.697224 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:00.698268 19466 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:00.698268 15523 raft_consensus.cc:2368] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9.
I20260709 13:20:00.699396 19459 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:00.691344 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:00.649377 15424 catalog_manager.cc:5697] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 3 to 8, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 8 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:00.667019 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:00.667649 19304 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:00.667158 19371 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:00.670644 15525 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1961) took 1907 ms (client timeout 1927 ms). Trace:
W20260709 13:20:00.671082 15524 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1965) took 1906 ms (client timeout 1595 ms). Trace:
W20260709 13:20:00.614001 15567 leader_election.cc:336] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.753s (SENT)
I20260709 13:20:00.675096 15568 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:00.676120 19438 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:00.677131 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 53987.5us
I20260709 13:20:00.678172 16203 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:00.621726 19282 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:00.717294 15519 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:20:00.618232 15522 rpcz_store.cc:276] 0709 13:19:59.541715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.492278 (+950563us) service_pool.cc:224] Handling call
0709 13:20:00.610075 (+117797us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.719534 15527 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
W20260709 13:20:00.715219 15525 rpcz_store.cc:269] 0709 13:19:58.763308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.714450 (+951142us) service_pool.cc:224] Handling call
0709 13:20:00.670341 (+955891us) spinlock_profiling.cc:243] Waited 725 ms on lock 0x7b54003cd2d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:00.670547 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1667529088}
I20260709 13:20:00.657399 15526 rpcz_store.cc:276] 0709 13:19:59.540026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.619934 (+1079908us) service_pool.cc:224] Handling call
0709 13:20:00.656863 (+ 36929us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.689953 19509 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:00.694307 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2002) took 1155 ms. Trace:
I20260709 13:20:00.721537 15521 rpcz_store.cc:276] 0709 13:19:59.539065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.688179 (+1149114us) service_pool.cc:224] Handling call
0709 13:20:00.694234 (+  6055us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.657035 19428 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:00.708755 15594 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:20:00.710747 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:00.692281 19429 consensus_meta.cc:287] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.944s	user 0.010s	sys 0.009s
W20260709 13:20:00.715308 15524 rpcz_store.cc:269] 0709 13:19:58.764167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.553677 (+789510us) service_pool.cc:224] Handling call
0709 13:20:00.670827 (+1117150us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b54003cebd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:00.670994 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1417750912}
W20260709 13:20:00.717054 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 105046us
I20260709 13:20:00.719220 15520 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 10: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:20:00.719672 15519 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2012) took 1178 ms (client timeout 1515 ms). Trace:
I20260709 13:20:00.699390 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1967) took 1934 ms. Trace:
W20260709 13:20:00.723623 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 48686.2us
I20260709 13:20:00.728081 15523 rpcz_store.cc:276] 0709 13:19:58.764597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.696817 (+1932220us) service_pool.cc:224] Handling call
0709 13:20:00.699331 (+  2514us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:00.720189 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 47487.6us
I20260709 13:20:00.722823 19509 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:00.740157 19509 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:00.726086 15424 catalog_manager.cc:5697] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 6 to 7, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 7 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:00.739571 16239 raft_consensus.cc:1240] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8. Ops: []
W20260709 13:20:00.745359 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 187854us
I20260709 13:20:00.746166 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:00.746814 15642 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:00.735916 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:00.748147 15524 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:20:00.749181 15672 raft_consensus.cc:1275] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:20:00.749279 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:00.750512 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:00.750725 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:00.750957 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:00.724360 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1969) took 1189 ms. Trace:
I20260709 13:20:00.750164 19410 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:00.751636 15528 rpcz_store.cc:276] 0709 13:19:59.535190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.693639 (+1158449us) service_pool.cc:224] Handling call
0709 13:20:00.724306 (+ 30667us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.751360 15671 raft_consensus.cc:2393] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 8: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:00.739831 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:00.753309 15526 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:00.752317 15676 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:00.754070 16205 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:00.753448 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:00.747242 15525 raft_consensus.cc:2393] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 8: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:00.754998 15528 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:00.755602 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:00.756404 16205 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:00.756598 19428 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:00.757059 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:00.757697 15526 raft_consensus.cc:1813] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Handling vote request from an unknown peer 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:00.758313 15526 raft_consensus.cc:2410] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:20:00.757571 15424 catalog_manager.cc:5697] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 6 to 7, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 7 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:00.759740 15524 raft_consensus.cc:1240] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:00.724970 19454 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:00.760378 15524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1946) took 2003 ms. Trace:
I20260709 13:20:00.735322 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:00.760649 19454 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:00.761271 19454 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:00.729640 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2015) took 1163 ms. Trace:
I20260709 13:20:00.766659 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:00.766812 15524 rpcz_store.cc:276] 0709 13:19:58.757025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.759161 (+2002136us) service_pool.cc:224] Handling call
0709 13:20:00.759563 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.760100 (+   537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.760282 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.766587 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:00.723076 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56840 (request call id 1214) took 1395 ms. Trace:
W20260709 13:20:00.751634 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 137745us
I20260709 13:20:00.767696 15520 rpcz_store.cc:276] 0709 13:19:59.566396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.688911 (+1122515us) service_pool.cc:224] Handling call
0709 13:20:00.729362 (+ 40451us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.750111 19371 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.751165 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:00.752234 19512 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:20:00.727572 15519 rpcz_store.cc:269] 0709 13:19:59.541469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.666558 (+1125089us) service_pool.cc:224] Handling call
0709 13:20:00.719608 (+ 53050us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.761327 15523 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:20:00.772395 15677 raft_consensus.cc:2410] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:20:00.729717 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:00.768218 15527 rpcz_store.cc:276] 0709 13:19:59.325209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.529717 (+1204508us) service_pool.cc:224] Handling call
0709 13:20:00.720503 (+190786us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.772802 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 1970) took 1237 ms. Trace:
I20260709 13:20:00.773288 15523 rpcz_store.cc:276] 0709 13:19:59.535561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.734950 (+1199389us) service_pool.cc:224] Handling call
0709 13:20:00.772749 (+ 37799us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.774377 15528 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
W20260709 13:20:00.775731 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 97667.5us
I20260709 13:20:00.775485 19414 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:00.785173 15522 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:20:00.787308 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2008) took 1246 ms. Trace:
I20260709 13:20:00.787503 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2014) took 1245 ms. Trace:
I20260709 13:20:00.787667 15522 rpcz_store.cc:276] 0709 13:19:59.540502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.720157 (+1179655us) service_pool.cc:224] Handling call
0709 13:20:00.787247 (+ 67090us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.788321 15521 rpcz_store.cc:276] 0709 13:19:59.565971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.722240 (+1156269us) service_pool.cc:224] Handling call
0709 13:20:00.787444 (+ 65204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.788641 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:00.788347 15526 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:20:00.790612 15677 raft_consensus.cc:1275] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:20:00.791903 15519 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:00.797569 19445 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:00.794802 19447 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:00.804216 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:00.804965 15677 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:00.773478 19304 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.807137 19304 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:00.809514 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1215) took 1484 ms. Trace:
W20260709 13:20:00.795236 15567 leader_election.cc:336] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.807s (SENT)
I20260709 13:20:00.810442 15526 rpcz_store.cc:276] 0709 13:19:59.325359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.796170 (+1470811us) service_pool.cc:224] Handling call
0709 13:20:00.808758 (+ 12588us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.809124 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:00.809165 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:00.809209 (+    44us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:00.809247 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.809303 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.809421 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.799592 15424 catalog_manager.cc:5697] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:00.794760 19505 consensus_queue.cc:1059] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:20:00.815971 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1212) took 1491 ms. Trace:
I20260709 13:20:00.816390 15522 rpcz_store.cc:276] 0709 13:19:59.324902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.788765 (+1463863us) service_pool.cc:224] Handling call
0709 13:20:00.815363 (+ 26598us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.815591 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:00.815622 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:00.815693 (+    71us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:00.815738 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.815800 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.815910 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.817415 19445 raft_consensus.cc:697] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:00.818742 19585 raft_consensus.cc:3055] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:00.819151 19585 raft_consensus.cc:740] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:00.819005 19445 consensus_queue.cc:237] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.820344 19585 consensus_queue.cc:260] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.827754 19409 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:00.829970 19414 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:00.834498 19371 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:00.836395 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:00.813953 15567 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:00.839537 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 48345.2us
I20260709 13:20:00.815002 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1211) took 1486 ms. Trace:
I20260709 13:20:00.840739 15528 rpcz_store.cc:276] 0709 13:19:59.324719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.785732 (+1461013us) service_pool.cc:224] Handling call
0709 13:20:00.794420 (+  8688us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.801761 (+  7341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:00.801787 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:00.811018 (+  9231us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:00.811063 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.811120 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.811238 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.847007 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:00.848531 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:00.840770 19585 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:00.850102 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:00.850853 15675 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:00.851856 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:00.852586 15671 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:20:00.853298 16204 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:00.854022 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 66024.5us
I20260709 13:20:00.855002 19414 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:00.855654 19414 raft_consensus.cc:697] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:00.856630 19414 consensus_queue.cc:237] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.846915 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:00.827618 19560 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:00.859496 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1210) took 1521 ms. Trace:
I20260709 13:20:00.860833 15676 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:00.859695 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1209) took 1533 ms. Trace:
I20260709 13:20:00.844776 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1213) took 1519 ms. Trace:
W20260709 13:20:00.863015 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 106694us
I20260709 13:20:00.860082 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1947) took 2089 ms. Trace:
I20260709 13:20:00.860111 15527 rpcz_store.cc:276] 0709 13:19:59.324566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.783136 (+1458570us) service_pool.cc:224] Handling call
0709 13:20:00.790152 (+  7016us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.841850 (+ 51698us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:00.841879 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:00.841944 (+    65us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:00.845371 (+  3427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.845438 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.845545 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.861949 15523 rpcz_store.cc:276] 0709 13:19:59.324377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.785101 (+1460724us) service_pool.cc:224] Handling call
0709 13:20:00.802904 (+ 17803us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.857740 (+ 54836us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:00.857768 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:00.857811 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:00.857848 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.857959 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.858066 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.863839 15520 rpcz_store.cc:276] 0709 13:19:58.757261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.768924 (+2011663us) service_pool.cc:224] Handling call
0709 13:20:00.794109 (+ 25185us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.846075 (+ 51966us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:00.846102 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:00.846146 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:00.846193 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.846248 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.846347 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:00.866990 15641 leader_election.cc:336] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.003s (SENT)
I20260709 13:20:00.867252 15674 raft_consensus.cc:1240] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:00.867658 15672 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:20:00.868242 19371 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:00.863142 15521 rpcz_store.cc:276] 0709 13:19:59.325061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.790001 (+1464940us) service_pool.cc:224] Handling call
0709 13:20:00.815043 (+ 25042us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.841805 (+ 26762us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:00.841833 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:00.841867 (+    34us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:00.841888 (+    21us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.841952 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.844714 (+  2762us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.872339 15424 catalog_manager.cc:5697] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:20:00.884135 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.888644 15677 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:00.866886 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:00.890314 15673 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:20:00.896256 19466 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:00.897073 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.859324 19504 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:00.909621 16205 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:00.911355 19445 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:00.912006 19445 raft_consensus.cc:697] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:20:00.913906 15567 leader_election.cc:336] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.927s (SENT)
W20260709 13:20:00.915077 15567 leader_election.cc:336] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.927s (SENT)
W20260709 13:20:00.916303 15567 leader_election.cc:336] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.930s (SENT)
I20260709 13:20:00.916714 15567 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:00.918069 19371 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.923941 19371 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:00.925137 15528 raft_consensus.cc:1275] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:20:00.925287 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:00.926218 15670 raft_consensus.cc:2393] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 8: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:00.926714 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:00.927197 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:00.927568 15673 raft_consensus.cc:2410] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:20:00.928310 15670 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:00.928907 15673 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:00.929001 15602 raft_consensus.cc:1240] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:20:00.931020 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:00.931762 15669 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:20:00.933961 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:00.934973 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:00.935487 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:00.936781 15670 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:20:00.926158 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1974) took 1389 ms. Trace:
I20260709 13:20:00.937479 15669 raft_consensus.cc:1275] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:20:00.913209 19445 consensus_queue.cc:237] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.883855 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:00.941524 15528 rpcz_store.cc:276] 0709 13:19:59.536573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.873581 (+1337008us) service_pool.cc:224] Handling call
0709 13:20:00.924778 (+ 51197us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.925883 (+  1105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.926102 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.951536 15676 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:20:00.956614 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:00.958791 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:00.965723 15520 raft_consensus.cc:1275] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:20:00.953694 19402 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:20:00.966814 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:00.967510 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1975) took 1430 ms. Trace:
I20260709 13:20:00.969975 15520 rpcz_store.cc:276] 0709 13:19:59.536823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.879845 (+1343022us) service_pool.cc:224] Handling call
0709 13:20:00.957398 (+ 77553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.966274 (+  8876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.967452 (+  1178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.972311 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2004) took 1432 ms. Trace:
I20260709 13:20:00.974349 15520 rpcz_store.cc:276] 0709 13:19:59.539544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.970901 (+1431357us) service_pool.cc:224] Handling call
0709 13:20:00.971282 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.971945 (+   663us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:00.971973 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:00.972043 (+    70us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:00.972078 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.972136 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.972249 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":123,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":148}
I20260709 13:20:00.976970 16239 raft_consensus.cc:2393] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 9: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:00.969090 15424 catalog_manager.cc:5697] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:00.977941 15520 raft_consensus.cc:1240] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 13:20:00.979382 15677 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:20:00.987063 15600 raft_consensus.cc:1240] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:00.987862 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 1998) took 1449 ms. Trace:
I20260709 13:20:00.988350 15528 rpcz_store.cc:276] 0709 13:19:59.538062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.954625 (+1416563us) service_pool.cc:224] Handling call
0709 13:20:00.987278 (+ 32653us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.987538 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:00.987562 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:00.987609 (+    47us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:20:00.987647 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.987699 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.987799 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.909825 19454 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:00.990813 15669 raft_consensus.cc:1240] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:00.992091 15527 raft_consensus.cc:1240] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:00.992823 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2011) took 1445 ms. Trace:
I20260709 13:20:00.993101 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2003) took 1453 ms. Trace:
I20260709 13:20:00.993268 15520 rpcz_store.cc:276] 0709 13:19:59.541201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.976063 (+1434862us) service_pool.cc:224] Handling call
0709 13:20:00.976462 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:00.986547 (+ 10085us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.986793 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.993806 15527 rpcz_store.cc:276] 0709 13:19:59.539311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.959338 (+1420027us) service_pool.cc:224] Handling call
0709 13:20:00.981037 (+ 21699us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.992874 (+ 11837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.993049 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.994985 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1217) took 1223 ms. Trace:
W20260709 13:20:00.997045 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:00.998282 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 115274us
I20260709 13:20:00.998677 15528 rpcz_store.cc:276] 0709 13:19:59.768265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.989319 (+1221054us) service_pool.cc:224] Handling call
0709 13:20:00.989706 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.991527 (+  1821us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:00.991558 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:00.991614 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:00.991646 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.991717 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.991836 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:01.009018 15519 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:20:01.011009 15526 raft_consensus.cc:1240] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:01.014418 15675 raft_consensus.cc:1240] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:20:01.015702 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:01.017439 15523 raft_consensus.cc:1240] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:20:01.014722 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:01.020563 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1216) took 1259 ms. Trace:
I20260709 13:20:01.018280 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:01.020982 15521 rpcz_store.cc:276] 0709 13:19:59.761398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.978486 (+1217088us) service_pool.cc:224] Handling call
0709 13:20:01.015207 (+ 36721us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:01.020249 (+  5042us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:01.020280 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:01.020326 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:01.020364 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.020417 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:01.020513 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:01.023896 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2000) took 1485 ms. Trace:
I20260709 13:20:01.024276 15523 rpcz_store.cc:276] 0709 13:19:59.538582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.967440 (+1428858us) service_pool.cc:224] Handling call
0709 13:20:01.012054 (+ 44614us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:01.023644 (+ 11590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.023844 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:01.023135 15596 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:20:01.021366 15600 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:20:01.026302 15520 raft_consensus.cc:1240] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:01.027329 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1973) took 1487 ms. Trace:
I20260709 13:20:01.025738 16237 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:20:01.027746 15526 rpcz_store.cc:276] 0709 13:19:59.536334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.826178 (+1289844us) service_pool.cc:224] Handling call
0709 13:20:01.009718 (+183540us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:01.012766 (+  3048us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.023371 (+ 10605us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b54003ce1d0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:01.023505 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":421179776}
I20260709 13:20:01.025662 15491 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:01.030364 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1218) took 1260 ms. Trace:
W20260709 13:20:01.030660 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 65439us
I20260709 13:20:01.030768 15520 rpcz_store.cc:276] 0709 13:19:59.769735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.011653 (+1241918us) service_pool.cc:224] Handling call
0709 13:20:01.026137 (+ 14484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:01.030128 (+  3991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.030301 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:01.028803 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:01.025409 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:01.031940 16236 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
W20260709 13:20:01.032833 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.031870 15672 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:20:01.035678 15523 raft_consensus.cc:1240] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:20:01.017522 19459 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.035861 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:01.039479 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1220) took 1039 ms. Trace:
I20260709 13:20:01.041297 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1219) took 1223 ms. Trace:
I20260709 13:20:01.041446 15566 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:01.042589 19573 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:01.043154 16231 raft_consensus.cc:1240] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9. Ops: []
W20260709 13:20:01.043489 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.026193 19509 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:01.046105 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:01.047012 16237 raft_consensus.cc:2393] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 9: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:01.047955 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:01.048300 15525 raft_consensus.cc:2368] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 5. Current term is 6.
I20260709 13:20:01.010476 19512 consensus_queue.cc:1048] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:20:01.032780 19368 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.050912 19368 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:01.037045 19418 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:01.032397 19508 consensus_queue.cc:1048] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:20:01.052510 19418 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:01.052929 19418 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:01.053243 19368 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:01.058439 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:01.059734 19454 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:01.060739 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:01.042980 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:01.060618 19572 consensus_queue.cc:1059] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:01.060029 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:01.041813 15527 rpcz_store.cc:276] 0709 13:19:59.817864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.994737 (+1176873us) service_pool.cc:224] Handling call
0709 13:20:01.031191 (+ 36454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:01.040957 (+  9766us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:01.040982 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:01.041034 (+    52us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:01.041070 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.041132 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:01.041236 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:01.058465 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:01.076231 15600 raft_consensus.cc:2393] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 9: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:01.040411 15519 rpcz_store.cc:276] 0709 13:20:00.000461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.009952 (+1009491us) service_pool.cc:224] Handling call
0709 13:20:01.032012 (+ 22060us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:01.038092 (+  6080us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:01.039182 (+  1090us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:01.039228 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:01.039268 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.039326 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:01.039427 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:01.077948 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.074368 15595 raft_consensus.cc:2393] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 8: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:01.077872 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:20:01.077431 15641 leader_election.cc:336] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.217s (SENT)
I20260709 13:20:01.081525 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:01.081743 15642 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:01.082221 15669 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:01.043026 19573 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:01.025326 19506 consensus_queue.cc:1048] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:20:01.087975 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:01.093238 15602 raft_consensus.cc:2393] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 7: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:01.077587 15424 catalog_manager.cc:5697] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 4 to 7, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 7 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:01.047888 19509 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:01.095299 15527 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:01.101307 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:01.090107 19573 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:01.062582 19402 consensus_queue.cc:1059] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:20:01.110991 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:01.110293 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:01.112080 15597 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:20:01.117911 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:01.118211 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:01.109014 19409 consensus_queue.cc:1059] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:20:01.119699 15599 raft_consensus.cc:1275] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:20:01.106665 19371 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:01.119626 15601 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:20:01.122843 15567 leader_election.cc:336] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.092s (SENT)
W20260709 13:20:01.125448 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.096524 19459 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:01.126296 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 51845.5us
I20260709 13:20:01.127104 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:01.114373 15595 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:20:01.140592 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:01.146895 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 1972) took 1610 ms. Trace:
I20260709 13:20:01.155978 15522 rpcz_store.cc:276] 0709 13:19:59.536079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.824092 (+1288013us) service_pool.cc:224] Handling call
0709 13:20:00.824452 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.824679 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:20:00.955791 (+131112us) log.cc:844] Serialized 14 byte log entry
0709 13:20:01.146365 (+190574us) spinlock_profiling.cc:243] Waited 958 us on lock 0x7b18002828e0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1898c0a 00007f5ceafd8219 00007f5ce1920a40 00007f5ce191b4e4 00007f5ce191b455 00007f5ce4a03060 00007f5ce4a15716 00007f5ce4a26287 00007f5ce4a26208
0709 13:20:01.146529 (+   164us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:01.146578 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:01.146625 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.146691 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:01.146810 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5406,"prepare.queue_time_us":211398,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":59,"raft.queue_time_us":116405,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"spinlock_wait_cycles":2202496,"thread_start_us":186670,"threads_started":2,"wal-append.queue_time_us":137064}
W20260709 13:20:01.151458 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 111452us
W20260709 13:20:01.160015 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 120785us
I20260709 13:20:01.165419 15603 raft_consensus.cc:1275] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:20:01.123139 19438 raft_consensus.cc:3055] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:01.170698 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:01.171460 16231 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:01.169354 19438 raft_consensus.cc:740] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:01.174309 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:01.175108 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:01.178301 19595 log.cc:927] Time spent T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:20:01.183857 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:01.184582 15603 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:20:01.185117 16203 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:01.185895 19572 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:01.142330 19371 raft_consensus.cc:697] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:01.187299 19371 consensus_queue.cc:237] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:01.187561 19599 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:01.188688 19576 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:01.189271 19576 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:01.191078 19566 raft_consensus.cc:3055] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:01.191432 19576 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:01.200398 19210 raft_consensus.cc:3055] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:01.175043 19438 consensus_queue.cc:260] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:01.200903 19210 raft_consensus.cc:740] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:01.201814 15491 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:01.201828 19210 consensus_queue.cc:260] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:01.202786 15491 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:01.203603 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:01.203681 15491 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:01.205307 19508 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:01.205698 19508 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:01.206099 19508 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:01.210371 15491 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:01.210640 19459 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:01.211558 19425 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:01.211951 19425 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:01.212285 19425 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:01.221081 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:01.222549 19509 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:01.222407 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:01.223485 15596 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:20:01.223901 19210 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:01.224963 19594 log.cc:927] Time spent T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:20:01.223109 19509 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:01.225805 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:01.226500 19509 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:01.226790 19447 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:01.227190 19447 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:01.227540 19447 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:01.233214 15524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1208) took 1909 ms. Trace:
I20260709 13:20:01.235256 16204 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:01.239590 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:01.180300 19466 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:01.233656 15524 rpcz_store.cc:276] 0709 13:19:59.324102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.785173 (+1461071us) service_pool.cc:224] Handling call
0709 13:20:00.802018 (+ 16845us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:00.811768 (+  9750us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:00.811808 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:00.870559 (+ 58751us) log.cc:844] Serialized 26 byte log entry
0709 13:20:01.035272 (+164713us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:01.035305 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.035367 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:01.232981 (+197614us) raft_consensus.cc:1685] finished
0709 13:20:01.233002 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:01.233146 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":72459,"threads_started":1,"wal-append.queue_time_us":161410}
I20260709 13:20:01.250555 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:01.191505 19566 raft_consensus.cc:740] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:01.251147 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:01.252171 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:01.250597 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 53937.3us
I20260709 13:20:01.254334 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:01.255409 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 134822us
I20260709 13:20:01.261565 19438 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:01.261906 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:01.221477 19425 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:01.265810 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:01.186314 19572 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:01.250402 19599 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:01.281823 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:01.250480 19466 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:01.280494 16236 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:01.285866 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:01.248466 19512 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:20:01.290108 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:01.251504 19573 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:01.280284 19572 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:01.298732 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 237507us
I20260709 13:20:01.299856 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:01.285032 19466 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:01.257460 19576 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
W20260709 13:20:01.299248 15567 leader_election.cc:336] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.217s (SENT)
I20260709 13:20:01.304260 15567 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:01.307750 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:01.305124 19304 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:01.313206 15566 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:20:01.326798 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.334152 19566 consensus_queue.cc:260] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:01.335625 19566 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:01.360378 15568 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:01.366261 15598 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 8: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
W20260709 13:20:01.379547 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 111980us
I20260709 13:20:01.384898 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:01.365998 19402 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:01.361857 19459 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:01.393299 19447 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:01.406219 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:01.378218 19506 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:20:01.392447 19560 consensus_meta.cc:287] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.528s	user 0.008s	sys 0.003s
I20260709 13:20:01.426281 19512 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:01.429662 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 130436us
I20260709 13:20:01.430003 19512 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:01.432866 19512 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:01.436105 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.412138 19402 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:01.454079 15522 raft_consensus.cc:1240] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:01.397364 19599 raft_consensus.cc:697] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:01.470981 15642 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:01.460237 19402 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:01.487032 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:01.491474 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 44293.4us
I20260709 13:20:01.447196 19371 consensus_queue.cc:1059] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:20:01.508556 15641 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:01.509608 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 75737.8us
I20260709 13:20:01.489787 19576 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:01.520632 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:01.514200 15597 raft_consensus.cc:1240] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:01.474849 19508 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:01.497354 19560 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:01.530052 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:01.531754 19604 log.cc:927] Time spent T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.376s	user 0.003s	sys 0.000s
I20260709 13:20:01.530596 15599 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:20:01.534365 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:01.538411 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:01.542076 15493 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.563545 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:01.564349 15672 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:01.542629 15493 connection.cc:256] RPC call timeout handler was delayed by -1.0736455399999159s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15680000000000002s for a response
W20260709 13:20:01.571522 15412 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet bd36c2cfd99c45cea9e48096d54bed1a with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 5498ef3e89a941e39c25f68cf64e6b0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:01.581830 19622 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.582600 19622 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:01.585819 19622 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:01.587577 19572 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:01.587973 16205 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:01.589607 19445 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:01.542177 19573 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:01.593462 15525 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:20:01.597184 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 110254us
W20260709 13:20:01.595664 19596 log.cc:927] Time spent T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:20:01.610565 16231 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:01.610564 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:01.610689 15642 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:01.612936 15598 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:20:01.616549 15599 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:20:01.540284 19418 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:01.589897 19572 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:01.623872 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:01.627413 15594 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:20:01.629810 19508 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:01.613945 19621 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:20:01.632711 19425 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:01.634214 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:01.650538 15528 raft_consensus.cc:1240] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:20:01.652925 19573 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.645557 19505 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.593632 19605 consensus_queue.cc:1059] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:20:01.578824 19466 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:01.624301 19447 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:01.658159 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 47534.2us
W20260709 13:20:01.678902 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.653620 19509 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:01.607833 19599 consensus_queue.cc:237] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:01.716496 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:01.716514 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 108147us
I20260709 13:20:01.718712 16205 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:01.723924 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 215423us
W20260709 13:20:01.746052 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 126957us
I20260709 13:20:01.749532 15493 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:01.750479 19447 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:01.750954 19447 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:01.751330 19447 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:01.761062 15491 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:01.766000 15642 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:01.728627 19425 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:01.759920 19447 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:01.773105 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:01.773267 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:01.774852 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:01.714113 19505 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:01.762275 19572 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:01.776640 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:01.786314 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:01.775557 15677 raft_consensus.cc:2376] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12. Re-sending same reply.
I20260709 13:20:01.727509 19573 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:01.768093 19508 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:01.775619 19425 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:01.802102 15674 raft_consensus.cc:2410] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:20:01.809078 15493 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:01.788380 19572 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:01.811902 15641 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:01.815152 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:01.816753 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 83983.7us
I20260709 13:20:01.809842 19637 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:01.804890 19508 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:01.809273 19425 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:01.783748 19371 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:20:01.830703 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:01.820567 19637 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:01.816161 19572 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:01.835451 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 51289.6us
I20260709 13:20:01.831580 15670 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:01.832343 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:01.816179 19418 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:01.809880 19632 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:01.842319 19632 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:01.842892 19626 consensus_queue.cc:1059] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:20:01.844333 19635 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:01.820812 19466 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:01.832264 19637 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:01.841104 15641 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:01.836365 15671 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:20:01.842760 19632 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:01.844260 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:01.827960 19508 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:01.862550 15642 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:01.874851 15674 raft_consensus.cc:3055] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:01.875309 15674 raft_consensus.cc:740] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:01.815873 19644 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:20:01.880407 15493 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:01.831094 19631 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:01.886884 15672 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:20:01.889555 15668 raft_consensus.cc:1240] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:20:01.888440 15491 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:20:01.892501 19630 log.cc:927] Time spent T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:20:01.900859 15567 leader_election.cc:400] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9.
I20260709 13:20:01.901212 15567 leader_election.cc:403] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:01.910169 19573 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:01.914143 19566 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9.
I20260709 13:20:01.890233 19509 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:01.919384 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 78436us
I20260709 13:20:01.894449 19649 raft_consensus.cc:3055] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:01.899878 19466 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:20:01.932447 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 52126.2us
I20260709 13:20:01.933328 15493 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:01.934684 19466 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:01.936075 19466 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:01.935799 15674 consensus_queue.cc:260] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:01.937052 15674 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:01.942154 19627 consensus_queue.cc:1048] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:20:01.942749 19508 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:01.942994 19625 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:20:01.943270 19650 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:01.943769 19650 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:01.944141 19650 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:01.904608 19402 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:01.931418 19649 raft_consensus.cc:740] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:01.953610 19638 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:01.952534 19641 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:01.922612 19418 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:01.985157 19644 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.986881 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:01.997202 15672 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:01.992215 19631 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:01.995074 15674 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:20:01.987429 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 139300us
W20260709 13:20:01.991684 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 58449.9us
I20260709 13:20:01.999902 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:02.000918 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:02.001116 19648 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:02.001396 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:02.001579 15672 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:02.006592 15670 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:20:02.002744 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:02.017768 15493 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:01.986778 19624 consensus_queue.cc:1059] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:20:02.020105 15603 raft_consensus.cc:2393] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 14: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:02.024427 19637 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.026479 15599 raft_consensus.cc:1275] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:20:02.026680 19637 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:02.027038 15673 raft_consensus.cc:1240] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:02.027112 19128 raft_consensus.cc:3055] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:02.027474 15567 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:02.028558 19566 raft_consensus.cc:2804] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:02.028939 19566 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:02.029366 19585 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:02.030618 19649 consensus_queue.cc:260] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.031668 19649 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:02.029322 19566 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:02.034231 19447 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:02.036846 19425 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.037879 15566 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:02.038836 15566 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:02.040663 15672 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:02.039958 16203 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:20:02.051891 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.034047 19128 raft_consensus.cc:740] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:02.050000 15491 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:02.021337 19638 raft_consensus.cc:697] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:02.015996 19418 raft_consensus.cc:697] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:02.020262 19505 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:02.038714 19585 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:02.071317 15523 raft_consensus.cc:1240] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:02.061990 19573 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.068352 19506 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:02.065321 19626 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:02.089529 15597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 1928) took 1033 ms. Trace:
I20260709 13:20:02.067906 19447 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:02.034148 19651 consensus_queue.cc:1059] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:20:02.075589 19512 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:02.095537 15597 rpcz_store.cc:276] 0709 13:20:01.056546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.120140 (+ 63594us) service_pool.cc:224] Handling call
0709 13:20:01.514005 (+393865us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:01.524737 (+ 10732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:02.089285 (+564548us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b540043a150. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:02.089460 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":870786304}
I20260709 13:20:02.085881 19585 raft_consensus.cc:697] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:02.019311 19644 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.085880 19505 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:02.092762 19506 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.092190 19573 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.121441 19512 raft_consensus.cc:697] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:02.128800 19506 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:02.114419 19466 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:02.143757 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 62653.3us
I20260709 13:20:02.129712 19634 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:02.123266 19505 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.149857 15493 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:02.144420 19663 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:02.176648 15493 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:02.161794 19627 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.154886 19425 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:02.164975 19509 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:02.136134 19128 consensus_queue.cc:260] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.166417 19635 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:02.166280 19402 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:02.168376 19508 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:02.175458 19601 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.209872 19663 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.225342 19663 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:02.192149 19631 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.227063 19631 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.229342 19631 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:02.192510 15493 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:02.162011 19638 consensus_queue.cc:237] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.194717 19573 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:02.196070 19572 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.251250 19572 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:02.196949 19512 consensus_queue.cc:237] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:02.182812 19627 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:02.253683 19627 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:02.211784 19505 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:02.211445 19466 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:02.254823 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 62403.1us
I20260709 13:20:02.146171 19585 consensus_queue.cc:237] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:02.202641 19641 consensus_queue.cc:1048] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:02.194761 19644 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:02.232034 19660 log.cc:927] Time spent T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:20:02.146917 19650 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.232635 19665 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 13:20:02.262378 19642 raft_consensus.cc:3055] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:20:02.197054 19658 log.cc:927] Time spent T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:20:02.234551 15567 leader_election.cc:336] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.595s (SENT)
I20260709 13:20:02.269341 15567 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:02.233289 16231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57258 (request call id 845) took 1038 ms. Trace:
I20260709 13:20:02.270048 16231 rpcz_store.cc:276] 0709 13:20:01.194845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.194984 (+   139us) service_pool.cc:224] Handling call
0709 13:20:02.233088 (+1038104us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b54001cfcd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:02.233216 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":640945408}
I20260709 13:20:02.235773 19566 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.143334 19418 consensus_queue.cc:237] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.298301 19634 raft_consensus.cc:697] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:02.299757 19634 consensus_queue.cc:237] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.256902 19631 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:02.259387 15493 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:20:02.309749 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 54282.2us
I20260709 13:20:02.216835 19601 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.218008 19447 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:02.216464 19632 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.263965 19650 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:02.264529 15677 raft_consensus.cc:1240] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:02.265503 15522 raft_consensus.cc:1275] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:20:02.267187 19641 consensus_queue.cc:1059] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:20:02.262835 19642 raft_consensus.cc:740] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:02.260174 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:02.318671 15671 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:02.310567 15641 leader_election.cc:336] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.647s (SENT)
I20260709 13:20:02.215282 19506 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.306108 19505 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:02.318020 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 83495.4us
I20260709 13:20:02.321063 15493 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:02.321556 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:02.322402 15567 leader_election.cc:336] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.646s (SENT)
I20260709 13:20:02.321405 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:20:02.316502 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 168937us
I20260709 13:20:02.323189 15670 raft_consensus.cc:2393] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 14: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:02.322814 15567 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:02.324122 15641 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:02.316591 19512 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:02.333081 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.325026 19631 raft_consensus.cc:3055] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:02.334877 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.333096 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.336532 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:02.339583 15641 leader_election.cc:336] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.515s (SENT)
I20260709 13:20:02.339917 15641 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:02.325085 19677 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:02.340675 19677 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.341743 19635 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.342288 19635 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.342935 19677 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:02.344372 19635 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:02.345898 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:02.346174 19560 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:20:02.325292 19585 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:02.347972 19572 consensus_queue.cc:1059] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:20:02.334195 19651 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:02.338085 19631 raft_consensus.cc:740] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:02.338052 19402 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.351341 19601 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:02.347491 19585 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.331435 19650 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:02.352614 19678 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.332604 19512 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:02.352746 15567 leader_election.cc:336] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.649s (SENT)
I20260709 13:20:02.366591 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.367311 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.368650 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:02.329363 19641 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:02.373049 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.359566 19631 consensus_queue.cc:260] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:02.378193 15526 raft_consensus.cc:2368] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 5. Current term is 6.
I20260709 13:20:02.378377 19641 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.380570 19641 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:02.373802 15641 leader_election.cc:336] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.541s (SENT)
I20260709 13:20:02.381569 19626 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.372138 19402 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:02.382131 19626 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:02.382573 19566 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:02.383499 19402 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:02.376961 19572 raft_consensus.cc:3055] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:02.384135 19572 raft_consensus.cc:740] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:02.384716 15567 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:02.374869 19625 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:02.385082 19572 consensus_queue.cc:260] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.377269 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:02.386488 19642 consensus_queue.cc:260] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.387025 19631 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:02.387813 19642 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:02.335500 19447 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:02.386559 15641 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:02.377616 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.389846 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.378999 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:02.397631 15527 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
W20260709 13:20:02.403225 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 66682.2us
I20260709 13:20:02.404453 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.405017 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.405578 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.405160 15519 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
W20260709 13:20:02.410892 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.405400 19650 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:02.390688 19512 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:02.414372 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.384313 19626 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:02.417794 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:02.415374 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:02.416178 16237 raft_consensus.cc:1240] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:02.419034 16236 raft_consensus.cc:2393] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 9: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:02.412528 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.417781 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:02.420377 16238 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:02.412225 19572 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:02.425086 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 72378.3us
I20260709 13:20:02.412421 19651 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.369030 19638 consensus_queue.cc:1059] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:20:02.428273 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.417371 19402 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:02.428143 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:20:02.430018 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 56250us
I20260709 13:20:02.429184 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:02.431835 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.433034 15523 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:02.433400 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:02.439791 19659 log.cc:927] Time spent T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.305s	user 0.003s	sys 0.000s
I20260709 13:20:02.443759 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:02.444392 15668 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:02.447154 19454 raft_consensus.cc:3055] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:02.447659 19454 raft_consensus.cc:740] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:02.451036 19454 consensus_queue.cc:260] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:02.452301 19454 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:02.451370 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.459045 15519 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:20:02.462241 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1226) took 1770 ms. Trace:
I20260709 13:20:02.462715 15528 rpcz_store.cc:276] 0709 13:20:00.691531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.045174 (+353643us) service_pool.cc:224] Handling call
0709 13:20:01.045523 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:01.710238 (+664715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:02.461911 (+751673us) spinlock_profiling.cc:243] Waited 605 ms on lock 0x7b54003daed4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:02.462144 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1390914304}
I20260709 13:20:02.390499 19678 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.464828 16238 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:02.465564 19678 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:02.466813 19585 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:02.467950 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:02.464704 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:02.482376 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.483062 15528 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:20:02.483130 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.482318 16240 raft_consensus.cc:2368] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 9. Current term is 10.
I20260709 13:20:02.487507 16237 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
I20260709 13:20:02.488188 19670 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.486616 15528 raft_consensus.cc:1240] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:02.488724 19670 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.489248 15671 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:02.490556 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:02.491078 19670 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:02.491907 15674 raft_consensus.cc:1240] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:02.492894 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:02.494464 19651 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.496753 19651 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:02.474275 19632 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:02.498322 15522 raft_consensus.cc:2410] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:20:02.498507 15642 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:02.499913 19650 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:02.500358 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:02.501346 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.502504 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.503057 15524 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:02.503007 15525 raft_consensus.cc:1240] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:20:02.501473 19506 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:02.504359 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.505652 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.486701 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:02.504105 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.507825 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:02.506745 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:02.508805 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.511078 16203 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:02.512076 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.512605 15676 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:02.508886 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.513585 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:02.518761 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.502966 15521 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:20:02.506955 19425 raft_consensus.cc:3055] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:02.523458 15642 leader_election.cc:400] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 9. Current term is 10.
I20260709 13:20:02.512827 15675 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:20:02.523886 15642 leader_election.cc:403] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 13:20:02.525303 15641 leader_election.cc:336] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.844s (SENT)
I20260709 13:20:02.526130 15525 raft_consensus.cc:2393] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 9: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:02.526700 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:02.517683 15524 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:20:02.528532 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:02.528209 15668 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:02.531816 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.533247 15641 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:02.532743 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:02.549707 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.564412 15524 raft_consensus.cc:1240] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:02.529166 19425 raft_consensus.cc:740] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:02.533193 19642 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 9. Current term is 10.
W20260709 13:20:02.573004 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 62012.9us
I20260709 13:20:02.574448 15597 raft_consensus.cc:1275] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:20:02.587141 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:20:02.591073 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 65802.6us
W20260709 13:20:02.592626 15641 leader_election.cc:336] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.848s (SENT)
I20260709 13:20:02.593039 15641 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:02.587623 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:02.584416 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:02.590796 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:02.548012 19632 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:02.606324 19632 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.606805 19632 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:02.583699 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:02.558334 19650 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:02.586596 19425 consensus_queue.cc:260] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.614672 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:02.603317 19626 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:02.603477 19605 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:20:02.603761 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.532182 19512 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.615401 15600 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
I20260709 13:20:02.618465 15520 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:20:02.692970 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:02.625833 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 61687.7us
W20260709 13:20:02.676160 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 83580.4us
I20260709 13:20:02.615222 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:02.686126 15567 leader_election.cc:336] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.762s (SENT)
I20260709 13:20:02.703284 15567 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:02.704717 15521 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
W20260709 13:20:02.673573 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.710427 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:02.710477 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:02.711508 15597 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:02.688267 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.710558 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:02.723932 15677 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:20:02.724124 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 126354us
I20260709 13:20:02.728014 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:02.573351 19506 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:02.710448 19425 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:02.735333 16205 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:02.738998 19402 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.741798 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:02.741777 19402 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.743206 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:02.743847 15597 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:20:02.744108 19402 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:02.744715 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:02.745541 15597 raft_consensus.cc:2393] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 11: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:02.738812 19651 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:02.748395 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:02.732956 19664 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 13:20:02.752578 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.753451 15522 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 9: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:02.749801 15642 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:02.756458 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:02.741457 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:02.759156 15598 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:20:02.757609 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:02.756104 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:02.758425 16235 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:02.760943 15600 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:02.761237 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:02.657902 19462 consensus_queue.cc:1059] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.093s
I20260709 13:20:02.762044 15673 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:02.763751 19633 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:02.765770 19633 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.756448 19605 raft_consensus.cc:697] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:02.766188 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:02.768958 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 82889.9us
W20260709 13:20:02.769737 15567 leader_election.cc:336] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.832s (SENT)
I20260709 13:20:02.770134 15567 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:02.762800 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:02.763516 15597 raft_consensus.cc:2393] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 10: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:02.767215 16240 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:02.739959 19506 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.777679 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:02.777908 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.778501 16235 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:20:02.780352 15568 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:02.781736 15568 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:02.766167 19633 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:02.783223 15568 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:02.616389 19621 consensus_queue.cc:1059] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.125s
I20260709 13:20:02.784883 15568 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:02.765146 19626 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:02.788481 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:02.792402 16240 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:02.788879 15567 leader_election.cc:336] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.847s (SENT)
I20260709 13:20:02.790849 16205 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:02.763963 19460 consensus_queue.cc:1059] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:20:02.791669 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.795090 15567 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:02.795559 15527 raft_consensus.cc:2393] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 12: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:02.774274 19560 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:02.797256 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:02.756771 19650 consensus_queue.cc:1048] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:20:02.807041 16204 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:02.791337 19459 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:02.799211 19454 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:02.802084 19512 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:02.802709 19585 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.803455 15568 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:02.766887 19445 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:20:02.805245 19572 raft_consensus.cc:3055] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:02.775880 19643 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:02.778069 19402 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:02.809226 15566 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:02.778301 19603 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:02.804764 19566 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:20:02.813812 15412 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet bd36c2cfd99c45cea9e48096d54bed1a with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 5498ef3e89a941e39c25f68cf64e6b0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:02.835176 15600 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:02.792189 19626 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.837314 19585 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:02.830838 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.847783 15527 raft_consensus.cc:2393] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 11: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:02.847872 15566 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:02.842859 15670 raft_consensus.cc:1275] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:20:02.843343 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.797210 19670 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:02.767737 19661 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:20:02.792397 19671 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:20:02.813313 19560 raft_consensus.cc:697] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:02.814915 19668 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:02.830698 19572 raft_consensus.cc:740] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:02.830798 19673 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:20:02.862592 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 46501.4us
I20260709 13:20:02.861786 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.847065 19626 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:02.845665 19566 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.862946 19505 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:02.864763 19673 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.864238 19651 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:02.847278 19603 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:02.880301 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 91464.4us
W20260709 13:20:02.912545 15567 leader_election.cc:336] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.880s (SENT)
I20260709 13:20:02.912968 15567 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:02.875373 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.876765 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:02.883616 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:02.869103 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 165183us
W20260709 13:20:02.914300 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.914714 15525 raft_consensus.cc:2393] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 8: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:02.916180 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.916993 15527 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 10: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:02.850160 19643 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:02.914783 15641 leader_election.cc:336] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.927s (SENT)
I20260709 13:20:02.876420 19438 raft_consensus.cc:3055] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:20:02.879155 19688 log.cc:927] Time spent T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
W20260709 13:20:02.884079 19686 log.cc:927] Time spent T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:20:02.869678 19566 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:02.923018 19560 consensus_queue.cc:237] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.946300 16235 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:02.830544 19605 consensus_queue.cc:237] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:02.876153 19672 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:02.911379 19673 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:02.881381 19414 consensus_queue.cc:1059] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.088s
I20260709 13:20:02.887610 19585 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:02.948254 19691 log.cc:927] Time spent T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
W20260709 13:20:02.988447 19675 log.cc:927] Time spent T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:20:02.990216 19603 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:02.991858 19506 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:02.995147 16240 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:02.957087 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:02.966900 15522 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:02.965955 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:02.965353 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:02.953032 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:02.997219 15594 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:20:02.951977 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:02.964690 19687 log.cc:927] Time spent T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
W20260709 13:20:02.948807 15567 leader_election.cc:336] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.896s (SENT)
I20260709 13:20:03.002293 15594 raft_consensus.cc:1240] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:03.003034 15567 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:02.910715 19505 raft_consensus.cc:697] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:02.870564 19634 consensus_queue.cc:1059] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.094s
I20260709 13:20:02.954355 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:20:02.978974 15641 leader_election.cc:336] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.986s (SENT)
I20260709 13:20:03.005620 15641 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:02.979332 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 50832.4us
I20260709 13:20:02.943938 19459 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:02.985266 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:02.883721 19428 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 13:20:02.926919 19438 raft_consensus.cc:740] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:02.986364 15671 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 13:20:02.997310 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.998790 15526 raft_consensus.cc:2393] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 11: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:20:02.962533 19685 log.cc:927] Time spent T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:20:02.954725 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:02.953989 15525 raft_consensus.cc:1275] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:20:02.996538 15528 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:20:03.002480 15601 raft_consensus.cc:2368] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 9. Current term is 10.
I20260709 13:20:02.998202 15524 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:20:03.003767 15600 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:20:03.003829 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:02.957321 19633 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:03.014293 15642 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:03.028056 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 180246us
I20260709 13:20:03.015251 19645 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:20:02.995990 19409 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:03.012434 19625 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:03.001987 19454 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:03.026065 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 77289.6us
W20260709 13:20:03.039811 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:03.029349 15566 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:03.030054 15524 raft_consensus.cc:1240] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:03.033708 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:20:03.018707 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:03.040505 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 61562.3us
I20260709 13:20:03.040786 15594 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:03.036403 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:03.021171 19651 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:03.036513 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:02.976770 19572 consensus_queue.cc:260] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:03.044066 15641 leader_election.cc:336] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.989s (SENT)
I20260709 13:20:03.044437 15641 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:03.047148 16203 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:03.050276 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:03.050865 15603 raft_consensus.cc:2393] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 8: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:03.055227 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:03.058188 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:03.058981 15669 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 13: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:20:03.060336 19700 log.cc:927] Time spent T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:20:03.053742 19573 consensus_queue.cc:1048] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:20:03.051326 19585 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:03.065318 19585 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:03.065688 19585 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:03.070232 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:03.070909 15670 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:03.074002 19438 consensus_queue.cc:260] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:03.075306 19438 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:03.080669 15594 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:03.080981 15566 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:03.046671 19454 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:03.095506 19626 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:03.096356 19454 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:03.097038 19632 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:03.077862 19505 consensus_queue.cc:237] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:03.098891 19690 log.cc:927] Time spent T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
I20260709 13:20:03.099148 19632 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:03.099786 19632 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:03.080188 19625 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:03.100680 19625 raft_consensus.cc:697] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:03.097687 19626 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:03.146359 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:03.197474 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:03.212567 15598 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:20:03.213685 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:20:03.215284 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 69681.4us
I20260709 13:20:03.216043 15597 raft_consensus.cc:1275] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:20:03.217036 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 120200us
I20260709 13:20:03.101578 19701 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:20:03.219326 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:03.222011 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:03.226671 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:03.227370 15671 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
W20260709 13:20:03.230922 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 183852us
I20260709 13:20:03.236755 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:03.146198 19572 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:03.242563 16239 raft_consensus.cc:1240] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:03.245322 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:03.248248 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:03.249469 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:03.260903 16240 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:20:03.248837 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:03.262851 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:03.266667 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:03.229255 19671 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:03.271039 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:03.275075 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:03.294065 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:03.226253 19701 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:03.240410 19599 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:03.302023 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:03.302863 15675 raft_consensus.cc:2393] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 11: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:03.229861 19425 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.046s
W20260709 13:20:03.304764 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 254138us
I20260709 13:20:03.340620 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:20:03.343297 19709 log.cc:927] Time spent T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:20:03.303781 19643 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:03.356523 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 312470us
I20260709 13:20:03.361440 19633 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:03.357092 19702 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:03.367758 15641 leader_election.cc:336] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.926s (SENT)
I20260709 13:20:03.368240 15641 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:03.355108 19625 consensus_queue.cc:237] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:03.302091 19701 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:03.304890 19599 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:20:03.414915 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 122161us
I20260709 13:20:03.298213 19671 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:03.413621 19599 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:03.414458 19642 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:20:03.438162 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 70433.9us
W20260709 13:20:03.438956 15641 leader_election.cc:336] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.966s (SENT)
W20260709 13:20:03.438449 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:03.439989 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:03.404691 19454 consensus_queue.cc:1048] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:20:03.413336 19566 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:03.408133 19702 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:03.508860 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 90703.6us
I20260709 13:20:03.547932 19671 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:03.550367 19566 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:03.551560 19585 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:03.553015 15598 raft_consensus.cc:2368] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 13. Current term is 14.
I20260709 13:20:03.554919 19585 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:03.562668 16233 raft_consensus.cc:2410] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
W20260709 13:20:03.566262 19710 log.cc:927] Time spent T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.160s	user 0.000s	sys 0.003s
I20260709 13:20:03.569489 15567 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:03.572688 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 119684us
W20260709 13:20:03.650647 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:03.680152 15670 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:20:03.672959 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:03.688680 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:03.674011 19454 raft_consensus.cc:2764] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 13:20:03.717407 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 148019us
I20260709 13:20:03.718452 15567 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:03.731945 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 293001us
W20260709 13:20:03.746905 15493 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.006s	sys 0.016s
W20260709 13:20:03.747320 15493 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.006s	sys 0.016s
W20260709 13:20:03.747571 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 534146us
W20260709 13:20:03.765715 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:03.713166 19642 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.673856 19633 consensus_queue.cc:1059] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:20:03.779801 19572 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:03.782045 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 63687.8us
I20260709 13:20:03.784353 15567 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:03.788059 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 360060us
I20260709 13:20:03.774216 19512 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:03.779202 19585 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:20:03.800410 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 56677.6us
W20260709 13:20:03.801059 15641 leader_election.cc:336] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.092s (SENT)
I20260709 13:20:03.801419 15641 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:03.802155 19625 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:20:03.803323 15641 leader_election.cc:336] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.103s (SENT)
I20260709 13:20:03.804347 15641 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:03.779870 19723 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.807024 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:03.781489 19702 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:03.798610 19454 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:03.786517 19642 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:03.782145 19572 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:03.820847 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 77904.2us
I20260709 13:20:03.820981 15567 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:03.811185 19723 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:03.846693 19725 log.cc:927] Time spent T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:20:03.797971 19512 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:03.854524 15641 leader_election.cc:336] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.145s (SENT)
I20260709 13:20:03.854908 19704 raft_consensus.cc:3055] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:03.855638 19704 raft_consensus.cc:740] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:03.855032 15641 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:03.856487 19704 consensus_queue.cc:260] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:03.857028 19626 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:03.857427 19612 raft_consensus.cc:3055] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:03.857755 19704 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:03.858385 19642 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:03.862497 19736 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:03.864964 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:03.865579 16234 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:03.866825 19454 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:03.873088 19634 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:03.873703 19634 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:03.874991 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 61079.7us
I20260709 13:20:03.875775 15567 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:03.876045 19634 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:03.874189 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 52563.7us
I20260709 13:20:03.876612 19454 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:03.879498 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:03.879801 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:03.882253 19599 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:03.877785 15567 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:03.820590 19625 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:03.906373 16234 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 15.
W20260709 13:20:03.867552 19673 consensus_meta.cc:287] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.865s	user 0.016s	sys 0.000s
W20260709 13:20:03.910611 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 106233us
I20260709 13:20:03.864044 19612 raft_consensus.cc:740] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:03.911099 19701 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:03.917335 19512 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:03.910408 19454 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:03.870456 19723 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:03.924489 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 46770.3us
I20260709 13:20:03.925334 19701 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:03.936466 19612 consensus_queue.cc:260] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:03.914709 19673 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:03.922186 19625 consensus_queue.cc:1059] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:20:03.950681 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:03.951508 15671 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:20:03.940434 19572 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:03.957867 16203 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:03.966624 15567 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:03.968962 15567 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:03.970173 19672 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:03.970603 19672 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:03.970927 19672 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:03.971352 19454 consensus_queue.cc:1059] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:03.972913 19454 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:03.973452 19702 raft_consensus.cc:3055] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:03.973881 19702 raft_consensus.cc:740] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:03.975041 15567 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:03.979303 19585 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:03.979696 15641 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:03.980337 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 165672us
I20260709 13:20:03.980965 19625 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:03.981293 15493 connection.cc:256] RPC call timeout handler was delayed by -1.7002120059999015s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3269s for a response
I20260709 13:20:03.981968 19673 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:03.983219 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:03.974872 19702 consensus_queue.cc:260] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:03.985375 19425 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.977473 19599 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:03.977811 19711 raft_consensus.cc:3055] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:03.985941 19425 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:03.986649 19711 raft_consensus.cc:740] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:03.987529 19711 consensus_queue.cc:260] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:03.988078 19425 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:03.988826 19711 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:03.988762 19425 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:03.960861 19737 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:03.993836 19737 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:03.994270 19737 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:03.993862 19425 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:03.997818 19425 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:03.980659 15567 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:03.998836 19701 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:03.985857 19702 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:04.002303 19128 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:04.003418 19701 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:04.003930 19701 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:04.006058 19701 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:03.990510 19664 raft_consensus.cc:3055] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:20:03.965690 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:04.015951 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:03.982376 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:04.014056 15641 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:04.044181 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 133709us
I20260709 13:20:04.070155 15602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 2207) took 1486 ms. Trace:
I20260709 13:20:04.074283 15602 rpcz_store.cc:276] 0709 13:20:02.531484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.533765 (+  2281us) service_pool.cc:224] Handling call
0709 13:20:02.683114 (+149349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:02.799559 (+116445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:02.799589 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:02.799629 (+    40us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:02.799670 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.017542 (+1217872us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b540032b9d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:04.017591 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:04.017729 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":266247936}
I20260709 13:20:03.988020 19672 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.089011 15520 raft_consensus.cc:1240] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:04.111728 19626 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:04.016927 19512 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:04.014835 19664 raft_consensus.cc:740] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:04.016052 19642 consensus_queue.cc:1059] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.099s
I20260709 13:20:04.140254 19642 consensus_queue.cc:1243] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer d3c4f5bd76e0445795bad5f6dd93ff69 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:20:04.140718 19650 raft_consensus.cc:3055] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:04.141104 19650 raft_consensus.cc:740] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:04.141947 19650 consensus_queue.cc:260] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:04.119707 19512 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:04.143152 19650 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:04.144954 19512 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:04.145478 19512 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.146081 19625 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:04.147267 19672 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:04.138141 19737 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.149447 19737 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:04.139178 19664 consensus_queue.cc:260] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.150655 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:04.156848 15675 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:04.170688 15677 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:04.155951 19512 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:04.165383 15493 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:04.170058 19664 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:04.215267 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:04.222868 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 310218us
I20260709 13:20:04.230688 15527 raft_consensus.cc:1240] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 11. Current term is 12. Ops: []
W20260709 13:20:04.250700 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 236747us
I20260709 13:20:04.251928 15641 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:04.252981 19625 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:04.255858 19698 log.cc:927] Time spent T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 1.265s	user 0.007s	sys 0.017s
I20260709 13:20:04.257429 15599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 2208) took 1725 ms. Trace:
I20260709 13:20:04.257807 15599 rpcz_store.cc:276] 0709 13:20:02.531752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.533821 (+  2069us) service_pool.cc:224] Handling call
0709 13:20:02.673816 (+139995us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:02.729253 (+ 55437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:02.729284 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:02.735914 (+  6630us) log.cc:844] Serialized 26 byte log entry
0709 13:20:02.871844 (+135930us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:20:02.871888 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:02.872714 (+   826us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:04.255280 (+1382566us) spinlock_profiling.cc:243] Waited 7.05 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:20:04.257133 (+  1853us) raft_consensus.cc:1685] finished
0709 13:20:04.257155 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:04.257367 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13400,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":8254,"spinlock_wait_cycles":16204160,"thread_start_us":126478,"threads_started":1,"wal-append.queue_time_us":200902}
I20260709 13:20:04.259387 16204 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:04.261142 19512 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:04.261705 19512 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.262182 19512 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:04.264298 19512 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:04.266659 19711 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:04.267189 15674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 1172) took 1552 ms. Trace:
I20260709 13:20:04.267572 15674 rpcz_store.cc:276] 0709 13:20:02.714603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.714711 (+   108us) service_pool.cc:224] Handling call
0709 13:20:02.732772 (+ 18061us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:02.755027 (+ 22255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:02.755060 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:02.755104 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:02.755142 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.266965 (+1511823us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b540009e2d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:04.267010 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:04.267121 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50683264}
I20260709 13:20:04.268703 15566 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:04.270290 19673 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:04.270701 19673 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:04.271037 19673 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:04.275548 19454 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.276173 19454 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.277967 19752 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:04.279047 19752 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.279551 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:04.280049 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:04.280467 16232 raft_consensus.cc:2393] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 15: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:04.280843 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:04.281065 19585 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.262107 19734 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:04.278028 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:04.283532 16233 raft_consensus.cc:2368] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10.
I20260709 13:20:04.278367 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:04.285142 16234 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:04.280371 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:04.286403 16237 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:04.279570 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:04.287377 16238 raft_consensus.cc:2410] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:20:04.290961 16239 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:20:04.279915 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:20:04.293500 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 45629.7us
I20260709 13:20:04.293319 16203 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:04.294450 16240 raft_consensus.cc:2368] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10.
I20260709 13:20:04.295822 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:04.296653 15677 raft_consensus.cc:2410] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:20:04.297230 15675 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:04.301584 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:04.302399 15675 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:04.250576 19650 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:04.312165 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:04.312259 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:04.312911 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:04.313335 15671 raft_consensus.cc:2368] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10.
I20260709 13:20:04.312933 15670 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:04.316588 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:04.317201 15674 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:04.317884 15669 raft_consensus.cc:2393] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 15: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:20:04.319633 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 154348us
I20260709 13:20:04.321039 15493 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:04.322285 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:04.330384 15568 leader_election.cc:400] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10.
I20260709 13:20:04.332340 15568 leader_election.cc:400] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10.
I20260709 13:20:04.332670 15568 leader_election.cc:403] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:20:04.335726 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:04.337662 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:20:04.339143 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:04.279528 19752 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:04.340339 15599 raft_consensus.cc:2393] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 15: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:04.341909 15566 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:04.342904 15412 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet bd36c2cfd99c45cea9e48096d54bed1a with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 5498ef3e89a941e39c25f68cf64e6b0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:04.335922 19438 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:04.350066 19664 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:04.281584 19585 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:04.354636 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:04.267220 19711 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:04.358880 15522 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2040) took 1599 ms (client timeout 1850 ms). Trace:
I20260709 13:20:04.357656 16237 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
W20260709 13:20:04.359333 15522 rpcz_store.cc:269] 0709 13:20:02.761702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.770889 (+  9187us) service_pool.cc:224] Handling call
0709 13:20:04.358715 (+1587826us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b540029c7d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:04.358823 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":433089792}
I20260709 13:20:04.362955 15568 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:04.367718 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 115927us
I20260709 13:20:04.367798 16235 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:04.316094 19409 consensus_queue.cc:1059] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:20:04.375193 15602 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:04.354444 19704 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.336927 19650 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:04.344977 19425 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:04.394369 15598 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34102 (request call id 1042) took 1175 ms (client timeout 1548 ms). Trace:
I20260709 13:20:04.362349 19757 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.366465 19749 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.395576 15674 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
W20260709 13:20:04.395833 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 57392.4us
I20260709 13:20:04.375133 19642 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:04.398370 16203 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:04.367434 19438 raft_consensus.cc:697] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:04.374888 19512 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:20:04.394865 15598 rpcz_store.cc:269] 0709 13:20:03.218642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.221338 (+  2696us) service_pool.cc:224] Handling call
0709 13:20:04.394123 (+1172785us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b5400232654. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:04.394277 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":758509312}
I20260709 13:20:04.404172 19752 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:04.388959 15670 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:04.390141 19626 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:04.398505 19750 log.cc:927] Time spent T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:20:04.390177 19754 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:04.396735 19759 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:04.416271 19665 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:04.416182 19626 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.399021 19425 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.418869 19665 raft_consensus.cc:697] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:04.418557 19626 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:04.419847 19665 consensus_queue.cc:237] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.420815 19454 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:04.420953 19761 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:04.421468 19454 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:04.421974 19454 raft_consensus.cc:697] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:04.422835 19454 consensus_queue.cc:237] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:04.437393 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:04.444191 15602 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:20:04.396075 19704 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:04.458515 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:04.459611 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 117777us
I20260709 13:20:04.398517 19642 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.399474 19755 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:20:04.473841 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:04.474399 15600 raft_consensus.cc:1240] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:20:04.476243 15600 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:20:04.477249 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:04.478259 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:04.476161 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:04.400187 19757 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:04.486543 15599 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:04.487972 16239 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:20:04.511148 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:04.511987 15566 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:04.400586 19749 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.512224 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:04.516845 15601 raft_consensus.cc:2410] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:20:04.510145 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:04.468962 19702 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:04.522879 15600 raft_consensus.cc:2393] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 11: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:04.520753 16205 leader_election.cc:400] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 13. Current term is 14.
I20260709 13:20:04.526932 16205 leader_election.cc:403] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:04.522084 16239 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:20:04.528791 16239 raft_consensus.cc:606] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:04.519212 15566 leader_election.cc:400] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10.
W20260709 13:20:04.521770 19751 log.cc:927] Time spent T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:20:04.521979 19762 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:04.519450 19765 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:04.529043 19746 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.519938 19210 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.553704 15566 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:04.555840 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:04.526837 19673 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.528136 19711 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:04.556213 15642 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:04.569648 16205 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:20:04.599668 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 43526.4us
I20260709 13:20:04.600536 15642 leader_election.cc:400] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 9. Current term is 10.
I20260709 13:20:04.600883 15599 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:04.507349 19438 consensus_queue.cc:237] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.602435 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:04.603122 16238 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:20:04.602990 16237 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:20:04.607251 16205 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:04.608453 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:04.536870 19702 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.595579 16239 raft_consensus.cc:2393] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 10: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:04.561580 15602 raft_consensus.cc:1240] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:04.559631 19672 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:04.610656 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 57036.8us
I20260709 13:20:04.587805 19762 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:04.611168 16231 raft_consensus.cc:2393] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 11: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:04.612526 19462 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:04.592283 19445 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:20:04.586599 19601 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 13. Current term is 14.
I20260709 13:20:04.613907 16205 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:04.606236 19748 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:04.592079 19736 consensus_queue.cc:1059] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:20:04.594530 19585 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:04.613795 19775 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:04.627738 15598 raft_consensus.cc:2368] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 11. Current term is 12.
W20260709 13:20:04.639886 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 89110.9us
I20260709 13:20:04.638564 19677 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:20:04.593632 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 264547us
I20260709 13:20:04.628860 19762 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:04.603260 19210 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.597615 19746 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.646829 19677 raft_consensus.cc:697] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:04.649600 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:04.650754 19770 log.cc:927] Time spent T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.095s	user 0.002s	sys 0.000s
W20260709 13:20:04.651523 19744 log.cc:927] Time spent T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.414s	user 0.000s	sys 0.005s
I20260709 13:20:04.639510 19748 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:04.674952 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:04.593119 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 230236us
I20260709 13:20:04.658914 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:04.678960 16239 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:04.646404 15493 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:04.671358 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:04.643409 19761 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.682579 19677 consensus_queue.cc:237] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.683106 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:04.683878 19673 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:04.684254 19776 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:04.684733 19757 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:04.685040 19763 log.cc:927] Time spent T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:20:04.685252 19642 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:04.687295 19210 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:04.688041 19749 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:04.642330 19409 consensus_queue.cc:1059] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
W20260709 13:20:04.690078 19771 log.cc:927] Time spent T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:20:04.690589 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:04.594719 19765 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.630252 19672 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.652098 19625 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:04.709003 19704 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:04.716638 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 70299.6us
I20260709 13:20:04.717080 19512 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:04.721702 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:04.717336 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:04.714082 19761 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.694525 19748 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:04.694232 19585 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:04.713191 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:04.716120 15568 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:04.717819 19603 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:04.738029 19603 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.732887 15493 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:04.742689 19752 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:04.743556 19765 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:04.743928 15493 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:04.744130 19765 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:04.744895 16205 leader_election.cc:400] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 11. Current term is 12.
I20260709 13:20:04.745303 15493 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:04.745522 19754 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:04.746250 19512 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.746651 15599 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:20:04.746749 19512 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.747349 19761 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:04.748835 19512 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:04.748976 19672 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:04.745220 16205 leader_election.cc:403] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:04.752408 15493 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:04.754113 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:04.754482 19723 raft_consensus.cc:3055] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:04.754765 16237 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:04.754854 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:04.753618 15493 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:04.754421 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:04.756731 15493 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:04.766142 15568 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:04.768173 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:04.746428 19724 raft_consensus.cc:3055] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:04.767802 19723 raft_consensus.cc:740] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:04.780737 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:04.781005 15642 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:04.786751 15676 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38190179f6494b5591027e1c41f83176"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:20:04.787401 15676 raft_consensus.cc:606] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:04.763654 19585 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.788813 19760 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:04.789575 19760 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:20:04.763248 19454 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:04.791165 19585 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:04.792969 19650 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:04.793567 19650 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:04.749372 19512 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:04.795321 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:04.763909 19409 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:20:04.796849 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:04.798909 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:04.799642 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:04.772318 19781 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:04.779654 19752 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:04.775373 19724 raft_consensus.cc:740] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:04.780929 19701 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:20:04.844293 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 87637.1us
I20260709 13:20:04.843118 19701 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:04.845551 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:04.794011 19650 raft_consensus.cc:697] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:04.796953 19643 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.847772 15493 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:04.848991 15493 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:04.845184 19701 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:04.803607 19454 raft_consensus.cc:697] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:20:04.860882 19780 log.cc:927] Time spent T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:20:04.863657 19746 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:04.864965 19702 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:04.865540 19702 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:04.849479 19512 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:04.890618 15493 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:04.853181 19760 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:04.901331 15493 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
W20260709 13:20:04.903681 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:04.907087 15676 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:20:04.907768 15676 raft_consensus.cc:606] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Received request to transfer leadership to TS 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:04.911476 15676 raft_consensus.cc:614] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:04.853295 19754 raft_consensus.cc:697] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:04.892783 19752 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:04.929177 15493 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:04.937687 16239 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "974a060267d24f00888c0942dfbca77e"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:20:04.938503 16239 raft_consensus.cc:606] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:04.860468 19643 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.904831 19788 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:04.889618 19702 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.891923 19781 raft_consensus.cc:697] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:04.889168 19723 consensus_queue.cc:260] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.978667 19760 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:04.909801 19724 consensus_queue.cc:260] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.979491 19762 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:04.982244 19787 consensus_queue.cc:1048] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:20:04.939186 19650 consensus_queue.cc:237] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:04.979043 19454 consensus_queue.cc:237] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:04.941025 19777 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:05.019831 19643 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:05.020927 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 91843.4us
I20260709 13:20:05.020447 19777 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:05.021854 19723 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:05.028601 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.030746 15526 raft_consensus.cc:2410] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:20:05.031970 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.032611 15522 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
W20260709 13:20:05.032030 19789 log.cc:927] Time spent T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:20:05.033756 15528 raft_consensus.cc:1275] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:20:05.034119 15528 pending_rounds.cc:85] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 4
I20260709 13:20:05.034333 15528 pending_rounds.cc:107] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:20:05.034591 15528 raft_consensus.cc:2889] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:05.035339 19603 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:05.029877 19777 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:05.043040 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:05.043658 15598 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:20:05.045495 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:05.050966 16205 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:05.053611 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.054337 15527 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:20:05.046183 15676 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:05.056950 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.057600 15527 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:05.059607 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.059968 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:05.054143 19765 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:05.061168 19791 log.cc:927] Time spent T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:20:05.061605 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:05.057786 19642 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:05.062680 19642 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:05.064770 19642 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:05.059094 19626 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.066246 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.067013 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:05.067448 15671 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:05.065625 19626 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:05.071703 19626 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:05.065580 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:05.066992 15522 raft_consensus.cc:2393] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 10: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:05.009147 19754 consensus_queue.cc:237] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:05.078974 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.074213 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.090412 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.102344 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:05.122354 15669 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:05.103905 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:05.058477 19428 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:05.125669 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:05.121316 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:05.089737 19759 consensus_meta.cc:287] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.638s	user 0.000s	sys 0.011s
I20260709 13:20:05.061801 19762 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:05.130183 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:05.135190 15674 raft_consensus.cc:2410] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 11 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:20:05.138618 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:05.147689 15567 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:05.149060 15521 raft_consensus.cc:2393] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 10: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:05.150508 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:05.120775 19585 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:05.159837 15567 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:05.061069 19765 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:05.162611 15567 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:05.163977 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:05.164372 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.165148 15521 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:20:05.169976 19785 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:05.170511 19785 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:20:05.170895 16235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57258 (request call id 857) took 1934 ms (client timeout 2178 ms). Trace:
W20260709 13:20:05.171340 16235 rpcz_store.cc:269] 0709 13:20:03.236070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.236220 (+   150us) service_pool.cc:224] Handling call
0709 13:20:05.170676 (+1934456us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54005848d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:05.170803 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2415590784}
I20260709 13:20:05.171693 19787 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:05.172086 19748 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:05.172245 19787 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:05.159425 19759 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10.
I20260709 13:20:05.175272 19765 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:05.175846 19776 raft_consensus.cc:2764] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:20:05.176564 19757 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:05.176977 19757 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:05.177286 19757 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:05.182538 15596 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:05.187522 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:05.188957 16203 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:05.190084 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:05.190953 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.192898 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.193513 15521 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:05.170872 19785 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:05.195412 19787 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:05.125109 19626 consensus_queue.cc:1048] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 13:20:05.135346 19428 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:05.049015 19781 consensus_queue.cc:237] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:05.203228 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:05.207399 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.186275 19761 consensus_queue.cc:1048] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:20:05.208505 15521 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:05.213703 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:05.215543 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.215957 15520 pending_rounds.cc:85] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 7
I20260709 13:20:05.216224 15520 pending_rounds.cc:107] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:20:05.216470 15520 raft_consensus.cc:2889] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:05.212361 16203 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:05.188160 19776 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:05.218466 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 55049.8us
I20260709 13:20:05.189007 19702 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:05.220359 15641 leader_election.cc:336] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.850s (SENT)
I20260709 13:20:05.220338 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.215369 19462 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:05.225514 15526 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:20:05.201490 19757 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:05.231658 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.208948 19723 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.236956 15526 raft_consensus.cc:2393] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 8: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:05.208261 19428 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:05.234800 15567 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:05.186600 19585 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:05.248536 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.249378 15525 raft_consensus.cc:2393] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 12: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:05.250272 15567 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:05.226737 19748 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:05.253633 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:05.257685 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:05.236037 19462 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:05.236050 19734 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:05.263669 15567 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:05.263677 16234 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:05.263593 15522 raft_consensus.cc:2393] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 11: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:05.255788 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:20:05.256338 15641 leader_election.cc:336] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.889s (SENT)
I20260709 13:20:05.279682 15641 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:05.256704 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:05.262600 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:05.267895 19425 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:05.222391 19776 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:05.270249 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:05.270931 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.280522 16237 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:20:05.288537 15671 raft_consensus.cc:1275] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:05.282274 15567 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:05.288185 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:05.284206 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:05.286943 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:05.295470 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:05.289572 16236 raft_consensus.cc:2376] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12. Re-sending same reply.
I20260709 13:20:05.255515 19454 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:05.299716 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:05.300931 15525 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:05.271960 19462 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:05.269531 19777 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:05.276937 19734 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:05.246769 19622 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:20:05.310302 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:05.303726 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:05.302918 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.276373 19802 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:05.246165 19723 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:05.304700 19800 log.cc:927] Time spent T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
W20260709 13:20:05.307071 19409 consensus_meta.cc:287] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.001s
I20260709 13:20:05.312578 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:05.296039 19672 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:05.319291 15493 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:05.309247 16239 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:05.320412 19754 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:05.311985 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.296104 19761 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:05.330986 19754 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:05.331357 15493 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:05.332274 19787 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:05.332634 15493 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:05.332800 19787 raft_consensus.cc:697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:05.333276 19754 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:05.334139 19753 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:05.335129 19409 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 11. Current term is 12.
I20260709 13:20:05.336014 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.333802 15493 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:05.310242 19734 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:05.338225 19788 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:05.339612 15493 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:05.341156 15493 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:05.341883 19760 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:05.347560 16233 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:20:05.324959 15521 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:20:05.338124 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:05.326839 15524 raft_consensus.cc:2393] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 7: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:05.350778 16236 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:05.350020 16231 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:05.360483 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:05.360628 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:05.307821 19701 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:05.387248 19672 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:05.388738 19672 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:05.393039 19785 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:05.520365 19428 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:05.521270 19777 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:05.524348 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.528405 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.529528 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.562268 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.550961 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:05.551981 19785 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:05.552686 19805 log.cc:927] Time spent T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.301s	user 0.002s	sys 0.000s
W20260709 13:20:05.555924 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:05.333817 19787 consensus_queue.cc:237] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:05.560581 15493 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:05.562923 15523 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
W20260709 13:20:05.559587 19804 log.cc:927] Time spent T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.198s	user 0.000s	sys 0.005s
I20260709 13:20:05.531455 15526 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:05.574640 19585 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:05.614202 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 53685.8us
I20260709 13:20:05.616992 15525 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:05.565196 19409 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.624213 15522 raft_consensus.cc:3055] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:05.627086 15522 raft_consensus.cc:740] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:05.627823 15493 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:05.634202 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.619871 19760 raft_consensus.cc:2804] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:05.615442 19757 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:05.610809 19428 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:05.650991 19776 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:05.632830 19409 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:05.660579 19409 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:05.661911 19748 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.662478 19748 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:05.664490 19748 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:05.665032 15642 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:05.665447 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 409114us
I20260709 13:20:05.667950 15491 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:05.669227 19605 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:05.669358 19425 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:05.669842 19605 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:05.670909 15642 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:05.672772 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:05.672983 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:05.675724 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:05.677294 19794 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:05.678040 19794 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:05.678406 19794 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:05.678753 19794 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:05.680631 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:05.681257 15674 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:05.682443 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:05.683055 15671 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:05.683185 19701 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:05.683615 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:05.684597 19612 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:05.685736 19711 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:05.685775 19650 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.687177 19650 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:05.686200 15641 leader_election.cc:336] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.938s (SENT)
I20260709 13:20:05.688592 15641 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:05.688846 19723 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:05.689519 19749 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:05.690680 19672 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:05.693609 15668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:20:05.694281 15668 raft_consensus.cc:606] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Received request to transfer leadership to TS 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:05.694576 15668 raft_consensus.cc:614] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:05.696305 15671 raft_consensus.cc:1275] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:20:05.698407 15670 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:05.700066 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:20:05.699386 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 77789.2us
I20260709 13:20:05.701937 15677 raft_consensus.cc:2393] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 7: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:05.699800 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:05.702898 15671 raft_consensus.cc:2393] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 10: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:05.705811 15674 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:05.700584 19784 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:05.707990 19650 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:05.708817 15641 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.999s (SENT) [suppressed 54 similar messages]
I20260709 13:20:05.709184 19753 consensus_queue.cc:1048] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:05.710822 15595 raft_consensus.cc:1275] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:05.711005 15675 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:20:05.712133 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:05.713743 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:20:05.708945 15596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34102 (request call id 1049) took 1232 ms. Trace:
I20260709 13:20:05.722290 15596 rpcz_store.cc:276] 0709 13:20:04.476879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.477004 (+   125us) service_pool.cc:224] Handling call
0709 13:20:05.708776 (+1231772us) spinlock_profiling.cc:243] Waited 705 ms on lock 0x7b540032b9d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:05.708888 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1620372224}
I20260709 13:20:05.723785 19462 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:05.634125 19777 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:05.727192 16234 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:05.727604 19734 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:05.730211 16236 raft_consensus.cc:2368] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11.
I20260709 13:20:05.665938 19781 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.731149 19781 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:05.733462 19781 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:05.646885 19760 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:05.735844 19760 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:20:05.749791 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 55106.8us
I20260709 13:20:05.761585 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:05.766767 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:05.766822 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 64846us
I20260709 13:20:05.768707 15493 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:05.729740 19734 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:05.782856 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:05.788516 19605 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:05.790835 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:05.794502 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:05.795176 15670 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:05.795938 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:05.796620 15668 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:05.789079 19777 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:05.799311 15493 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:05.806716 15668 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:05.765111 19755 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:05.810859 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:05.786069 19462 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:05.814450 15493 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:05.784905 19644 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:20:05.822821 15673 raft_consensus.cc:1275] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:20:05.809526 19753 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:05.830440 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:05.831048 15668 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:05.812594 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 103799us
I20260709 13:20:05.820176 19777 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:05.838541 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:05.841372 16204 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:05.836602 15641 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:05.838155 15668 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:05.842049 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:05.843046 15673 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:05.843325 15641 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:05.819865 19425 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:20:05.848949 15522 consensus_queue.cc:260] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:05.851323 19650 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:05.838653 19748 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.855818 15642 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:05.856644 16204 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:05.859221 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:05.859500 16239 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:20:05.862823 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:05.864274 15568 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:05.865864 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:05.871773 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:05.872400 16233 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:05.868772 16203 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:05.873351 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:05.877233 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:05.886052 15521 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:05.887514 15596 raft_consensus.cc:1240] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:05.889380 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:05.840744 19777 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:05.854519 19748 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:05.897615 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:05.895704 16203 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:05.891628 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:05.891832 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.893527 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:05.899875 15521 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:05.863135 19665 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:05.893887 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:05.930394 15675 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:05.890185 16238 raft_consensus.cc:2393] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 10: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:05.860051 19788 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:05.900058 15596 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:05.817335 19755 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:05.936410 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:05.937052 15520 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:05.904726 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:05.937971 15528 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:20:05.867152 19785 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:05.869635 19711 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:05.932413 15566 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:05.890256 19757 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:05.936203 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:05.897452 19777 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:05.866542 19745 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:05.902622 19428 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:05.867635 19749 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:20:05.944164 15674 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:20:05.859558 19813 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.094s
I20260709 13:20:05.955013 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:05.932463 19599 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:05.939860 16237 raft_consensus.cc:1240] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:05.962733 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:05.967996 15601 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:20:05.965827 16231 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:20:05.974961 15521 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
W20260709 13:20:05.976759 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 162385us
I20260709 13:20:05.946501 19802 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:06.027051 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:05.955449 19428 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:05.987428 15599 raft_consensus.cc:1275] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:20:05.982157 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 117033us
I20260709 13:20:06.131896 15568 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:06.133180 15641 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:05.966527 19757 raft_consensus.cc:697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:05.947100 19785 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:05.955101 19768 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:20:05.997576 15669 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:05.976341 19711 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:05.983238 19599 raft_consensus.cc:697] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:06.091521 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:06.095544 15566 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:06.160607 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 65186.2us
I20260709 13:20:06.162050 15566 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:06.097019 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:06.115607 19605 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.936805 19642 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
W20260709 13:20:06.124716 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:06.125222 15412 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet bd36c2cfd99c45cea9e48096d54bed1a with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 5498ef3e89a941e39c25f68cf64e6b0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:20:06.174326 15412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 49167.4us
I20260709 13:20:06.128026 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:06.175304 15603 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:20:06.090200 19701 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:06.177157 19701 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.179476 19701 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:06.180088 19701 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:06.180773 19701 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:06.097376 19748 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:06.182281 19748 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:06.097442 19776 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:06.183081 19776 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.185235 19776 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:06.063856 19754 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:06.156037 19768 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:06.163681 15521 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:06.163215 15525 raft_consensus.cc:2393] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 6: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:06.193804 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:06.195523 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:06.196568 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:06.198073 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:06.198499 19757 consensus_queue.cc:237] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.202180 19650 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:06.160727 19794 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:06.150272 19811 log.cc:927] Time spent T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 13:20:06.220445 19599 consensus_queue.cc:237] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.159816 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:06.169353 19605 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.171130 19767 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:06.135265 19428 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:06.213405 15596 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:20:06.154668 19672 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:06.214736 15669 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:20:06.216225 15520 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:06.217113 15521 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:06.185792 19776 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:06.218479 15673 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:06.218772 16233 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
W20260709 13:20:06.223788 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 90693.7us
I20260709 13:20:06.163599 19784 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:20:06.218184 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 85344.6us
I20260709 13:20:06.243324 15568 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:06.237237 19724 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:06.240406 16239 raft_consensus.cc:1240] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 9. Current term is 10. Ops: [8.9-9.10]
I20260709 13:20:06.240453 15641 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:06.164281 19711 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:06.249647 15641 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:06.252429 15568 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:06.261656 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:06.262413 15596 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:20:06.234367 19767 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.258880 15641 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:06.264258 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:06.234112 19760 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.269179 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:06.252944 19827 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:06.270798 15520 raft_consensus.cc:1275] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:20:06.256700 19701 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:06.271009 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:06.269951 15521 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:20:06.260841 19776 raft_consensus.cc:697] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:06.274194 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:06.274319 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 112333us
I20260709 13:20:06.258659 19829 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:06.264346 19672 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:06.275871 16232 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:20:06.267503 19824 raft_consensus.cc:2764] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:20:06.276834 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:06.278573 15641 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:06.279788 15520 raft_consensus.cc:2376] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8. Re-sending same reply.
I20260709 13:20:06.280683 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:06.277581 15566 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:06.281250 15520 raft_consensus.cc:2393] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 8: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:06.281987 15566 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:06.283813 15566 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:06.284883 19825 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:06.285374 19825 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:06.285723 19825 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:06.283392 19642 consensus_queue.cc:1048] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:20:06.287624 15602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 1063) took 1244 ms. Trace:
I20260709 13:20:06.288022 15602 rpcz_store.cc:276] 0709 13:20:05.042952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:05.043877 (+   925us) service_pool.cc:224] Handling call
0709 13:20:05.070423 (+ 26546us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:05.181601 (+111178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:05.181633 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:05.181674 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:05.181709 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:06.287412 (+1105703us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b540043c954. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:06.287455 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:06.287568 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":255178624}
W20260709 13:20:06.279748 19815 log.cc:927] Time spent T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.296s	user 0.000s	sys 0.003s
I20260709 13:20:06.338380 19757 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:06.339026 19757 raft_consensus.cc:697] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:06.339910 19757 consensus_queue.cc:237] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:06.289997 19823 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:06.343891 19823 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:06.344303 19823 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:06.291538 19828 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:06.346125 19755 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:06.346728 19785 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:06.347254 19828 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:06.347671 19828 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:06.348562 19767 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:06.351501 19605 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:06.354162 19794 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:06.355974 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 2018) took 1037 ms. Trace:
I20260709 13:20:06.356431 15519 rpcz_store.cc:276] 0709 13:20:05.319024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:05.326745 (+  7721us) service_pool.cc:224] Handling call
0709 13:20:05.327067 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:05.327436 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:05.327464 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:05.328407 (+   943us) log.cc:844] Serialized 26 byte log entry
0709 13:20:05.664175 (+335768us) spinlock_profiling.cc:243] Waited 2.64 ms on lock 0x7b180023bde0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab0d1 00007f5ce49c6a0f
0709 13:20:05.826615 (+162440us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:05.838597 (+ 11982us) log.cc:844] Serialized 14 byte log entry
0709 13:20:05.855741 (+ 17144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:05.855838 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:06.229198 (+373360us) spinlock_profiling.cc:243] Waited 90.6 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:20:06.285022 (+ 55824us) spinlock_profiling.cc:243] Waited 6.21 ms on lock 0x7b4800002958. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce9934aa1 00007f5ce992bb37 00007f5ce992b9ca 00007f5ce9a2326f 00007f5ce9a22f37 00007f5ce9a3755c 00007f5ce9a37508 00007f5ce9a37488 00007f5ce9a37443 00007f5ce9a366ec 00007f5ce8960a96
0709 13:20:06.355667 (+ 70645us) raft_consensus.cc:1685] finished
0709 13:20:06.355694 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:06.355902 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":879,"prepare.queue_time_us":11840,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":22428,"raft.run_cpu_time_us":7549,"raft.run_wall_time_us":430210,"spinlock_wait_cycles":228772224,"thread_start_us":162259,"threads_started":1,"wal-append.queue_time_us":571632}
I20260709 13:20:06.358914 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:06.343919 19428 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.370342 19727 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:06.370972 19727 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.372138 19428 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:06.290077 19672 raft_consensus.cc:697] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:20:06.374928 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 91203.9us
I20260709 13:20:06.377460 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:06.381592 16240 raft_consensus.cc:2368] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9.
I20260709 13:20:06.373093 19727 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:06.366138 19711 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:06.390730 19830 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:06.407845 19830 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.409572 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:06.410277 15528 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:20:06.412035 15568 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:06.412451 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:06.413118 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:06.395437 19672 consensus_queue.cc:237] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.397097 19760 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:06.401417 19701 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:06.378968 19777 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.423774 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:06.424923 15520 raft_consensus.cc:1240] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:20:06.384994 19673 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:20:06.423269 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:06.418391 19802 raft_consensus.cc:2804] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:06.404246 19784 consensus_queue.cc:1059] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:06.405720 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:06.423812 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:06.361765 19776 consensus_queue.cc:237] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:06.425499 19777 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:06.420809 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:06.421244 15603 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:06.421375 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:06.423070 16231 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:06.407470 19711 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:20:06.423604 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:06.426275 19802 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:06.432173 19802 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:06.437053 19828 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:06.439369 16233 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:20:06.448995 19823 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:06.419001 19766 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:06.446776 19802 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.450531 19766 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.451931 19802 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:06.453044 19766 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:06.422066 19830 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:06.461287 15596 raft_consensus.cc:1240] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:06.469179 19825 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:06.472314 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:06.495139 16231 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:06.473207 15568 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:06.478396 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:06.498646 16233 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:06.499656 19825 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:06.501051 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:06.501698 15675 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:06.504717 19749 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:06.505196 19784 consensus_queue.cc:1059] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:20:06.505306 19749 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.506166 19784 raft_consensus.cc:3055] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:06.506510 19784 raft_consensus.cc:740] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:06.507402 19784 consensus_queue.cc:260] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.508821 19794 raft_consensus.cc:3055] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:06.509253 19794 raft_consensus.cc:740] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:06.510123 19794 consensus_queue.cc:260] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:06.511258 19794 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:06.511315 19828 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:06.513389 19823 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:06.514387 15602 raft_consensus.cc:1240] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:20:06.514480 19827 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:06.515082 15522 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:06.518667 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:06.519606 15669 raft_consensus.cc:2368] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7.
I20260709 13:20:06.521035 15566 leader_election.cc:400] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7.
I20260709 13:20:06.521330 15566 leader_election.cc:403] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:20:06.523684 19672 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:06.526854 19633 raft_consensus.cc:993] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9: : Instructing follower d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
W20260709 13:20:06.527098 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 54104.2us
I20260709 13:20:06.527365 19633 raft_consensus.cc:1081] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Signalling peer d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:20:06.527701 19830 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:06.530303 16235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57258 (request call id 877) took 1212 ms. Trace:
I20260709 13:20:06.531719 16231 raft_consensus.cc:1275] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:20:06.530711 16235 rpcz_store.cc:276] 0709 13:20:05.318164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:05.318322 (+   158us) service_pool.cc:224] Handling call
0709 13:20:05.333911 (+ 15589us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:05.334294 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:05.334323 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:05.346371 (+ 12048us) log.cc:844] Serialized 26 byte log entry
0709 13:20:05.629884 (+283513us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:05.645208 (+ 15324us) log.cc:844] Serialized 14 byte log entry
0709 13:20:05.781418 (+136210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:05.781490 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:06.220512 (+439022us) raft_consensus.cc:1685] finished
0709 13:20:06.220531 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:06.530108 (+309577us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b5400514350. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:06.530221 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":134809,"prepare.queue_time_us":16034,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":136505,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":162,"spinlock_wait_cycles":31334784,"thread_start_us":283100,"threads_started":1,"wal-append.queue_time_us":516884}
I20260709 13:20:06.533813 15595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 1065) took 1490 ms. Trace:
I20260709 13:20:06.534215 15595 rpcz_store.cc:276] 0709 13:20:05.043479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:05.044282 (+   803us) service_pool.cc:224] Handling call
0709 13:20:05.710530 (+666248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:05.744027 (+ 33497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:06.533622 (+789595us) spinlock_profiling.cc:243] Waited 666 ms on lock 0x7b540032b9d0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:06.533758 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1531732352}
I20260709 13:20:06.541306 16233 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:20:06.541836 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:06.542538 15601 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:20:06.538587 15568 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:06.546065 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:06.548563 19830 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:06.548952 19830 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:06.549304 19830 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:06.552321 19817 log.cc:927] Time spent T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.401s	user 0.002s	sys 0.002s
I20260709 13:20:06.552371 19748 raft_consensus.cc:993] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4: : Instructing follower d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:20:06.553620 19748 raft_consensus.cc:1081] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Signalling peer d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:20:06.557987 19784 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:06.563171 15567 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:06.563783 15601 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:20:06.564343 19785 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:06.564837 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:06.566994 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:06.571331 16231 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:06.575194 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:06.494177 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 78459.7us
I20260709 13:20:06.576256 15642 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:06.577392 15642 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:06.575371 19622 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:06.578373 19622 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.580394 15567 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:06.581874 15642 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:20:06.649330 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 67535.6us
I20260709 13:20:06.582500 19711 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:06.580439 19622 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:06.584020 16235 raft_consensus.cc:1240] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:06.584894 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:06.660326 15674 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:06.651693 16203 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:06.585569 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:06.661213 15672 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:06.586622 16240 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57258 (request call id 868) took 1332 ms. Trace:
I20260709 13:20:06.488095 19839 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:06.591256 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 52755.1us
I20260709 13:20:06.596166 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:06.616196 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1311) took 1046 ms. Trace:
W20260709 13:20:06.634656 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:06.638579 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:06.666028 15600 raft_consensus.cc:2393] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 7: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:06.639938 15602 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:06.640859 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:06.642017 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:06.645900 15675 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:06.664080 15522 rpcz_store.cc:276] 0709 13:20:05.570107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:05.570320 (+   213us) service_pool.cc:224] Handling call
0709 13:20:05.624083 (+ 53763us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:05.848514 (+224431us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b4c003d80c0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce2cd7a20 00007f5ce2ce8a29 00007f5ce2cae644 00007f5ce2cd5f2d 00007f5ce2cd5b93 00007f5ce4a0618b 00007f5ce4a06cb5 00007f5ce4a01f30 00007f5ce4a037de 00007f5ce4a0db49 00007f5ce4a0e416
0709 13:20:06.514819 (+666305us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b580080fcd0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce49670aa 00007f5ce49e9447 00007f5ce4a01fe7 00007f5ce4a037de 00007f5ce4a0db49 00007f5ce4a0e416 00007f5ce4a0ad8f 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4
0709 13:20:06.571017 (+ 56198us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:20:06.571046 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:06.615769 (+ 44723us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:06.615846 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:06.615929 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:06.616079 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":83,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79,"spinlock_wait_cycles":923033728}
I20260709 13:20:06.648000 15493 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:06.648584 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:06.671442 15596 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:20:06.673090 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:06.662878 16240 rpcz_store.cc:276] 0709 13:20:05.254588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:05.256023 (+  1435us) service_pool.cc:224] Handling call
0709 13:20:06.586427 (+1330404us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54004efa54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:06.586563 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2531829120}
I20260709 13:20:06.673699 15520 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:06.669067 15566 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:20:06.676328 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 96016.4us
I20260709 13:20:06.677269 15568 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:06.678736 15642 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:06.689899 15672 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:06.692766 15520 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:20:06.694957 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:06.695188 15642 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:06.696249 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:06.696990 15676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 1212) took 1001 ms. Trace:
I20260709 13:20:06.698889 15493 leader_election.cc:400] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11.
I20260709 13:20:06.699658 19711 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:06.699949 15522 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:56840
I20260709 13:20:06.700534 15522 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:06.700744 19672 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7.
I20260709 13:20:06.699430 15493 leader_election.cc:403] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:06.704439 15522 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:06.658944 19796 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:06.672755 19626 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:06.694597 19748 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:06.689059 19768 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:06.698642 15676 rpcz_store.cc:276] 0709 13:20:05.699471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:05.706468 (+  6997us) service_pool.cc:224] Handling call
0709 13:20:05.706708 (+   240us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:05.711841 (+  5133us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:05.711900 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:05.712788 (+   888us) log.cc:844] Serialized 26 byte log entry
0709 13:20:05.839587 (+126799us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:06.053447 (+213860us) log.cc:844] Serialized 14 byte log entry
0709 13:20:06.161931 (+108484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:06.162018 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:06.524072 (+362054us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b1800296140. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce18ec2a6 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce18ec2a6 00007f5ce18ec240
0709 13:20:06.524907 (+   835us) spinlock_profiling.cc:243] Waited 25.6 ms on lock 0x7b1800296140. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce18ec2a6 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db
0709 13:20:06.525568 (+   661us) spinlock_profiling.cc:243] Waited 1.88 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:20:06.580497 (+ 54929us) spinlock_profiling.cc:243] Waited 43.7 ms on lock 0x7b1800296140. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce191e3f3 00007f5ce19207c1 00007f5ce1920779 00007f5ce1920709 00007f5ce19206d1 00007f5ce191f9cd 000000000038f354 000000000038f199 00007f5ce190183a
0709 13:20:06.696804 (+116307us) raft_consensus.cc:1685] finished
0709 13:20:06.696829 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:06.696939 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4464,"prepare.queue_time_us":327610,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":88421,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":189,"spinlock_wait_cycles":167964288,"thread_start_us":389797,"threads_started":3,"wal-append.queue_time_us":271260}
I20260709 13:20:06.713790 16203 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:06.696317 19728 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:06.694396 19785 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:06.718395 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:06.696974 19788 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:06.720187 15520 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:20:06.698508 19823 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:06.719805 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:06.722066 15525 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:20:06.699291 19761 raft_consensus.cc:2804] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:06.726480 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:06.723729 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:06.727363 15520 raft_consensus.cc:2393] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 9: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:06.724696 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:06.724944 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:06.725785 15527 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "38190179f6494b5591027e1c41f83176"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:57300
I20260709 13:20:06.730008 15527 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:06.730378 15527 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:06.694275 19625 consensus_queue.cc:1048] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:20:06.709614 19512 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11.
I20260709 13:20:06.714758 19794 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:06.717103 19748 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:06.729013 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:06.730579 15522 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.723451 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:06.750339 15523 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:06.736199 15670 raft_consensus.cc:1240] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:06.729636 15521 raft_consensus.cc:2368] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7.
I20260709 13:20:06.693866 19839 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.716827 19711 raft_consensus.cc:697] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:06.694296 19673 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:20:06.759111 15600 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:20:06.764077 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:06.720608 19796 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:06.767849 19796 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:06.768784 19749 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:06.765769 15642 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:06.769627 15672 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:06.770095 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:06.770555 19728 raft_consensus.cc:697] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:06.771709 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:06.772446 15642 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:06.771445 19728 consensus_queue.cc:237] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.774124 19749 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:20:06.774709 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 61003.2us
I20260709 13:20:06.784392 16203 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:06.775225 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:06.775710 15522 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:06.779772 15672 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:20:06.780786 16239 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:20:06.774677 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 127544us
I20260709 13:20:06.731303 19768 raft_consensus.cc:697] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:06.716437 19829 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:20:06.779129 15527 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.795110 15523 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:20:06.752882 19748 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:06.756328 19794 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:06.749456 19823 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:06.795928 19445 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:06.823279 16204 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:06.772462 19827 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:06.805310 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:06.803107 15641 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:06.809494 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:06.810124 15527 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:06.781306 15642 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:06.816234 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:06.818822 15493 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:06.733304 19761 raft_consensus.cc:697] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:06.797254 19727 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:06.833084 15493 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:06.833174 15676 raft_consensus.cc:1240] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 13:20:06.729596 19781 consensus_queue.cc:1059] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 13:20:06.831756 15601 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:06.832074 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 50846.5us
I20260709 13:20:06.840412 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:06.803027 19776 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:06.810132 19749 raft_consensus.cc:697] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:06.844183 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:06.844444 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:06.845491 15491 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:20:06.850029 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 88149.1us
I20260709 13:20:06.815076 19794 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:06.858736 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:06.859335 16239 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:06.822782 19445 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:06.855726 15670 raft_consensus.cc:3055] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:06.803949 19788 consensus_queue.cc:1059] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:20:06.859696 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:06.871694 16233 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:06.829906 19767 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:06.859736 15491 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:06.861011 16204 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:06.859350 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:06.863055 19727 raft_consensus.cc:697] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:06.866881 15670 raft_consensus.cc:740] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:06.851301 15493 leader_election.cc:400] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9.
I20260709 13:20:06.838986 19760 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:06.836274 19777 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:06.856695 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:06.886296 16235 raft_consensus.cc:2368] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 7. Current term is 8.
I20260709 13:20:06.837124 19626 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:06.811986 19828 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:20:06.823035 19819 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:20:06.881311 16239 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
W20260709 13:20:06.883430 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 55061.9us
I20260709 13:20:06.822559 19826 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:20:06.827231 19823 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.855373 19753 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:06.860339 19781 consensus_queue.cc:1243] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer 5498ef3e89a941e39c25f68cf64e6b0f log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:20:06.858371 19776 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:06.890251 16205 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:06.866631 19512 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:06.869055 19445 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:06.889331 15670 consensus_queue.cc:260] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.919334 15670 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:06.908391 16233 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:20:06.870219 19704 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:20:06.904371 19633 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:06.856186 19761 consensus_queue.cc:237] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.868556 19830 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.853641 19711 consensus_queue.cc:237] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:06.889343 19760 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.893110 19788 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:20:06.887440 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 74024.9us
I20260709 13:20:06.907011 19656 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:06.906169 19626 raft_consensus.cc:697] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:06.911669 19757 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:20:06.948298 19626 consensus_queue.cc:237] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:06.948892 19823 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:06.891780 19754 consensus_queue.cc:1059] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:20:06.944810 15567 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:06.931463 19856 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:06.938488 19633 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.971052 16205 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:06.917620 19784 consensus_queue.cc:1059] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:20:06.887116 19768 consensus_queue.cc:237] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:06.958093 15670 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:20:06.911322 19642 consensus_queue.cc:1059] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:20:06.990130 19603 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:20:06.988399 16205 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:07.001688 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:07.002470 16239 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:07.006009 15568 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:06.987920 19857 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:06.989988 19428 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:06.989758 19856 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:07.026285 19856 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:07.035568 19830 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:07.036291 19833 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:07.036692 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:07.001621 19825 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:07.038255 19825 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:07.038661 19825 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:07.040663 19794 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:06.938968 19727 consensus_queue.cc:237] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:07.016629 16239 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:07.019364 19819 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:07.022156 19760 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:07.022343 19633 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:07.069073 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:06.939443 19749 consensus_queue.cc:237] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:07.074827 19625 raft_consensus.cc:3055] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:07.075246 19625 raft_consensus.cc:740] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:07.076059 19625 consensus_queue.cc:260] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:07.005843 19839 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:07.077473 19625 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:07.079806 19819 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:07.024474 15601 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:20:07.036749 19833 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:07.102720 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:07.044351 19776 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:07.136689 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 52287.1us
I20260709 13:20:07.080857 19428 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:20:07.145932 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 259573us
I20260709 13:20:07.238003 15493 leader_election.cc:400] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 7. Current term is 8.
I20260709 13:20:07.238343 15493 leader_election.cc:403] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:20:07.115568 19833 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:07.214339 19850 log.cc:927] Time spent T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.302s	user 0.002s	sys 0.001s
W20260709 13:20:07.235942 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:07.209800 19755 raft_consensus.cc:3055] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:07.082063 19830 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:20:07.239869 15493 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:07.242133 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 96651.3us
I20260709 13:20:07.121687 19788 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:07.247905 15493 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:07.248144 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:07.251587 15595 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:07.195176 19854 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.255707 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:07.255708 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:07.268757 16235 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:07.074764 19856 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:07.240842 19428 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:07.287631 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 409996us
I20260709 13:20:07.244621 19755 raft_consensus.cc:740] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:07.242331 19753 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 7. Current term is 8.
I20260709 13:20:07.299522 19644 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.300099 19644 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:07.300570 16231 raft_consensus.cc:3055] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:07.300990 16231 raft_consensus.cc:740] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:07.301829 16231 consensus_queue.cc:260] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:07.307807 16231 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:07.309772 19856 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:07.310719 16233 raft_consensus.cc:2368] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10.
I20260709 13:20:07.311275 19794 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:07.318126 19644 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:07.318222 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:07.292547 19428 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:07.321014 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:07.321758 15676 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:07.322606 19819 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:07.295320 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:07.324311 16203 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:07.331451 15670 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:07.335803 15595 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:20:07.337334 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:07.325244 19704 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:07.294579 19512 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:07.352344 16231 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:20:07.294749 19787 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:20:07.353490 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 58252.1us
I20260709 13:20:07.354696 15493 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:07.356815 19819 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:07.358949 19819 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:07.359511 19748 consensus_meta.cc:287] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.536s	user 0.008s	sys 0.001s
I20260709 13:20:07.359973 19748 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:07.330084 19825 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:07.361235 15601 raft_consensus.cc:1240] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:07.362250 19868 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:07.362541 19787 raft_consensus.cc:697] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:07.359378 19819 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:07.364764 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:07.365806 15527 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:07.366598 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:07.271891 19788 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:07.369316 19787 consensus_queue.cc:237] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:07.370684 15597 raft_consensus.cc:2393] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 9: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
W20260709 13:20:07.371703 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 512038us
I20260709 13:20:07.344993 19858 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:20:07.376636 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:07.378968 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 434251us
I20260709 13:20:07.380914 15567 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:07.279554 19854 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:07.386162 15527 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:07.387802 19830 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:07.388852 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:07.389520 15597 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:07.346997 19794 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:07.354851 19704 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:07.396529 15673 raft_consensus.cc:1275] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:20:07.403769 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:07.362294 19825 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:07.421408 15668 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 13:20:07.421957 15668 pending_rounds.cc:85] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting all ops after (but not including) 9
I20260709 13:20:07.422190 15668 pending_rounds.cc:107] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:20:07.422422 15668 raft_consensus.cc:2889] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:07.428900 19833 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:07.430161 15594 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:07.374904 19755 consensus_queue.cc:260] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:07.431128 19864 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:07.431633 19755 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:07.431798 19864 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:07.405545 19748 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:07.362874 19428 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:07.454542 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 73707.4us
I20260709 13:20:07.460145 15567 leader_election.cc:400] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7.
W20260709 13:20:07.462291 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 94706.8us
I20260709 13:20:07.466044 15641 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:07.402522 19704 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:07.470850 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:07.472740 19788 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:07.473140 16232 raft_consensus.cc:1240] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:07.473892 19854 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:07.477780 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:07.480370 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:07.482882 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:07.490710 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:07.492669 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:07.493556 15523 raft_consensus.cc:2393] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 9: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:07.435325 19794 raft_consensus.cc:697] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:07.500924 15672 raft_consensus.cc:1275] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:20:07.391227 19445 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:07.505313 16234 raft_consensus.cc:2368] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10.
I20260709 13:20:07.507901 15520 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:07.507581 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:07.509469 15523 pending_rounds.cc:85] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 10
I20260709 13:20:07.511835 15523 pending_rounds.cc:107] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:20:07.512120 15523 raft_consensus.cc:2889] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:07.507557 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:07.513864 16237 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:20:07.515254 15568 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:07.516369 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:07.522014 15491 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:07.511245 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:07.523689 16236 raft_consensus.cc:1240] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:20:07.521715 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:07.486785 19864 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:07.527305 19864 consensus_queue.cc:1059] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:07.527513 19880 consensus_queue.cc:1048] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:20:07.528235 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:07.528865 15519 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:07.530649 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:07.532310 16240 raft_consensus.cc:1240] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:07.533353 19445 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:07.536073 15493 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:07.546080 15519 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:07.504115 19754 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:07.562500 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 47346.7us
I20260709 13:20:07.504474 19625 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:07.504346 19788 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:07.606642 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 70784.2us
I20260709 13:20:07.608078 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:07.608122 15567 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:07.475589 19868 raft_consensus.cc:697] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:07.521153 19512 raft_consensus.cc:697] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:07.617198 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:07.628845 15526 raft_consensus.cc:3055] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:07.629345 15526 raft_consensus.cc:740] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:07.567631 19855 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.631994 19833 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:07.632203 19855 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:07.636633 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 114704us
I20260709 13:20:07.641083 15668 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:20:07.610453 19802 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:07.616675 19701 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:07.651710 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:07.653419 15642 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:07.638197 19625 raft_consensus.cc:697] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:07.649282 15526 consensus_queue.cc:260] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:07.740447 15526 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:07.668462 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:07.610679 19754 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:07.643034 19854 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:07.625934 19672 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:20:07.747350 15519 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:20:07.646786 19884 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:07.784147 15528 raft_consensus.cc:1275] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:20:07.655977 19643 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.128s
I20260709 13:20:07.617704 19788 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:07.673693 19829 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:07.658983 19642 consensus_queue.cc:1059] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.131s
I20260709 13:20:07.744933 19888 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:07.666751 19802 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:07.817716 15597 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
W20260709 13:20:07.826951 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 218930us
I20260709 13:20:07.654292 19794 consensus_queue.cc:237] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:07.836762 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 221286us
I20260709 13:20:07.808687 19753 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:07.832978 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:07.823486 19870 log.cc:927] Time spent T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.197s	user 0.000s	sys 0.000s
I20260709 13:20:07.856145 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:07.855168 15566 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:07.860289 16205 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:07.864336 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:07.865159 15597 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:07.821954 19890 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:07.791987 19868 consensus_queue.cc:237] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:07.809998 19512 consensus_queue.cc:237] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:07.792575 19884 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:07.851627 19886 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:07.859826 19428 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:07.863025 15595 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 11. Current term is 12. Ops: [10.10-11.11]
W20260709 13:20:07.924432 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 64236.6us
I20260709 13:20:07.870241 19828 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:07.936436 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:07.938304 15595 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:07.886373 19895 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:20:07.933284 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:07.932515 15597 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:07.861119 19753 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:07.953398 19749 consensus_queue.cc:1048] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:20:07.954209 19749 consensus_queue.cc:1243] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer d3c4f5bd76e0445795bad5f6dd93ff69 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 13:20:07.958602 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 61816.3us
I20260709 13:20:07.959785 16204 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:07.914430 19881 consensus_queue.cc:1048] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:20:07.924293 19893 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:07.963446 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 310112us
I20260709 13:20:07.964839 16204 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:07.968142 15595 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:07.865625 19625 consensus_queue.cc:237] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:07.969858 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:07.973603 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:07.974840 15642 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:07.975935 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:07.944051 19603 consensus_queue.cc:1059] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:20:07.979393 15525 raft_consensus.cc:1275] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:20:07.981243 15528 raft_consensus.cc:1275] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:20:07.981397 16232 raft_consensus.cc:1275] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:20:07.986814 19802 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:07.975069 19883 log.cc:927] Time spent T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.321s	user 0.001s	sys 0.000s
W20260709 13:20:07.975203 19887 log.cc:927] Time spent T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260709 13:20:07.987917 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:08.000102 19889 log.cc:927] Time spent T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.203s	user 0.000s	sys 0.002s
I20260709 13:20:07.918680 19886 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:08.002578 19892 log.cc:927] Time spent T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:20:08.003710 19860 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:20:08.004122 19886 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:07.992424 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:08.007323 15526 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:20:07.970698 19906 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:20:07.968138 19895 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:08.014873 15603 raft_consensus.cc:1275] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:20:07.966512 19888 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:08.018991 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 44355.7us
I20260709 13:20:07.971271 19428 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:07.974884 19760 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:08.021741 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:07.979086 19890 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:08.019768 19829 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.019332 15603 raft_consensus.cc:1275] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:20:08.026062 19866 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:08.027495 15526 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:07.972431 19893 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.014547 19802 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:08.020308 19754 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:08.024631 19642 consensus_queue.cc:1048] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:20:07.990500 19749 consensus_queue.cc:1059] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
W20260709 13:20:08.081314 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 71657.1us
I20260709 13:20:08.024776 19895 raft_consensus.cc:697] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:08.084694 16204 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:08.089196 19625 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.103871 19753 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:08.105051 19819 consensus_meta.cc:287] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.733s	user 0.010s	sys 0.000s
W20260709 13:20:08.082700 19704 consensus_meta.cc:287] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.577s	user 0.012s	sys 0.003s
I20260709 13:20:08.030444 19760 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.023108 19888 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.111884 19673 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:08.111145 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 86821.3us
I20260709 13:20:08.106191 19913 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:08.077109 19802 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.108903 19749 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:08.114214 19860 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:08.033654 19829 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.144981 19871 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:08.145702 19871 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.146047 19895 consensus_queue.cc:237] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.031023 19890 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:08.152475 19901 consensus_queue.cc:1048] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:20:08.131278 19673 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:08.117455 19625 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.129243 19884 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:08.143191 19749 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.127301 19704 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.125646 19819 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.143569 19642 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:08.151883 19860 raft_consensus.cc:697] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:08.199462 19754 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:08.201417 19886 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:08.203183 19754 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.203352 19886 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.205384 19754 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:08.205488 19886 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:08.202617 19915 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.138408 19913 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.206600 19915 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:08.214357 15522 raft_consensus.cc:2368] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 12. Current term is 13.
I20260709 13:20:08.159454 19901 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:08.214653 16235 raft_consensus.cc:1240] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:20:08.227448 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:08.160496 19916 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:08.206122 19905 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.201733 19642 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.249462 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:08.308041 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:08.308854 15676 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:20:08.245431 19914 raft_consensus.cc:3055] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:08.312214 15491 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:08.315266 19753 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:08.315701 19753 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:08.316099 19753 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:08.246769 19819 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:08.249624 19802 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:08.247037 19704 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:08.255064 19749 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:08.248247 19871 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:08.255052 19893 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:08.236395 19673 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:08.303131 19890 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:08.319404 19642 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:08.271369 19605 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:08.317333 19914 raft_consensus.cc:740] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:08.323644 19879 raft_consensus.cc:3055] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:08.331086 19879 raft_consensus.cc:740] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:08.331549 19914 consensus_queue.cc:260] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:08.331933 19879 consensus_queue.cc:260] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.332835 19914 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:08.333138 19879 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:08.319823 19911 log.cc:927] Time spent T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 13:20:08.248343 19915 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:08.343065 19888 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:08.344799 19920 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:08.346428 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:08.347681 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:08.348986 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:08.351823 15525 raft_consensus.cc:1275] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:20:08.354161 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:08.354859 15526 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:20:08.237026 19916 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.328338 19794 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.367076 19828 consensus_queue.cc:1048] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:08.367595 19916 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:08.283519 19905 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:08.345422 19920 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.249554 19860 consensus_queue.cc:237] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.396790 19905 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:08.402657 19760 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:08.403530 19920 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:08.355556 19673 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:08.404485 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:08.404485 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:08.233551 19901 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.330245 19829 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:08.412281 19859 raft_consensus.cc:3055] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:08.412786 19859 raft_consensus.cc:740] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:08.414631 16235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1328) took 1088 ms. Trace:
I20260709 13:20:08.415081 16235 rpcz_store.cc:276] 0709 13:20:07.326118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:07.332865 (+  6747us) service_pool.cc:224] Handling call
0709 13:20:07.333221 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:08.219723 (+886502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:08.414357 (+194634us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b5400496954. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:08.414562 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2026541824}
W20260709 13:20:08.416320 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.347580 19605 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.413697 19859 consensus_queue.cc:260] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.409992 19673 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.422519 19859 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:08.421883 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:08.427918 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:08.429198 15676 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:20:08.432931 19913 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:08.365855 19776 consensus_queue.cc:1048] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:20:08.434206 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:08.423861 19755 consensus_meta.cc:287] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.992s	user 0.007s	sys 0.006s
I20260709 13:20:08.374743 19794 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:08.439714 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:08.440398 15669 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:08.444967 15521 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:20:08.445379 15521 pending_rounds.cc:85] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 9
I20260709 13:20:08.445585 15521 pending_rounds.cc:107] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:20:08.445827 15521 raft_consensus.cc:2889] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:08.441547 15566 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:08.451241 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:20:08.455801 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.460094 19901 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:08.449473 19768 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.417s	user 0.016s	sys 0.001s
I20260709 13:20:08.473191 16232 raft_consensus.cc:1275] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:20:08.426658 19829 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:08.455214 19776 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:20:08.473623 19768 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.441s	user 0.017s	sys 0.001s
I20260709 13:20:08.499505 15596 raft_consensus.cc:1240] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11. Ops: []
W20260709 13:20:08.507705 19872 log.cc:927] Time spent T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 1.090s	user 0.004s	sys 0.002s
I20260709 13:20:08.503789 19871 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:08.512971 19871 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.515039 19871 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:08.516758 16235 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:08.495746 19605 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:08.494156 19776 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:08.519371 19901 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.478358 19829 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.533964 15674 raft_consensus.cc:1275] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:20:08.495965 19673 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:08.597728 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:08.552924 19776 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:08.606674 15669 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:08.609163 19829 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:08.613088 19625 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:08.582722 19585 consensus_queue.cc:1048] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:20:08.621016 19585 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:08.579975 19901 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.632110 19625 consensus_queue.cc:1048] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:20:08.621577 19585 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:08.638692 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:08.645273 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:08.646315 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:08.652350 16239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1326) took 1603 ms. Trace:
I20260709 13:20:08.652590 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:08.652799 16239 rpcz_store.cc:276] 0709 13:20:07.049252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:07.049562 (+   310us) service_pool.cc:224] Handling call
0709 13:20:07.049876 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:07.121246 (+ 71370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:07.121278 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:07.144364 (+ 23086us) log.cc:844] Serialized 26 byte log entry
0709 13:20:07.326132 (+181768us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:07.365698 (+ 39566us) log.cc:844] Serialized 14 byte log entry
0709 13:20:07.367520 (+  1822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:07.367587 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:08.146564 (+778977us) spinlock_profiling.cc:243] Waited 4.35 ms on lock 0x7b1800379240. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce191e3f3 00007f5ce19207c1 00007f5ce1920779 00007f5ce1920709 00007f5ce19206d1 00007f5ce191f9cd 000000000038f354 000000000038f199 00007f5ce190183a
0709 13:20:08.650446 (+503882us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b1800379240. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce191e3f3 00007f5ce19207c1 00007f5ce1920779 00007f5ce1920709 00007f5ce19206d1 00007f5ce191f9cd 000000000038f354 000000000038f199 00007f5ce190183a
0709 13:20:08.652126 (+  1680us) raft_consensus.cc:1685] finished
0709 13:20:08.652158 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:08.652290 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":112627,"prepare.queue_time_us":30752,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":80125,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":12032,"spinlock_wait_cycles":38654976,"thread_start_us":66239,"threads_started":1,"wal-append.queue_time_us":131928}
I20260709 13:20:08.654706 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:08.667299 19794 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:08.671228 16233 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:20:08.601171 19753 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:08.739755 19776 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:09.088078 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:08.918625 15596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 1963) took 2113 ms. Trace:
I20260709 13:20:09.082767 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:07.258307 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:08.917464 15595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 1106) took 1066 ms. Trace:
W20260709 13:20:09.040540 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:09.137704 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:08.883839 19901 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:08.913241 19900 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:09.093189 15596 rpcz_store.cc:276] 0709 13:20:06.804854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.805382 (+   528us) service_pool.cc:224] Handling call
0709 13:20:06.805649 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:08.499989 (+1694340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:08.918343 (+418354us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b540016b054. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:08.918526 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3895115136}
W20260709 13:20:09.122718 19882 log.cc:927] Time spent T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 1.338s	user 0.003s	sys 0.004s
I20260709 13:20:09.083503 19626 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:09.044823 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 2034) took 1555 ms. Trace:
I20260709 13:20:08.883358 19767 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:09.138198 19585 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:08.814699 19864 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:20:08.834048 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.708815 19802 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.708562 19777 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:09.036458 19891 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 1.214s	user 0.003s	sys 0.000s
I20260709 13:20:09.188755 19760 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:08.953429 19794 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:08.955219 19753 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:08.807528 19888 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:08.972672 19859 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:09.218603 19648 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:08.946973 19829 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:20:08.757710 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 299322us
I20260709 13:20:09.091416 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:08.748822 19788 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:09.186439 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:09.137868 19749 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:08.747084 19776 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:09.187690 15595 rpcz_store.cc:276] 0709 13:20:07.850852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:07.992976 (+142124us) service_pool.cc:224] Handling call
0709 13:20:08.012337 (+ 19361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:08.147907 (+135570us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:08.147934 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:08.147974 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:08.148003 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:08.917182 (+769179us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b5400439c54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:08.917234 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:08.917388 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":311313280}
W20260709 13:20:09.190505 15670 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.015s	sys 0.027s
I20260709 13:20:09.190739 15602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 2240) took 2344 ms. Trace:
W20260709 13:20:09.257072 15493 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.775s	user 0.017s	sys 0.037s
W20260709 13:20:09.200444 16205 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.026s	sys 0.017s
I20260709 13:20:09.200403 15522 rpcz_store.cc:276] 0709 13:20:07.488927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:07.489039 (+   112us) service_pool.cc:224] Handling call
0709 13:20:09.044610 (+1555571us) spinlock_profiling.cc:243] Waited 683 ms on lock 0x7b540030f554. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:09.044759 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1570690304}
W20260709 13:20:09.270650 16205 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.026s	sys 0.018s
W20260709 13:20:09.270767 15493 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.018s	sys 0.037s
W20260709 13:20:09.271025 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 744713us
I20260709 13:20:09.184387 19885 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:09.274520 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:09.220010 19626 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:09.200057 19728 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.013s	sys 0.028s
I20260709 13:20:09.279624 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:09.225644 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:09.195387 19855 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:09.218870 19605 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:09.192776 15675 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.027s	sys 0.034s
W20260709 13:20:09.285105 15675 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.027s	sys 0.034s
W20260709 13:20:09.255951 15670 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.016s	sys 0.027s
I20260709 13:20:09.259527 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:09.259793 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:09.260102 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:09.260480 15568 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.015s	sys 0.028s
I20260709 13:20:09.261142 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:09.256729 15602 rpcz_store.cc:276] 0709 13:20:06.846184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.846324 (+   140us) service_pool.cc:224] Handling call
0709 13:20:06.846679 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:06.929863 (+ 83184us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:20:06.952934 (+ 23071us) log.cc:844] Serialized 14 byte log entry
0709 13:20:07.068919 (+115985us) spinlock_profiling.cc:243] Waited 3.9 ms on lock 0x7b1800036d60. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce191e3f3 00007f5ce19207c1 00007f5ce1920779 00007f5ce1920709 00007f5ce19206d1 00007f5ce191f9cd 000000000038f354 000000000038f199 00007f5ce190183a
0709 13:20:07.144895 (+ 75976us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:07.209810 (+ 64915us) log.cc:844] Serialized 26 byte log entry
0709 13:20:07.209986 (+   176us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:07.210021 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:09.190390 (+1980369us) spinlock_profiling.cc:243] Waited 80.5 ms on lock 0x7b54001db0d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:09.190457 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:09.190508 (+    51us) raft_consensus.cc:1685] finished
0709 13:20:09.190522 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:09.190671 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12441,"prepare.queue_time_us":39905,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":50,"raft.queue_time_us":10209,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":4801,"spinlock_wait_cycles":194065408,"thread_start_us":132890,"threads_started":1,"wal-append.queue_time_us":511163,"wal-append.run_cpu_time_us":3867,"wal-append.run_wall_time_us":1486535}
I20260709 13:20:09.245522 19901 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:09.271098 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 789753us
W20260709 13:20:09.202983 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:09.248065 19830 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:08.872923 19907 log.cc:927] Time spent T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.787s	user 0.000s	sys 0.002s
W20260709 13:20:09.261201 19512 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.004s	sys 0.017s
W20260709 13:20:09.415748 19512 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.005s	sys 0.017s
W20260709 13:20:09.266870 19787 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.009s	sys 0.012s
W20260709 13:20:09.416467 19787 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.010s	sys 0.012s
I20260709 13:20:09.223599 19767 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:09.280102 15600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53324 (request call id 1962) took 2470 ms (client timeout 1682 ms). Trace:
I20260709 13:20:09.200495 19900 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:09.226063 19864 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:09.282207 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:09.226243 19802 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:09.417328 15527 raft_consensus.cc:1240] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:20:09.260248 19749 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:09.333117 15672 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:20:09.246368 19794 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:09.322021 19865 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.255916 19829 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:09.250276 19888 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:09.253490 19859 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:09.304258 19855 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:09.280076 19728 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.014s	sys 0.029s
I20260709 13:20:09.282651 19643 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:20:09.540818 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:09.306697 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:09.328274 19626 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:09.226337 19777 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:09.311276 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:09.245401 19760 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:09.305048 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:09.254299 19648 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:09.498193 19825 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:09.371368 19938 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.280743 19885 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:09.387262 19585 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:09.327308 15669 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:09.310796 19701 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:09.310763 15599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 2242) took 2063 ms. Trace:
I20260709 13:20:09.371522 19605 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:09.594759 19910 log.cc:927] Time spent T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 1.441s	user 0.003s	sys 0.000s
I20260709 13:20:09.410547 15676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 2084) took 1006 ms. Trace:
I20260709 13:20:09.560133 19865 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:09.413142 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:09.541908 19855 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:09.417243 15600 rpcz_store.cc:269] 0709 13:20:06.804502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.805024 (+   522us) service_pool.cc:224] Handling call
0709 13:20:09.274514 (+2469490us) spinlock_profiling.cc:243] Waited 641 ms on lock 0x7b540043b554. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:09.274670 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1474682752}
I20260709 13:20:09.416960 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:09.410760 19901 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:09.412195 19830 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:09.455725 15568 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.015s	sys 0.047s
W20260709 13:20:09.597674 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 978055us
W20260709 13:20:09.472579 15677 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.017s	sys 0.007s
W20260709 13:20:09.598433 15677 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.975s	user 0.019s	sys 0.007s
I20260709 13:20:09.598982 19701 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:09.602780 19585 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:09.416477 19656 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:09.515128 15493 leader_election.cc:400] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10.
I20260709 13:20:09.516664 15567 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:09.632125 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 115536us
I20260709 13:20:09.587500 15599 rpcz_store.cc:276] 0709 13:20:07.247407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:07.257594 (+ 10187us) service_pool.cc:224] Handling call
0709 13:20:07.257972 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:07.434317 (+176345us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:20:07.434597 (+   280us) log.cc:844] Serialized 14 byte log entry
0709 13:20:07.607176 (+172579us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:07.613696 (+  6520us) log.cc:844] Serialized 26 byte log entry
0709 13:20:07.613850 (+   154us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:07.613884 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:08.358415 (+744531us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b540032c3d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:08.358467 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:09.034024 (+675557us) spinlock_profiling.cc:243] Waited 29.3 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:20:09.138733 (+104709us) spinlock_profiling.cc:243] Waited 5.73 ms on lock 0x7b1800092200. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce191e522 00007f5ce19207c1 00007f5ce1920779 00007f5ce1920709 00007f5ce19206d1 00007f5ce191f9cd 000000000038f354 000000000038f199 00007f5ce190183a
0709 13:20:09.310534 (+171801us) raft_consensus.cc:1685] finished
0709 13:20:09.310556 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:09.310702 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":121240,"prepare.queue_time_us":523,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":99440,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":21201,"spinlock_wait_cycles":485276928,"thread_start_us":85593,"threads_started":1,"wal-append.queue_time_us":237769}
W20260709 13:20:09.511386 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 95074.8us
I20260709 13:20:09.520123 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:09.411484 19876 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.004s	sys 0.020s
I20260709 13:20:09.529901 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:09.635707 15597 raft_consensus.cc:2393] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 9: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:09.533286 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:09.634377 16237 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:09.636737 15523 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:20:09.540953 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:09.543998 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2094) took 2075 ms. Trace:
I20260709 13:20:09.638598 15527 rpcz_store.cc:276] 0709 13:20:07.468698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:07.468815 (+   117us) service_pool.cc:224] Handling call
0709 13:20:07.469113 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:09.418058 (+1948945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:09.543723 (+125665us) spinlock_profiling.cc:243] Waited 1.95 s on lock 0x7b540029c7d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:09.543933 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4480011008}
I20260709 13:20:09.640053 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:09.641160 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:09.480065 19860 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.946s	user 0.009s	sys 0.025s
I20260709 13:20:09.510635 19787 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.499656 19599 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.312012 15594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53324 (request call id 1967) took 1994 ms. Trace:
I20260709 13:20:09.638211 15603 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:20:09.510275 19762 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:09.515818 19757 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:09.515895 19898 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:09.643775 15594 rpcz_store.cc:276] 0709 13:20:07.317646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:07.317945 (+   299us) service_pool.cc:224] Handling call
0709 13:20:09.311880 (+1993935us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b540016b554. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:09.311964 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":255929600}
I20260709 13:20:09.517795 19785 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:09.573676 15598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 1097) took 1740 ms. Trace:
I20260709 13:20:09.668226 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:09.668579 15598 rpcz_store.cc:276] 0709 13:20:07.832927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:07.834085 (+  1158us) service_pool.cc:224] Handling call
0709 13:20:07.834448 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:07.834704 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:07.834731 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:07.852330 (+ 17599us) log.cc:844] Serialized 26 byte log entry
0709 13:20:07.973476 (+121146us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:07.973510 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:07.973571 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:08.872294 (+898723us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:20:09.562878 (+690584us) raft_consensus.cc:1685] finished
0709 13:20:09.562900 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:09.573592 (+ 10692us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42560768,"thread_start_us":78838,"threads_started":1,"wal-append.queue_time_us":159848}
I20260709 13:20:09.518023 19869 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:09.584237 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:09.584765 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:09.579849 19825 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:09.413825 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:09.414525 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:09.621400 16239 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:09.623261 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:09.624133 15675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 1253) took 1035 ms. Trace:
I20260709 13:20:09.624300 19855 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:09.622002 19605 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:09.624650 19749 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:09.625056 19585 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:09.625288 19829 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:09.625551 15568 leader_election.cc:400] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10.
I20260709 13:20:09.721055 15568 leader_election.cc:403] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:20:09.626701 19830 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:09.595255 15676 rpcz_store.cc:276] 0709 13:20:08.404002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.404198 (+   196us) service_pool.cc:224] Handling call
0709 13:20:09.396753 (+992555us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b540042b654. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:09.410474 (+ 13721us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51726976}
I20260709 13:20:09.627305 19760 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:09.627564 19802 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:09.628010 19885 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:09.628479 19864 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:09.625487 19901 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:09.629160 19935 log.cc:927] Time spent T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 13:20:09.622310 15670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2421) took 1195 ms. Trace:
I20260709 13:20:09.629606 16234 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:20:09.619987 15596 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:20:09.621989 19767 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:09.633497 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:09.634706 15493 leader_election.cc:403] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:20:09.635329 19876 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.004s	sys 0.020s
I20260709 13:20:09.585844 19938 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:09.645601 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:09.788733 15599 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:09.516160 19512 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:09.578490 19944 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.853138 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:09.782727 19830 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:09.686496 15600 raft_consensus.cc:2393] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 9: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
W20260709 13:20:09.723834 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:09.667609 19762 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:09.686010 15675 rpcz_store.cc:276] 0709 13:20:08.588375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.589815 (+  1440us) service_pool.cc:224] Handling call
0709 13:20:08.603736 (+ 13921us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:08.607953 (+  4217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:08.607986 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:08.608465 (+   479us) log.cc:844] Serialized 26 byte log entry
0709 13:20:09.285555 (+677090us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:09.285600 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:09.285675 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:09.623922 (+338247us) raft_consensus.cc:1685] finished
0709 13:20:09.623957 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:09.624077 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3232,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"thread_start_us":676753,"threads_started":1,"wal-append.queue_time_us":962806}
W20260709 13:20:09.643707 19860 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.109s	user 0.009s	sys 0.026s
I20260709 13:20:09.700346 19785 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:09.682278 19787 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:09.734256 19855 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:09.699252 19898 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:09.722802 19869 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:09.700094 19757 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:09.748075 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:09.748754 15521 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:20:09.749437 19648 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:09.750563 19829 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.752532 15670 rpcz_store.cc:276] 0709 13:20:08.427246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.427543 (+   297us) service_pool.cc:224] Handling call
0709 13:20:08.427841 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:08.428214 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:20:08.455011 (+ 26797us) log.cc:844] Serialized 14 byte log entry
0709 13:20:09.381393 (+926382us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:09.388862 (+  7469us) log.cc:844] Serialized 26 byte log entry
0709 13:20:09.389006 (+   144us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:09.389039 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:09.389106 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:09.622065 (+232959us) raft_consensus.cc:1685] finished
0709 13:20:09.622093 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:09.622249 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":576493,"prepare.queue_time_us":44069,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":280233,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":9047,"thread_start_us":792849,"threads_started":1,"wal-append.queue_time_us":1066825}
I20260709 13:20:09.897899 15520 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:20:09.637319 19656 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:09.637795 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:09.830664 16235 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.014s	sys 0.018s
W20260709 13:20:09.904640 16235 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.015s	sys 0.019s
W20260709 13:20:09.789188 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 274213us
I20260709 13:20:09.849808 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:09.834018 15527 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:20:09.850337 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:09.851559 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:09.684125 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 403097us
I20260709 13:20:09.853811 16238 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:09.685555 15601 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:09.788313 19585 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:09.856690 15672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2429) took 1190 ms. Trace:
I20260709 13:20:09.784271 19859 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:09.921232 15602 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:20:09.921319 15523 raft_consensus.cc:1275] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:20:09.923559 15520 raft_consensus.cc:1275] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:20:09.924919 15595 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
W20260709 13:20:09.850423 19622 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.019s	sys 0.007s
I20260709 13:20:09.860792 15669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57710 (request call id 1262) took 1214 ms. Trace:
I20260709 13:20:09.829746 19864 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10.
I20260709 13:20:09.863713 19830 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10.
I20260709 13:20:09.683614 19599 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:09.940946 19599 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:09.865451 15525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 2054) took 1430 ms. Trace:
I20260709 13:20:09.941990 19753 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:09.942587 19753 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:09.944553 19753 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:09.866088 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:09.945984 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:09.947077 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:09.947976 19946 log.cc:927] Time spent T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
I20260709 13:20:09.948455 16233 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:20:09.866283 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:09.949702 19938 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:09.854688 19950 log.cc:927] Time spent T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.101s	user 0.000s	sys 0.005s
I20260709 13:20:09.867695 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:09.870402 15492 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.011s	sys 0.023s
W20260709 13:20:09.950737 15492 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.014s	sys 0.023s
I20260709 13:20:09.851481 19605 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.951447 19787 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:09.951639 19605 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:09.941498 15525 rpcz_store.cc:276] 0709 13:20:08.435343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.435549 (+   206us) service_pool.cc:224] Handling call
0709 13:20:08.435836 (+   287us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:08.436235 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:08.436261 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:08.436860 (+   599us) log.cc:844] Serialized 52 byte log entry
0709 13:20:09.091677 (+654817us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:09.538084 (+446407us) log.cc:844] Serialized 14 byte log entry
0709 13:20:09.538321 (+   237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:09.538392 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:09.865217 (+326825us) raft_consensus.cc:1685] finished
0709 13:20:09.865235 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:09.865391 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":650003,"prepare.run_cpu_time_us":299,"prepare.run_wall_time_us":1454,"raft.queue_time_us":7798,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":27265,"thread_start_us":648276,"threads_started":2,"wal-append.queue_time_us":920164}
I20260709 13:20:09.854339 19737 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.859845 19944 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:09.954401 19898 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:09.954191 19737 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:09.956265 19737 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:09.891966 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:09.953743 19605 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:09.957655 19599 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:09.958238 19599 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:09.891103 15491 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:20:09.906112 15673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49100 (request call id 2422) took 1478 ms (client timeout 1934 ms). Trace:
I20260709 13:20:09.906862 15521 pending_rounds.cc:85] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 3
I20260709 13:20:09.918066 15527 raft_consensus.cc:1275] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:20:09.918799 15672 rpcz_store.cc:276] 0709 13:20:08.666761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:09.516873 (+850112us) service_pool.cc:224] Handling call
0709 13:20:09.855599 (+338726us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:09.855757 (+   158us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:09.855778 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:09.855814 (+    36us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:09.855841 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:09.855903 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:09.856529 (+   626us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b5400429350. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:09.856622 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":680275328}
I20260709 13:20:09.920523 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:09.928122 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:09.888149 19888 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:09.933670 15526 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.012s	sys 0.018s
I20260709 13:20:09.960961 15521 pending_rounds.cc:107] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:20:09.962849 15521 raft_consensus.cc:2889] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:09.968080 16238 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:09.969156 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:09.970218 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:09.960301 15673 rpcz_store.cc:269] 0709 13:20:08.427540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.427662 (+   122us) service_pool.cc:224] Handling call
0709 13:20:09.849695 (+1422033us) spinlock_profiling.cc:243] Waited 870 us on lock 0x7b5800012398. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6eee 0000000000375939 00007f5ce995a1e0 00007f5ce9a230c4 00007f5ce9a23131 00007f5ce99d10a0 00007f5ce99d0984 00007f5ce2f4b6d4 00007f5ce2f4b662
0709 13:20:09.906051 (+ 56356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2001152}
I20260709 13:20:09.856031 19512 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:09.864918 19794 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:09.929195 15493 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:09.934620 15669 rpcz_store.cc:276] 0709 13:20:08.641628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.642557 (+   929us) service_pool.cc:224] Handling call
0709 13:20:09.855478 (+1212921us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b5400429354. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:09.855608 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":539164416}
I20260709 13:20:09.935871 16205 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:09.970319 19753 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:09.974607 15674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2431) took 1177 ms. Trace:
W20260709 13:20:09.960412 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 69395.5us
I20260709 13:20:09.893033 19855 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:09.973845 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 45327.7us
I20260709 13:20:09.981201 15674 rpcz_store.cc:276] 0709 13:20:08.797230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:09.527219 (+729989us) service_pool.cc:224] Handling call
0709 13:20:09.527955 (+   736us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:09.567373 (+ 39418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:09.567403 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:09.567451 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:09.567483 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:09.974372 (+406889us) spinlock_profiling.cc:243] Waited 39.2 ms on lock 0x7b5400186854. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:09.974420 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:09.974548 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90094976}
I20260709 13:20:09.982286 15493 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
W20260709 13:20:09.981463 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 45693.5us
W20260709 13:20:09.977558 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 440963us
W20260709 13:20:09.936329 19622 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.020s	sys 0.007s
W20260709 13:20:09.962364 15526 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.012s	sys 0.018s
I20260709 13:20:09.979645 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:09.983655 15671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2434) took 1122 ms. Trace:
I20260709 13:20:09.941803 19757 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:09.984041 19794 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.922320 19885 consensus_queue.cc:1059] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:20:09.935016 19859 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:09.988540 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:09.971289 19949 log.cc:927] Time spent T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:20:09.989305 15599 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:09.990136 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:10.012540 15670 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:09.991019 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:10.014816 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:10.015028 19944 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:09.987182 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:10.016218 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:10.017331 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:09.991753 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:09.985616 15493 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:09.994488 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:10.020673 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:09.995914 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:09.943122 19762 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:09.983762 19626 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:09.989737 19737 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:09.917783 19829 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:10.002071 15672 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:10.027032 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:09.978870 19888 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:10.020562 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:10.021492 15673 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:20:09.928580 19585 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:10.021898 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:09.999334 15642 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:09.989158 19767 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.026063 15523 raft_consensus.cc:2410] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:20:10.027758 16236 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
W20260709 13:20:10.028324 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:09.999701 19787 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:09.991117 19825 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:09.991544 19749 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:09.999820 19753 raft_consensus.cc:697] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:09.998536 19864 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:10.032413 15566 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:10.035239 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:10.037073 16203 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:10.032255 19951 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.043043 15675 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:10.031162 19626 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:10.044626 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:09.990722 19794 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:10.046954 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:10.044927 15677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2427) took 1423 ms. Trace:
I20260709 13:20:10.021468 19859 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:10.048447 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 415267us
W20260709 13:20:10.123555 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:10.040251 19787 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:10.052496 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:10.212721 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:10.069052 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 443620us
W20260709 13:20:10.123915 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 127028us
I20260709 13:20:10.329043 16234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 953) took 1023 ms. Trace:
W20260709 13:20:10.401327 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 347956us
W20260709 13:20:10.381521 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 358564us
I20260709 13:20:10.340606 15672 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:20:10.154577 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 155434us
I20260709 13:20:10.330523 15524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1361) took 1023 ms. Trace:
I20260709 13:20:10.420991 15524 rpcz_store.cc:276] 0709 13:20:09.307800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:09.307974 (+   174us) service_pool.cc:224] Handling call
0709 13:20:09.308260 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:09.619263 (+311003us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:09.619304 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:09.619367 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:09.619416 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:10.330288 (+710872us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b54002fa8d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:10.330337 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:10.330463 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":658657536}
W20260709 13:20:10.239967 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 77107.7us
I20260709 13:20:10.161288 15677 rpcz_store.cc:276] 0709 13:20:08.621553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.621835 (+   282us) service_pool.cc:224] Handling call
0709 13:20:08.622188 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:08.622510 (+   322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:08.622534 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:08.622958 (+   424us) log.cc:844] Serialized 26 byte log entry
0709 13:20:09.619702 (+996744us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:09.619738 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:09.619804 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:10.044683 (+424879us) raft_consensus.cc:1685] finished
0709 13:20:10.044703 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:10.044843 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4577,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":86,"thread_start_us":975519,"threads_started":1,"wal-append.queue_time_us":1110994}
W20260709 13:20:10.415995 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 383638us
W20260709 13:20:10.424932 15424 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49132 (request call id 339) took 32973 ms (client timeout 15000 ms). Trace:
I20260709 13:20:10.425310 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:10.420444 15493 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:10.418548 15491 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:10.427614 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:10.040616 19964 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:10.133733 19963 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:10.335872 19974 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:10.068485 19829 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:10.084393 19954 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:10.443591 19738 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:10.439023 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:10.331043 19975 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.135s
I20260709 13:20:10.418937 19855 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:10.044281 19599 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:10.045631 19767 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:10.678773 16235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53976 (request call id 942) took 1421 ms. Trace:
W20260709 13:20:10.711419 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 292949us
I20260709 13:20:10.716441 19760 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:10.446079 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:10.405803 19777 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:10.137640 19512 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:10.727901 16203 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:10.423195 19871 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:10.668627 19976 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.457190 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:10.541941 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:10.715633 19885 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.008s	sys 0.017s
W20260709 13:20:10.539894 15526 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.017s	sys 0.013s
I20260709 13:20:10.433178 15567 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
W20260709 13:20:10.737093 15526 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.026s	sys 0.014s
I20260709 13:20:10.738993 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:10.739056 15491 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:20:10.739181 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:10.739853 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 306763us
I20260709 13:20:10.092068 19585 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:10.488965 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 64392us
I20260709 13:20:10.456737 16237 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:20:10.492799 19876 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.052273 19951 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:10.119103 19901 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:10.577306 19622 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.001s	sys 0.032s
I20260709 13:20:10.419406 16234 rpcz_store.cc:276] 0709 13:20:09.305841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:09.306456 (+   615us) service_pool.cc:224] Handling call
0709 13:20:10.328819 (+1022363us) spinlock_profiling.cc:243] Waited 53.4 ms on lock 0x7b540072e2d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:10.328965 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":122780928}
W20260709 13:20:10.626119 15569 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.026s	sys 0.021s
I20260709 13:20:10.594645 15595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53324 (request call id 1974) took 1180 ms. Trace:
I20260709 13:20:10.429669 19787 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:10.122591 19445 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:10.118896 19856 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:10.154376 19753 consensus_queue.cc:237] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:10.593253 19940 log.cc:927] Time spent T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:20:10.716161 16235 rpcz_store.cc:276] 0709 13:20:09.257261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:09.258163 (+   902us) service_pool.cc:224] Handling call
0709 13:20:09.258521 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:09.905080 (+646559us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:09.905109 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:09.930534 (+ 25425us) log.cc:844] Serialized 26 byte log entry
0709 13:20:10.425414 (+494880us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:10.425451 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:10.425508 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:10.678524 (+253016us) raft_consensus.cc:1685] finished
0709 13:20:10.678548 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:10.678707 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18694,"raft.queue_time_us":46890,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":74,"thread_start_us":1054332,"threads_started":2,"wal-append.queue_time_us":660122}
I20260709 13:20:10.054898 19785 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:10.058751 19869 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:10.734117 15566 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:10.065326 19701 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:10.446812 19895 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:10.430362 19625 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:10.545977 19794 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:10.308140 19962 consensus_queue.cc:1048] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.440s
I20260709 13:20:10.718516 19964 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:10.829608 19964 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:10.770994 19901 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:10.573554 19900 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:10.720695 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 300328us
I20260709 13:20:10.725677 19738 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:10.832552 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:10.070624 19927 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:10.425117 19656 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:10.836650 15493 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:10.766911 19879 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:20:10.446472 15642 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:10.723989 19760 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:10.570099 19865 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:10.744436 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1366) took 1418 ms. Trace:
I20260709 13:20:10.768214 15677 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:20:10.752517 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:10.768585 15595 rpcz_store.cc:276] 0709 13:20:09.414121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:09.414283 (+   162us) service_pool.cc:224] Handling call
0709 13:20:10.594427 (+1180144us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b540025b054. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:10.594573 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":545273728}
W20260709 13:20:10.737104 19885 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.008s	sys 0.018s
I20260709 13:20:10.846259 15526 rpcz_store.cc:276] 0709 13:20:09.326051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:09.326192 (+   141us) service_pool.cc:224] Handling call
0709 13:20:09.326495 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:09.326695 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:20:09.986144 (+659449us) log.cc:844] Serialized 14 byte log entry
0709 13:20:10.744025 (+757881us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:10.744163 (+   138us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:10.744213 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:10.744281 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:10.744379 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6312,"prepare.queue_time_us":540028,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":11950,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":22506,"thread_start_us":1408327,"threads_started":2,"wal-append.queue_time_us":857829}
W20260709 13:20:10.768143 15569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.108s	user 0.034s	sys 0.028s
I20260709 13:20:10.770468 15567 leader_election.cc:400] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 12. Current term is 13.
I20260709 13:20:10.052759 19864 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:10.788374 15568 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:10.735441 19605 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:09.986526 15671 rpcz_store.cc:276] 0709 13:20:08.859450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:09.573322 (+713872us) service_pool.cc:224] Handling call
0709 13:20:09.573690 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:09.573939 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:20:09.615692 (+ 41753us) log.cc:844] Serialized 14 byte log entry
0709 13:20:09.981208 (+365516us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:09.981262 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:09.981316 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:09.981388 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:09.981524 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1135,"prepare.queue_time_us":55510,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":6743,"raft.queue_time_us":4281,"thread_start_us":349266,"threads_started":1}
I20260709 13:20:10.745869 19830 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:20:10.749734 19622 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.001s	sys 0.041s
I20260709 13:20:10.794203 19445 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:10.526227 19970 consensus_queue.cc:1059] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.276s
I20260709 13:20:10.842082 19927 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:10.790608 19856 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:10.800490 19512 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:10.828876 19626 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:10.790896 19906 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:10.840095 19895 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:10.069855 19749 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:10.718292 19963 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:10.742424 19985 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:10.765017 19876 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:10.843976 19777 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:20:10.882658 19979 log.cc:927] Time spent T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 13:20:10.883001 19777 raft_consensus.cc:697] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:10.883641 19895 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:10.883958 19777 consensus_queue.cc:237] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:10.885236 19767 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:10.886544 19876 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:10.832103 19869 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:10.876770 19906 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:10.837216 19975 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:10.888075 19975 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:10.889081 19906 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:10.837800 19859 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:10.743922 19976 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:10.871124 19927 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:10.848197 19760 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:10.847923 19900 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:10.856431 19785 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.859227 19830 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:10.887502 19738 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:10.425549 15424 rpcz_store.cc:269] 0709 13:19:37.451773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.451942 (+   169us) service_pool.cc:224] Handling call
0709 13:20:10.424837 (+32972895us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:01.422156 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:01.422879 (+   723us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:01.422935 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:20:01.422957 (+    22us) tablet.cc:662] Decoding operations
0709 13:20:01.424226 (+  1269us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:01.424336 (+   110us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:01.424367 (+    31us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:20:01.425732 (+  1365us) tablet.cc:701] Row locks acquired
0709 13:20:01.425790 (+    58us) write_op.cc:260] PREPARE: finished
0709 13:20:01.448905 (+ 23115us) write_op.cc:270] Start()
0709 13:20:01.474533 (+ 25628us) write_op.cc:276] Timestamp: P: 1783603201428900 usec, L: 0
0709 13:20:01.477115 (+  2582us) op_driver.cc:348] REPLICATION: starting
0709 13:20:01.558266 (+ 81151us) log.cc:844] Serialized 5104 byte log entry
0709 13:20:02.038250 (+479984us) op_driver.cc:464] REPLICATION: finished
0709 13:20:02.183652 (+145402us) write_op.cc:301] APPLY: starting
0709 13:20:02.220933 (+ 37281us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:02.388852 (+167919us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:02.388967 (+   115us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:02.483809 (+ 94842us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:02.484197 (+   388us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=0,mrs_lookups=0
0709 13:20:02.484248 (+    51us) write_op.cc:312] APPLY: finished
0709 13:20:02.484659 (+   411us) log.cc:844] Serialized 121 byte log entry
0709 13:20:02.512331 (+ 27672us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:02.512479 (+   148us) write_op.cc:454] Released schema lock
0709 13:20:02.528613 (+ 16134us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:02.981460 (+452847us) spinlock_profiling.cc:243] Waited 8.42 ms on lock 0x7b54003693a8. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ceadc79e1 00007f5ceadc664c 00007f5ce84573fd 00007f5ce8443389 00007f5ce8442b53 00007f5ce8446d61 00007f5ce8446d19 00007f5ce8446ca9 00007f5ce8446c71 00007f5ce8445e9d 000000000038f354
0709 13:20:02.993379 (+ 11919us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:03.002588 (+  9209us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":204563,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":127503,"apply.run_cpu_time_us":26974,"apply.run_wall_time_us":807093,"cfile_cache_hit":50,"cfile_cache_hit_bytes":52350,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6221,"cfile_init":1,"lbm_read_time_us":43256,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":6,"num_ops":13,"prepare.queue_time_us":115814,"prepare.run_cpu_time_us":12850,"prepare.run_wall_time_us":252201,"raft.queue_time_us":173070,"raft.run_cpu_time_us":40165,"raft.run_wall_time_us":94377,"replication_time_us":560938,"spinlock_wait_cycles":19376000,"thread_start_us":302700,"threads_started":4,"wal-append.queue_time_us":169874,"wal-append.run_cpu_time_us":29670,"wal-append.run_wall_time_us":1816487}]]}
I20260709 13:20:11.003026 19753 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:11.004199 19753 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:10.802767 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 68732.3us
I20260709 13:20:11.005117 19753 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.008805 19753 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:10.851044 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 80718.4us
I20260709 13:20:11.013938 19762 raft_consensus.cc:2764] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 13:20:11.018350 19762 raft_consensus.cc:3055] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:11.019027 19762 raft_consensus.cc:740] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:10.902388 19785 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:11.025309 19785 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:10.951836 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 115141us
I20260709 13:20:10.953835 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2100) took 1441 ms. Trace:
I20260709 13:20:10.871711 19856 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:10.957157 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 168715us
I20260709 13:20:11.026170 15521 rpcz_store.cc:276] 0709 13:20:09.512304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:09.905505 (+393201us) service_pool.cc:224] Handling call
0709 13:20:09.906679 (+  1174us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:09.984300 (+ 77621us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:09.984333 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:09.991321 (+  6988us) log.cc:844] Serialized 52 byte log entry
0709 13:20:10.417792 (+426471us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:20:10.417832 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:10.417900 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:10.952867 (+534967us) raft_consensus.cc:1685] finished
0709 13:20:10.953021 (+   154us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:10.953502 (+   481us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13468,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":7341,"thread_start_us":387447,"threads_started":1,"wal-append.queue_time_us":746781}
W20260709 13:20:10.843850 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 397436us
I20260709 13:20:10.958690 15668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2433) took 2141 ms. Trace:
W20260709 13:20:10.958879 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:10.745609 15491 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:11.031356 19749 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:11.032529 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 287030us
I20260709 13:20:11.030098 15668 rpcz_store.cc:276] 0709 13:20:08.817580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:09.536063 (+718483us) service_pool.cc:224] Handling call
0709 13:20:09.539219 (+  3156us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:20:09.972034 (+432815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:09.972065 (+    31us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:20:09.991972 (+ 19907us) log.cc:844] Serialized 78 byte log entry
0709 13:20:10.382820 (+390848us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:10.382867 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:10.382938 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:10.958032 (+575094us) raft_consensus.cc:1685] finished
0709 13:20:10.958072 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:10.958442 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":195172,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":10220,"thread_start_us":790804,"threads_started":2,"wal-append.queue_time_us":631689}
I20260709 13:20:10.961728 15670 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:11.034876 19626 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:11.035795 19626 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:10.957243 15641 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.883s (SENT) [suppressed 3 similar messages]
W20260709 13:20:11.038655 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 81620.1us
I20260709 13:20:11.039932 19626 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:10.970973 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 92388.3us
I20260709 13:20:11.009002 15566 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:11.047003 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:11.048188 19585 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:11.048849 19585 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:11.049273 15566 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:11.049245 19585 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:10.993104 19445 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:10.875125 19512 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:11.053351 19626 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:11.053526 19512 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:11.054152 19626 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.031435 15642 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:11.056944 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:11.057446 19794 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:10.831957 19625 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.057948 19794 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:11.058518 19794 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:11.060201 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:11.061004 19625 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:10.998773 19701 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:10.902586 19830 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:11.061748 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:11.062742 15671 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:11.062994 19701 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:11.063426 19701 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:11.063831 19701 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:11.064411 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:11.033061 19767 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:11.079706 15671 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:20:11.065263 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:10.960705 15673 raft_consensus.cc:2410] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:20:11.081293 15668 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:11.087602 19830 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:11.068188 15493 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:11.070240 15568 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:11.072846 19777 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:11.074785 19626 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:11.020977 19762 consensus_queue.cc:260] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.077435 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:11.066250 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:11.094347 16203 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:11.102262 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:10.887580 19869 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:11.088806 19856 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:10.915910 19888 raft_consensus.cc:3055] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:10.895174 19900 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:11.139525 19900 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:10.904583 19901 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:10.919456 19927 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:10.913990 19951 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:10.917516 19864 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:10.920491 19964 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:11.226850 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 158794us
I20260709 13:20:10.836644 19968 consensus_queue.cc:1048] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:20:10.890993 19963 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:10.722316 19974 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.234738 19974 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:10.905586 19975 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:11.235100 19987 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:10.906145 19976 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:11.235802 19987 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
W20260709 13:20:10.919008 19977 log.cc:927] Time spent T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.056s	user 0.002s	sys 0.000s
I20260709 13:20:11.105080 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:11.236325 19987 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:11.093274 19701 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:11.100239 19777 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:11.093046 19794 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.104172 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:11.118383 15673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57710 (request call id 1273) took 1066 ms. Trace:
I20260709 13:20:11.112062 19585 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.136108 16235 raft_consensus.cc:2393] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 13: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:11.157210 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:11.158210 19996 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:11.158874 19888 raft_consensus.cc:740] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:11.159801 19760 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.160329 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:11.160948 19869 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:11.162326 19856 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:11.139583 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:11.166078 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:11.166366 19991 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:11.167403 19830 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:11.167938 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:11.173617 19762 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:11.228619 19964 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:11.208101 19876 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:11.211817 19927 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:11.212332 19978 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:11.218628 15672 raft_consensus.cc:2393] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 11: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:20:11.228731 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:11.237319 15671 raft_consensus.cc:1240] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:11.228936 15493 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:11.241111 15673 rpcz_store.cc:276] 0709 13:20:10.048222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.049476 (+  1254us) service_pool.cc:224] Handling call
0709 13:20:11.106146 (+1056670us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b540021aa54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:11.114699 (+  8553us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1247339264}
W20260709 13:20:11.195171 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 100966us
I20260709 13:20:11.261232 19760 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:11.253702 19991 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:11.240886 19777 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:11.213640 19787 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.267297 16203 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:11.272171 16234 raft_consensus.cc:2393] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 13: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:11.274811 15677 raft_consensus.cc:1240] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 13:20:11.279505 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 50787.8us
I20260709 13:20:11.280563 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:11.281234 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:11.283780 19992 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:11.284473 19978 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.271235 19991 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:11.287461 19978 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:11.288203 19978 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:11.253587 19996 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:11.290315 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 220200us
I20260709 13:20:11.301944 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:11.304188 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:11.303535 19738 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:11.310341 19738 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:11.311784 19738 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:11.310850 16203 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:11.303714 19976 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:11.319888 15520 raft_consensus.cc:1240] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:11.322293 19888 consensus_queue.cc:260] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.324041 19888 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:11.284672 19992 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.328064 19992 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:11.328995 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.330344 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.331269 15523 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:20:11.331808 19927 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:11.316823 19976 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:11.332444 19927 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.333103 19976 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:11.331568 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.338101 15595 raft_consensus.cc:1240] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 13:20:11.338464 16233 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:11.339800 15493 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:11.340492 19787 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:11.343382 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.344305 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.344745 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.345497 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:11.346019 19998 log.cc:927] Time spent T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:20:11.343426 15493 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:11.346135 15601 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:11.347767 15601 raft_consensus.cc:1275] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:20:11.348245 15603 raft_consensus.cc:1240] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:11.349292 19787 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:11.349756 19787 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:11.346159 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.350724 19794 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:11.351063 19760 raft_consensus.cc:2804] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:11.351442 19760 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:11.351780 19760 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:11.352821 19996 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:11.353487 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:11.354614 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 56494.1us
I20260709 13:20:11.353989 19585 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:11.368775 16203 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:11.339713 19701 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:11.354703 19787 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:11.376535 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.377312 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.362947 19856 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:11.382084 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:11.381695 15599 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:20:11.386004 15601 raft_consensus.cc:2393] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 14: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:11.355317 19512 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:11.387935 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.388669 15528 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:20:11.389071 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:11.388487 19724 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:11.391738 16239 raft_consensus.cc:2393] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 11: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:11.391572 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:11.379345 19856 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:11.395747 19856 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:11.396575 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.396932 19991 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.398227 15519 raft_consensus.cc:1240] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:11.399220 16233 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:11.405441 15670 raft_consensus.cc:2393] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 11: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:11.406517 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.377987 19927 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:11.415359 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.420150 15599 raft_consensus.cc:2393] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 11: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:11.422000 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.431551 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.432245 15521 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:20:11.433179 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.434587 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.435428 15521 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:11.436249 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:11.437341 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.437582 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.438268 15602 raft_consensus.cc:2393] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 11: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:11.441135 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.442764 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.447415 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.448853 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.452014 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.452564 15597 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:20:11.452301 19777 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.452816 15520 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:11.454561 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.457141 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:11.457892 15670 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:11.458581 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:11.459980 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:11.462353 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:11.462780 19738 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.468158 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.469158 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:11.469483 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.470623 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:11.471127 15596 raft_consensus.cc:1275] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:20:11.472076 15520 raft_consensus.cc:1240] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:11.465106 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.437207 15528 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:11.461117 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:11.463044 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:11.474258 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:11.479943 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.479771 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:11.462484 19760 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.483839 15595 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:20:11.484407 19760 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:11.484679 15602 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:11.485008 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.486279 19738 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:11.487594 16238 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 15: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:11.489022 15596 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:20:11.490082 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.490981 15596 raft_consensus.cc:2393] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 16: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:11.491780 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:11.502657 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.506044 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.516455 16205 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:11.508983 19445 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:11.527189 15670 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:11.528522 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.545282 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.551705 15528 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:20:11.555065 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:11.524566 19900 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.563612 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:11.565332 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.567615 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.551945 19777 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:11.553725 19724 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:11.534935 19445 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.574512 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.576386 19976 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.577301 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.579311 19879 consensus_queue.cc:1059] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:20:11.586136 15596 raft_consensus.cc:2368] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9.
I20260709 13:20:11.587400 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.588068 15595 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
W20260709 13:20:11.596390 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 80197.4us
I20260709 13:20:11.600847 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.603281 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.570101 19991 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:11.605587 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.609020 15597 raft_consensus.cc:2393] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 9: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:11.606870 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.624576 19445 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:11.626466 19787 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.627342 15525 raft_consensus.cc:1240] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:11.628183 16205 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:11.629351 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.630815 19665 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:11.595158 19512 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:11.633988 19900 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:11.599862 19869 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:11.642433 16233 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:20:11.647452 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:11.651386 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:11.652439 15596 raft_consensus.cc:2393] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 11: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:11.653259 15642 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:11.651723 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.656038 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.659600 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:11.662120 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 244208us
I20260709 13:20:11.663931 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.652426 19762 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:11.655262 19749 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:11.670220 15598 raft_consensus.cc:2393] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 10: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:11.674054 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.675637 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:11.676584 20013 log.cc:927] Time spent T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:20:11.676826 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.678037 19749 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:11.689313 19749 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:11.678414 19787 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:11.681720 19210 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:11.694492 19210 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.695200 19869 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:11.696079 15600 raft_consensus.cc:2410] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:20:11.684737 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:11.686005 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:11.686583 19762 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.696516 19976 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:11.703934 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:11.704272 19762 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:11.670338 19701 consensus_queue.cc:1059] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:20:11.689210 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.706696 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.707731 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.683930 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:11.696570 19210 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:11.697149 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:11.678882 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 50825.1us
I20260709 13:20:11.698871 15595 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:20:11.638443 19987 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.660535 19879 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:11.714839 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:11.715843 19762 raft_consensus.cc:3055] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:11.717057 16205 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:11.718292 19856 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.723430 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.722473 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.724045 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.726320 15597 raft_consensus.cc:2393] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 11: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:11.726739 15671 raft_consensus.cc:2368] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9.
I20260709 13:20:11.722494 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:11.732144 16239 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:11.734501 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.725309 19210 consensus_queue.cc:1059] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:20:11.737668 19210 raft_consensus.cc:3055] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:11.738071 19210 raft_consensus.cc:740] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:11.738996 19210 consensus_queue.cc:260] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.740386 19210 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:11.740695 15672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57710 (request call id 1280) took 1003 ms. Trace:
I20260709 13:20:11.741045 15672 rpcz_store.cc:276] 0709 13:20:10.736905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.737066 (+   161us) service_pool.cc:224] Handling call
0709 13:20:11.740456 (+1003390us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b540009d3d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:11.740618 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":807068928}
I20260709 13:20:11.735325 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:11.742274 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:11.736519 19856 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:11.743517 15600 raft_consensus.cc:2393] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 14: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:11.743806 15566 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:11.746034 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.746201 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.700533 19749 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.749686 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.724951 19762 raft_consensus.cc:740] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:11.719586 19879 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:11.754144 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:11.756275 15676 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:11.758283 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.759763 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:11.760843 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:11.763700 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.762063 15671 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:11.764505 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:11.769726 15491 leader_election.cc:400] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9.
I20260709 13:20:11.770104 15491 leader_election.cc:403] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:20:11.755894 19879 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:11.762969 19856 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:11.777426 19991 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:11.792294 16239 raft_consensus.cc:1240] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:20:11.792779 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:11.794426 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 43523us
I20260709 13:20:11.792186 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.796923 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.797566 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.800102 15524 raft_consensus.cc:2393] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 13: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:11.798244 15526 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:20:11.802510 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.803846 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.807123 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.787935 19777 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:11.814498 19777 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:11.816762 19777 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:11.781762 19512 consensus_queue.cc:1059] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:20:11.818856 19512 raft_consensus.cc:3055] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:11.819226 19512 raft_consensus.cc:740] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:11.820103 19512 consensus_queue.cc:260] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.821347 19512 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:11.822444 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.824146 15528 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:11.817376 19777 consensus_queue.cc:1059] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 13:20:11.827391 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:11.829299 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:11.829615 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.835075 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:20:11.833662 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 116761us
I20260709 13:20:11.844817 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.846201 15671 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:11.850589 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.799005 19888 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:11.852779 15566 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:11.856978 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:11.862807 19585 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:11.849114 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:11.866757 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:11.873556 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.874614 15528 raft_consensus.cc:1813] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Handling vote request from an unknown peer 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:11.870247 15524 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:20:11.875243 15528 raft_consensus.cc:2393] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 7: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:20:11.877619 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 82628.9us
I20260709 13:20:11.880890 16235 raft_consensus.cc:1240] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:11.882174 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.883476 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:11.889425 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:11.889631 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:11.890933 15526 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:11.892248 16205 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:20:11.895895 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 94152.7us
I20260709 13:20:11.899993 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:11.900837 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.904858 15673 raft_consensus.cc:2410] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 12 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:20:11.877291 19970 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:11.894295 19585 raft_consensus.cc:697] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:11.924412 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.925072 15521 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:20:11.926034 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.914140 19665 consensus_queue.cc:1059] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 13:20:11.922587 15567 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:11.929260 15523 raft_consensus.cc:2393] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 9: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:11.907497 19987 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:11.925855 19210 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:11.934072 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.901971 19888 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:11.940385 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 75553.4us
I20260709 13:20:11.941133 19656 raft_consensus.cc:3055] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:11.919874 19970 raft_consensus.cc:697] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:11.897433 19762 consensus_queue.cc:260] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:11.947415 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.950380 19871 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:11.960665 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:11.949601 19585 consensus_queue.cc:237] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:11.961354 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:11.963497 16234 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:20:11.964994 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:11.965521 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 50779.4us
I20260709 13:20:11.970685 16205 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:11.940811 19749 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:11.944631 19656 raft_consensus.cc:740] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:11.977516 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:11.977604 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:11.982208 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:11.983242 15412 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet bd36c2cfd99c45cea9e48096d54bed1a with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 5498ef3e89a941e39c25f68cf64e6b0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:11.983659 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:11.985598 19599 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:11.987128 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:11.987973 19970 consensus_queue.cc:237] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:11.964296 19871 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:11.990253 19871 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:11.990942 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:11.992496 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:11.993938 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:11.994993 19888 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:11.995188 19991 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9.
I20260709 13:20:11.995959 15526 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:20:11.996045 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:11.996657 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:11.997248 16232 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:12.001091 19599 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:11.996209 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:12.004895 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:12.005188 16236 raft_consensus.cc:1240] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:12.005735 16235 raft_consensus.cc:2393] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 11: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:12.006389 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:12.007776 20009 log.cc:927] Time spent T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.429s	user 0.003s	sys 0.000s
I20260709 13:20:12.015161 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:12.018420 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:12.019400 16236 raft_consensus.cc:2393] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 11: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:12.020072 16205 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:12.024433 16232 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:20:12.023164 19895 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:20:12.027062 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 103221us
I20260709 13:20:12.037364 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:12.038422 15521 raft_consensus.cc:2393] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 8: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:12.037369 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:12.048903 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:12.049454 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:12.049835 15525 raft_consensus.cc:2410] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:20:12.034303 19895 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
W20260709 13:20:12.053534 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 130473us
I20260709 13:20:12.056737 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:12.050966 16238 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:20:12.060322 16237 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:12.061712 15568 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:12.062639 15673 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 11. (term mismatch)
I20260709 13:20:12.062991 15673 pending_rounds.cc:85] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting all ops after (but not including) 10
I20260709 13:20:12.045226 19738 raft_consensus.cc:3055] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:12.064069 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:12.069285 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 49301.5us
I20260709 13:20:12.039511 19879 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:12.072472 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:12.070497 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:12.063221 15673 pending_rounds.cc:107] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting uncommitted NO_OP operation due to leader change: 10.11
I20260709 13:20:12.071123 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:12.058254 19895 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:12.074324 15673 raft_consensus.cc:2889] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:12.075712 15525 raft_consensus.cc:2410] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:20:12.077184 15677 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:12.066507 19738 raft_consensus.cc:740] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:12.083194 15567 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:12.085338 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:20:12.084158 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 140216us
I20260709 13:20:12.090641 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:12.094890 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:12.096676 15567 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:12.097231 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:12.097806 15568 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.098059 16240 raft_consensus.cc:2393] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 11: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:12.096266 19738 consensus_queue.cc:260] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:12.102208 19738 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:12.107378 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:12.110967 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:12.093978 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:12.109782 16205 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:12.115513 15602 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:12.127151 16240 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:12.127426 19970 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:12.127919 19970 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:12.128109 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:12.127911 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:20:12.133865 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 72334.7us
I20260709 13:20:12.062487 19656 consensus_queue.cc:260] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:12.136230 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:12.138315 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:12.137595 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:12.140848 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:12.142366 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:12.143239 15596 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:20:12.128320 19970 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:12.154995 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 57245us
I20260709 13:20:12.155853 15568 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.146927 19856 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:12.156812 19962 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:12.158144 15568 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.159328 19962 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:12.160106 19879 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:12.145815 15567 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:12.140828 19785 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:12.147042 19967 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:12.101682 19871 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:12.160604 15568 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.174309 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:12.175045 15599 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:12.174909 20032 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:12.175549 19967 raft_consensus.cc:697] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:12.177699 19967 consensus_queue.cc:237] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:12.159242 19968 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:20:12.180821 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 71158us
I20260709 13:20:12.175078 15567 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.175282 19785 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:12.170820 20023 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:12.190042 19785 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:12.201048 15567 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:12.201669 19766 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:12.203419 15493 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.208381 20034 consensus_queue.cc:1048] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:20:12.208801 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 48279us
I20260709 13:20:12.205907 19785 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:12.209583 15568 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.190900 19869 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:12.210098 20023 raft_consensus.cc:697] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:12.214146 15425 catalog_manager.cc:5697] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:12.215902 15641 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.218993 16205 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:12.219430 19724 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:12.221740 16239 raft_consensus.cc:1240] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:12.170281 19879 consensus_queue.cc:1059] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:20:12.221894 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:12.236528 19656 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:12.236878 19787 raft_consensus.cc:3055] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:12.237285 19787 raft_consensus.cc:740] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:12.237612 19665 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:12.238292 19787 consensus_queue.cc:260] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:12.220723 20023 consensus_queue.cc:237] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:12.239403 19787 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:12.225037 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 51184.3us
I20260709 13:20:12.226296 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:12.242169 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:12.211390 20033 consensus_queue.cc:1048] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:20:12.230899 19749 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:20:12.232712 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 113584us
I20260709 13:20:12.243099 16205 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:12.244735 19968 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:12.231056 19962 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:12.253485 15568 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.239512 19871 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:12.248435 19991 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:12.265954 19968 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:12.272032 19665 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:12.279865 16205 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:12.280689 19210 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:12.254448 19766 consensus_queue.cc:1059] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:20:12.288947 19766 raft_consensus.cc:3055] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:12.289378 19766 raft_consensus.cc:740] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:12.295528 15641 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.298566 16204 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:12.299515 15493 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:12.307615 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 80046us
I20260709 13:20:12.285535 15425 catalog_manager.cc:5697] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 4 to 5, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 5 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:12.315222 15493 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:12.317613 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 43978us
I20260709 13:20:12.318547 15568 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.290779 19962 raft_consensus.cc:697] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:12.254613 19970 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:12.328635 15568 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:12.321578 19762 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:12.304497 19991 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:12.331411 15493 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.336373 15493 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.331691 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:12.337000 15597 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:12.304111 19879 consensus_queue.cc:1059] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:20:12.324915 19987 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:12.324466 20034 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:12.333827 19888 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:12.333774 19871 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:12.337903 19445 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:12.338446 19760 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:12.377413 20036 consensus_queue.cc:1059] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:20:12.348843 19895 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:12.378204 19991 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:12.340065 19968 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:12.297387 19766 consensus_queue.cc:260] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:12.345903 19777 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:12.354558 20034 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:12.426167 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 153163us
I20260709 13:20:12.398506 20034 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:12.384820 19445 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:12.458459 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 138762us
I20260709 13:20:12.400107 20025 raft_consensus.cc:3055] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:12.397065 19968 raft_consensus.cc:697] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:12.404814 19766 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:20:12.490183 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 150748us
W20260709 13:20:12.496415 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 153039us
I20260709 13:20:12.497910 19724 raft_consensus.cc:3055] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:12.498399 19724 raft_consensus.cc:740] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:12.499286 19724 consensus_queue.cc:260] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:12.500833 19724 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:12.503661 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:12.475371 15597 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:20:12.476276 16204 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:12.511207 16204 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:12.511911 19970 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:12.515201 19738 raft_consensus.cc:3055] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:12.515715 19738 raft_consensus.cc:740] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:12.516677 19738 consensus_queue.cc:260] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:12.518343 19738 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:12.518862 20026 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:12.521813 20043 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:12.531711 15677 raft_consensus.cc:1240] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 10. Current term is 11. Ops: []
W20260709 13:20:12.532858 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:12.540410 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:12.553808 20056 consensus_queue.cc:1059] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:12.555578 20055 raft_consensus.cc:3055] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:12.555990 20055 raft_consensus.cc:740] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:12.556815 20055 consensus_queue.cc:260] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:12.560770 16205 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
W20260709 13:20:12.565248 20041 log.cc:927] Time spent T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:20:12.495945 19785 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:12.566551 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 222617us
I20260709 13:20:12.565896 20055 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:12.608196 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 128110us
I20260709 13:20:12.610813 20054 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:20:12.610903 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 50231.4us
I20260709 13:20:12.479803 20025 raft_consensus.cc:740] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
W20260709 13:20:12.620357 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 58048.8us
I20260709 13:20:12.565749 19749 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:20:12.571100 20038 log.cc:927] Time spent T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.137s	user 0.003s	sys 0.007s
I20260709 13:20:12.631145 19445 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:12.633082 19785 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:12.633975 20039 log.cc:927] Time spent T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.241s	user 0.000s	sys 0.004s
I20260709 13:20:12.636807 15601 raft_consensus.cc:1240] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:12.397773 19962 consensus_queue.cc:237] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:12.647333 19895 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:12.653671 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 44817.4us
I20260709 13:20:12.659823 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:12.660211 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:12.660920 15600 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:12.653720 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 142600us
I20260709 13:20:12.664508 16204 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:12.660446 15602 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:20:12.611495 20054 raft_consensus.cc:697] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:12.663690 16231 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:12.645727 19987 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:12.670401 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 322045us
I20260709 13:20:12.674952 16205 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:12.679555 19895 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:12.680015 19895 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:12.680366 19895 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:12.677662 15641 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.687628 19749 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:12.681592 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:12.687826 15642 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:12.696082 15600 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:12.697016 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:12.697433 16205 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:12.696648 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:12.702338 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:12.702943 19727 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:12.703485 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:12.707863 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:12.708644 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:12.709780 15521 raft_consensus.cc:2393] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 7: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:12.713884 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:12.698032 19656 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:12.717968 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:12.718595 15528 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:12.705294 19749 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:12.722761 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:12.721371 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:12.690215 19512 consensus_queue.cc:1059] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:20:12.722499 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:12.725219 15521 raft_consensus.cc:2393] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 8: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:12.694831 20060 consensus_queue.cc:1059] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:20:12.711932 20032 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:12.716830 19727 raft_consensus.cc:697] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:20:12.740434 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 52691.2us
W20260709 13:20:12.751372 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 84424.9us
I20260709 13:20:12.761062 15526 raft_consensus.cc:1240] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:12.762306 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:12.766708 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 63619.9us
I20260709 13:20:12.768764 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.775447 15493 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.780453 15528 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:12.783336 15642 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.784308 15641 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.783468 15675 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:12.786093 20064 log.cc:927] Time spent T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
W20260709 13:20:12.787899 20048 log.cc:927] Time spent T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.291s	user 0.000s	sys 0.004s
I20260709 13:20:12.789279 20032 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:12.789722 20032 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:12.790107 20032 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:12.743726 20054 consensus_queue.cc:237] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:12.795702 15493 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:12.797184 15493 leader_election.cc:400] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9.
I20260709 13:20:12.797355 19512 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:12.776119 20044 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.129s
I20260709 13:20:12.705475 19968 consensus_queue.cc:237] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:12.807593 19787 raft_consensus.cc:3055] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:12.808109 19787 raft_consensus.cc:740] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:12.808956 19787 consensus_queue.cc:260] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:12.777344 19749 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:12.818341 19727 consensus_queue.cc:237] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:12.816421 20034 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:12.819990 15677 raft_consensus.cc:1240] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 12. Current term is 13. Ops: []
W20260709 13:20:12.831857 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 53833.4us
I20260709 13:20:12.832767 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.855333 15493 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:12.721050 20025 consensus_queue.cc:260] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:12.785238 19888 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:12.858723 15642 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.839545 19927 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:12.851038 19787 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:12.876626 15642 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.859728 19749 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:20:12.878749 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 100208us
W20260709 13:20:12.886758 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 62961.3us
I20260709 13:20:12.887765 19656 raft_consensus.cc:3055] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:12.888967 19978 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:12.889413 15677 raft_consensus.cc:1275] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:20:12.891749 15642 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.891749 15493 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.875216 20025 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:12.875771 19900 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:12.906170 20036 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:12.894711 19656 raft_consensus.cc:740] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:12.907856 19656 consensus_queue.cc:260] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:12.910027 20037 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:12.915560 15642 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.900931 19888 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:12.918283 16231 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:20:12.900305 19927 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:12.919564 15493 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.920519 15641 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.922025 15675 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:20:12.930143 15677 raft_consensus.cc:1275] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:20:12.932473 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:12.955312 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:12.956516 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 84914.7us
I20260709 13:20:12.957996 19512 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:12.956270 19871 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:20:12.958465 15493 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:12.905619 19762 consensus_queue.cc:1048] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:20:12.920892 19900 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:20:12.971230 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 53131us
I20260709 13:20:12.972594 15642 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.968827 19585 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:12.960103 19978 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:12.976229 19656 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:12.979722 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 74044.5us
I20260709 13:20:12.960683 20037 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:12.973728 19585 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:12.985131 19585 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:12.983114 19927 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:12.978581 19512 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:12.996348 15525 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:13.001137 15522 raft_consensus.cc:1240] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:13.001451 15521 raft_consensus.cc:1275] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:20:13.005607 15641 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:13.006031 19895 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:13.007227 20074 log.cc:927] Time spent T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:20:13.011520 15493 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:13.012478 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:12.981904 20034 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:13.017454 15641 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:13.015524 15642 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:12.998924 19512 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:12.990113 19724 consensus_queue.cc:1059] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:20:13.014552 19895 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:13.020401 19927 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:13.036863 19927 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:13.037285 19927 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:13.020169 19976 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:13.014576 20073 consensus_queue.cc:1059] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:20:13.042696 15642 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:13.034103 19512 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:13.044304 20046 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:20:13.026684 20037 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:13.056501 19749 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:20:13.058471 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 47048.6us
I20260709 13:20:13.065399 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:13.068053 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 50685us
I20260709 13:20:13.068912 15641 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:13.078418 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:13.080595 15602 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:13.081518 15493 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:13.083860 15493 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:13.095422 15493 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:13.096684 15493 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:13.100508 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:13.094933 20037 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:20:13.104190 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 55757.4us
I20260709 13:20:13.105475 19856 consensus_queue.cc:1059] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:13.078567 20034 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:13.094161 19787 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:13.106519 19856 raft_consensus.cc:3055] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:13.100060 19978 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:20:13.113678 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 44845.4us
I20260709 13:20:13.103883 20067 raft_consensus.cc:3055] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:13.097733 19888 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:13.101714 19991 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:20:13.117302 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.103166 19976 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:13.119493 15641 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:13.047410 20032 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:13.100219 19749 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:20:13.132320 20076 log.cc:927] Time spent T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.226s	user 0.002s	sys 0.000s
I20260709 13:20:13.118491 20067 raft_consensus.cc:740] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:13.136857 20067 consensus_queue.cc:260] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:13.138247 20067 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:13.112829 19856 raft_consensus.cc:740] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:13.097990 20055 consensus_queue.cc:1048] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
W20260709 13:20:13.145771 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 103150us
I20260709 13:20:13.102504 20062 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:20:13.155365 15642 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:13.158406 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 50289.7us
I20260709 13:20:13.159099 15493 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:13.159861 20087 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:13.160367 20087 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:13.160785 20087 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:13.161612 15595 raft_consensus.cc:1240] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:13.163461 15567 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:13.164350 20046 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:13.164727 15567 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:13.171489 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 57090.8us
I20260709 13:20:13.172412 15567 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:13.153391 19749 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:13.143769 19978 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:13.179785 15493 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:13.179777 19991 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:13.195842 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:13.214774 15525 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:20:13.165836 19762 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:20:13.177947 15422 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58802 (request call id 260) took 33187 ms (client timeout 15000 ms). Trace:
I20260709 13:20:13.218624 16237 raft_consensus.cc:1240] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10. Ops: []
W20260709 13:20:13.219230 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 62756.9us
I20260709 13:20:13.152014 19787 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
W20260709 13:20:13.223104 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 52494.6us
W20260709 13:20:13.224485 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 77824.7us
I20260709 13:20:13.225698 15491 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:13.227378 16233 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:20:13.232450 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.233351 15642 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:20:13.235014 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 62672.9us
I20260709 13:20:13.235878 15567 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:13.237321 15567 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:13.215852 15422 rpcz_store.cc:269] 0709 13:19:39.827701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.835155 (+  7454us) service_pool.cc:224] Handling call
0709 13:20:13.014454 (+33179299us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":238,"mutex_wait_us":1378000,"reported_tablets":44}
I20260709 13:20:13.209267 19888 consensus_queue.cc:1059] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:20:13.192091 19749 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:13.260850 16237 raft_consensus.cc:1240] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:13.260994 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:13.239399 15596 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:13.189879 20086 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:13.218353 19976 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:13.163542 19856 consensus_queue.cc:260] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:13.270649 15596 raft_consensus.cc:1240] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:20:13.241833 16233 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13. Ops: []
W20260709 13:20:13.267972 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 48363.2us
I20260709 13:20:13.274415 15641 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:13.223788 20046 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:13.279541 15642 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:13.239881 19766 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:13.288884 15596 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13. Ops: [10.10-12.12]
I20260709 13:20:13.226845 19991 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:13.291703 19991 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:13.294390 19927 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
W20260709 13:20:13.298820 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.299743 15595 raft_consensus.cc:1240] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9. Ops: []
W20260709 13:20:13.301451 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 64177.4us
I20260709 13:20:13.303056 15567 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:13.265115 19749 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:13.268507 20032 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:13.271714 20062 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.277202 20036 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:13.275349 19762 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:13.316397 15641 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:13.317705 19766 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:13.318411 19991 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:13.319506 15599 raft_consensus.cc:1240] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 13:20:13.322206 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:13.244190 19978 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:13.322365 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 96744.4us
I20260709 13:20:13.281809 19976 raft_consensus.cc:697] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:13.333981 15567 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:13.337689 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:13.338452 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:13.339273 16233 raft_consensus.cc:2393] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 13: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:13.320493 20086 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:13.286103 20037 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
W20260709 13:20:13.342032 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 120105us
I20260709 13:20:13.344343 20055 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:13.347878 19665 raft_consensus.cc:3055] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:13.348336 19665 raft_consensus.cc:740] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:13.350245 15641 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:13.351428 20023 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:13.351871 20023 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:13.352272 20023 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:13.352833 15641 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:13.354153 19976 consensus_queue.cc:237] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:13.355265 19900 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:13.356056 19900 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:13.331974 20034 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:20:13.362689 15425 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49144 (request call id 351) took 33339 ms (client timeout 15000 ms). Trace:
I20260709 13:20:13.362172 20023 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:13.329568 20088 consensus_queue.cc:1059] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:20:13.363719 19991 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:13.324416 20036 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:13.366569 20032 consensus_queue.cc:1059] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:20:13.365800 20094 consensus_queue.cc:1059] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:20:13.371073 19512 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:13.372217 20023 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:13.377383 19900 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:13.343389 19888 consensus_queue.cc:1243] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer 5498ef3e89a941e39c25f68cf64e6b0f log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:20:13.379741 19978 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:13.363909 15602 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:20:13.327989 20062 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:13.342760 20086 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:13.342607 20044 consensus_queue.cc:1059] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
W20260709 13:20:13.401446 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 55689.7us
I20260709 13:20:13.404798 15596 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:20:13.424611 16235 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:20:13.428774 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:13.433455 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 60986.8us
I20260709 13:20:13.437590 15595 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:20:13.441424 15567 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:13.440411 15603 raft_consensus.cc:1240] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:20:13.407203 20036 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:13.425938 20088 consensus_queue.cc:1243] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer dd1a14c3173d49dc9b55bb116e3783c9 log is divergent from this leader: its last log entry 10.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:20:13.432085 20055 raft_consensus.cc:697] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:13.434890 20032 consensus_queue.cc:1243] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer dd1a14c3173d49dc9b55bb116e3783c9 log is divergent from this leader: its last log entry 13.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:20:13.481180 15601 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:20:13.426204 20025 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:20:13.490358 16205 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:13.470291 20086 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:13.433008 19766 consensus_queue.cc:1048] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:20:13.435827 20087 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:13.436259 20037 consensus_queue.cc:1059] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:20:13.508404 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:13.515452 20032 consensus_queue.cc:1059] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:20:13.519693 20025 consensus_queue.cc:1243] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer 5498ef3e89a941e39c25f68cf64e6b0f log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:20:13.520419 20023 consensus_queue.cc:1059] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:20:13.519758 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.538517 15675 raft_consensus.cc:1240] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:13.495636 20044 raft_consensus.cc:3055] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:20:13.556058 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 47734.3us
I20260709 13:20:13.576278 15491 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:20:13.548465 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.511091 20060 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:13.579891 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:13.580708 15642 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:13.583725 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 230968us
W20260709 13:20:13.363116 15425 rpcz_store.cc:269] 0709 13:19:40.023323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.023577 (+   254us) service_pool.cc:224] Handling call
0709 13:20:13.362584 (+33339007us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:12.615344 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:12.615994 (+   650us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:12.616043 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:20:12.616076 (+    33us) tablet.cc:662] Decoding operations
0709 13:20:12.616580 (+   504us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:12.616674 (+    94us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:12.616701 (+    27us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:20:12.617114 (+   413us) tablet.cc:701] Row locks acquired
0709 13:20:12.617162 (+    48us) write_op.cc:260] PREPARE: finished
0709 13:20:12.617330 (+   168us) write_op.cc:270] Start()
0709 13:20:12.617475 (+   145us) write_op.cc:276] Timestamp: P: 1783603212617254 usec, L: 0
0709 13:20:12.617498 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:20:12.661695 (+ 44197us) log.cc:844] Serialized 918 byte log entry
0709 13:20:12.883814 (+222119us) op_driver.cc:464] REPLICATION: finished
0709 13:20:13.023645 (+139831us) write_op.cc:301] APPLY: starting
0709 13:20:13.069482 (+ 45837us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:13.071530 (+  2048us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:13.071627 (+    97us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:13.072901 (+  1274us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:13.073184 (+   283us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 13:20:13.073225 (+    41us) write_op.cc:312] APPLY: finished
0709 13:20:13.073494 (+   269us) log.cc:844] Serialized 33 byte log entry
0709 13:20:13.100896 (+ 27402us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:13.107705 (+  6809us) write_op.cc:454] Released schema lock
0709 13:20:13.112437 (+  4732us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:13.172593 (+ 60156us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:13.174952 (+  2359us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":445107,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":112071,"apply.run_cpu_time_us":5351,"apply.run_wall_time_us":138429,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12503,"num_ops":2,"prepare.queue_time_us":80377,"prepare.run_cpu_time_us":16071,"prepare.run_wall_time_us":111259,"raft.queue_time_us":15985,"raft.run_cpu_time_us":11619,"raft.run_wall_time_us":71968,"replication_time_us":266147,"spinlock_wait_cycles":64256,"thread_start_us":187329,"threads_started":4,"wal-append.queue_time_us":92461}]]}
I20260709 13:20:13.627904 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:13.519855 20095 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.628846 16235 raft_consensus.cc:2393] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 13: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:13.629892 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:13.539533 20056 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:20:13.631924 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 51294.7us
I20260709 13:20:13.555653 20044 raft_consensus.cc:740] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:13.637580 15641 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:13.556540 19927 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:13.648638 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 218951us
I20260709 13:20:13.584412 19976 consensus_queue.cc:1059] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:20:13.631510 19900 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:13.640233 20103 raft_consensus.cc:3055] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:13.640918 20060 raft_consensus.cc:697] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:13.636343 20056 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:13.639698 20037 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:20:13.696358 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 47036.5us
W20260709 13:20:13.697661 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 60181.1us
I20260709 13:20:13.699529 15641 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:13.662111 19991 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:13.723793 15672 raft_consensus.cc:1240] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:13.725967 15597 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:20:13.682973 20103 raft_consensus.cc:740] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:13.730362 15594 raft_consensus.cc:1217] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Deduplicated request from leader. Original: 6.4->[8.5-9.6]   Dedup: 8.5->[9.6-9.6]
I20260709 13:20:13.732075 15594 raft_consensus.cc:1240] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10. Ops: [8.5-9.6]
I20260709 13:20:13.733793 20088 consensus_queue.cc:1059] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:20:13.734516 20088 consensus_queue.cc:1243] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer 5498ef3e89a941e39c25f68cf64e6b0f log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:20:13.697260 20056 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:13.639542 20055 consensus_queue.cc:237] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:13.719321 19900 raft_consensus.cc:697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:13.718495 20037 raft_consensus.cc:697] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:13.719449 20111 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:13.638851 20095 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:13.777014 20087 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:13.778254 20062 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:13.780201 20095 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:13.798568 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 99117.3us
I20260709 13:20:13.799589 15641 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:13.748953 19987 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.042s
W20260709 13:20:13.806493 20097 log.cc:927] Time spent T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:20:13.807713 19900 consensus_queue.cc:237] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:13.804338 20042 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.810386 20042 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:13.811626 20093 consensus_queue.cc:1059] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.152s
I20260709 13:20:13.812909 20042 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:13.813870 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.804858 20103 consensus_queue.cc:260] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:13.743927 20044 consensus_queue.cc:260] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:13.826508 20044 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:13.828542 19749 consensus_meta.cc:287] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.509s	user 0.006s	sys 0.002s
I20260709 13:20:13.829015 19749 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:13.749255 20099 raft_consensus.cc:2764] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:20:13.835355 20032 raft_consensus.cc:3055] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:13.835857 20032 raft_consensus.cc:740] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:13.836714 20032 consensus_queue.cc:260] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:13.840432 20098 raft_consensus.cc:3055] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:13.840937 20098 raft_consensus.cc:740] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:13.841795 20098 consensus_queue.cc:260] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:13.843281 20098 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:13.844075 20037 consensus_queue.cc:237] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:13.847374 20075 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.838s	user 0.004s	sys 0.001s
I20260709 13:20:13.847312 20099 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:20:13.849686 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 50196.4us
I20260709 13:20:13.775419 20068 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:20:13.794842 20060 consensus_queue.cc:237] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:13.852295 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:13.855791 20036 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:13.858274 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:13.859567 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:13.860445 15525 raft_consensus.cc:1240] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 13:20:13.801064 19871 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.862901 19871 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:13.863996 15525 raft_consensus.cc:1240] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:20:13.865089 19871 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:13.823350 15527 raft_consensus.cc:3055] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:20:13.825062 20100 log.cc:927] Time spent T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:20:13.867326 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:13.869745 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.872102 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:13.879259 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 129982us
I20260709 13:20:13.874791 15527 raft_consensus.cc:740] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:13.882737 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:13.883424 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:13.884167 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:13.895498 15523 raft_consensus.cc:1240] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 13:20:13.908088 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:13.908869 16240 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:20:13.910140 15568 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:13.912498 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:20:13.918668 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 342473us
I20260709 13:20:13.921350 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:13.922005 15674 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:13.925209 15528 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:20:13.926026 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:13.926402 15677 raft_consensus.cc:1240] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:13.927237 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:13.927872 15676 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:20:13.929010 16237 consensus_queue.cc:237] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:13.934679 15528 raft_consensus.cc:1240] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:13.934464 15491 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:13.881757 20036 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:13.976750 15668 raft_consensus.cc:1275] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:20:13.925544 15527 consensus_queue.cc:260] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:13.978272 15668 pending_rounds.cc:85] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4: Aborting all ops after (but not including) 3
I20260709 13:20:13.978578 15668 pending_rounds.cc:107] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:20:13.904312 19895 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.982722 15491 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:13.917785 20032 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:13.934242 20103 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:13.984664 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:13.985332 15567 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.012s	sys 0.028s
W20260709 13:20:14.004703 15567 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.013s	sys 0.028s
I20260709 13:20:14.005012 15594 raft_consensus.cc:2410] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
W20260709 13:20:14.005340 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 563966us
I20260709 13:20:13.978761 15527 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:13.988480 15677 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:13.978804 15668 raft_consensus.cc:2889] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:14.006641 15668 pending_rounds.cc:107] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:20:14.006979 15668 raft_consensus.cc:2889] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:13.948938 19976 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:20:13.980466 19762 consensus_meta.cc:287] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.648s	user 0.010s	sys 0.000s
W20260709 13:20:14.015249 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 105201us
I20260709 13:20:14.016093 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:14.016832 15675 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:20:14.024012 20104 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.024624 20104 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.018900 19762 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:14.028127 19701 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:20:14.033551 20106 log.cc:927] Time spent T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.152s	user 0.002s	sys 0.000s
I20260709 13:20:14.002157 19895 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.045202 15522 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
W20260709 13:20:14.054183 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:14.062435 20126 log.cc:927] Time spent T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:20:14.063539 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:14.036850 19701 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:14.077190 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 62585.8us
I20260709 13:20:14.079332 20132 consensus_queue.cc:1059] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:20:14.040518 20128 consensus_queue.cc:1059] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:20:14.070515 20062 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:14.076517 19701 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:14.095310 20046 consensus_meta.cc:287] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.813s	user 0.006s	sys 0.005s
W20260709 13:20:14.108484 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 102379us
I20260709 13:20:14.110889 15522 raft_consensus.cc:1275] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 13:20:14.110647 20086 consensus_meta.cc:287] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.593s	user 0.000s	sys 0.010s
I20260709 13:20:14.111780 20046 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:14.168238 20024 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.168881 20024 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.172173 20080 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.172763 20080 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.124801 20060 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:14.166415 20093 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.168906 20129 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.179512 20024 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:14.128047 20086 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:14.188813 19895 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:14.222482 15527 raft_consensus.cc:1275] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:20:14.223107 15527 pending_rounds.cc:85] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 5
I20260709 13:20:14.223382 15527 pending_rounds.cc:107] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:20:14.225338 15527 raft_consensus.cc:2889] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:14.229527 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 2090) took 1036 ms. Trace:
I20260709 13:20:14.230289 15673 raft_consensus.cc:1240] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:14.204363 20093 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.216694 20104 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:14.202423 20060 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.229966 15527 rpcz_store.cc:276] 0709 13:20:13.193631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:13.193747 (+   116us) service_pool.cc:224] Handling call
0709 13:20:13.223359 (+ 29612us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:13.821822 (+598463us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b54003064a0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4982dc0 00007f5ce4a0d1c0 00007f5ce4a0e231 00007f5ce4a0ad8f 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c
0709 13:20:14.229258 (+407436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:14.229462 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":446323584}
I20260709 13:20:14.208963 20129 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.239912 20080 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:14.263885 20129 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:14.269814 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:14.270560 15525 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:20:14.270831 15528 raft_consensus.cc:1240] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:14.253909 20058 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.276610 15522 raft_consensus.cc:1275] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:20:14.278128 20115 raft_consensus.cc:3055] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:14.278625 20115 raft_consensus.cc:740] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:14.279663 20115 consensus_queue.cc:260] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:14.280280 20060 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:14.302417 16235 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:20:14.304275 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:14.304678 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:14.280907 20034 raft_consensus.cc:3055] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:14.281764 19978 raft_consensus.cc:3055] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:14.282382 20123 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:14.283896 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:14.284343 20062 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:14.276486 20144 log.cc:927] Time spent T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:20:14.288733 19749 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:14.289525 19701 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.297959 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:14.299594 20129 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:14.299440 19665 consensus_queue.cc:260] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.300462 15519 raft_consensus.cc:3055] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:14.300927 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:14.316447 16240 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:14.301317 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:14.317420 16237 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:14.306509 20087 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.313865 19210 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:14.320657 19210 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.316233 15519 raft_consensus.cc:740] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:14.318493 15568 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:14.307520 20034 raft_consensus.cc:740] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:14.322801 19210 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:14.310087 20062 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:14.324885 16204 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:14.307654 19978 raft_consensus.cc:740] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:14.326045 15568 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:14.307062 20095 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:14.326720 19210 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:14.327600 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:14.311925 20123 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.329756 20104 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:14.346377 20104 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:14.346839 20104 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:14.319774 20129 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.307066 20055 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:20:14.330451 20093 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:14.331458 19665 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:14.336359 19210 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:14.368721 19210 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:14.336278 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:14.326490 20024 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.331648 19978 consensus_queue.cc:260] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.317580 20043 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.333094 15519 consensus_queue.cc:260] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.320605 20115 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:14.323628 20086 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:14.308403 20058 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.373790 15669 raft_consensus.cc:2410] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:20:14.380069 15519 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:14.376768 19856 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:14.356293 20095 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:14.388034 20095 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:14.372608 19787 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:20:14.390611 20034 consensus_queue.cc:260] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.392023 20034 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:14.336295 20054 consensus_queue.cc:1059] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:20:14.395547 20043 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.326148 20087 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.399021 20058 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:14.379777 20024 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.402961 20123 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:14.372840 20056 consensus_meta.cc:287] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.621s	user 0.011s	sys 0.000s
W20260709 13:20:14.404150 20116 log.cc:927] Time spent T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.558s	user 0.004s	sys 0.000s
I20260709 13:20:14.371461 20132 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:14.404615 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:14.405355 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:14.409972 20129 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:14.412098 20054 raft_consensus.cc:3055] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:14.448899 19210 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.415891 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:14.374783 20098 consensus_meta.cc:287] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.531s	user 0.011s	sys 0.002s
I20260709 13:20:14.435436 15677 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
W20260709 13:20:14.451391 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 124202us
I20260709 13:20:14.451566 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:14.453851 16240 raft_consensus.cc:1240] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:14.455845 15671 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:20:14.369860 20131 consensus_queue.cc:1048] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.098s
I20260709 13:20:14.456915 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:14.463981 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:14.457515 15669 raft_consensus.cc:2393] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 10: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:14.458602 15527 raft_consensus.cc:1275] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:20:14.437839 20132 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:14.470709 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:14.471345 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:14.472045 15672 raft_consensus.cc:2393] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 11: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:14.469411 19210 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:14.477751 15642 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:14.442745 20054 raft_consensus.cc:740] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:14.439925 19978 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:14.486097 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 78634.1us
I20260709 13:20:14.482730 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:14.487243 16203 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:14.488143 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:14.438149 20056 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:14.474828 20132 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:14.498291 15641 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:14.499729 15641 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:14.500262 15595 raft_consensus.cc:1240] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:14.501397 20056 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:14.501871 20098 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:14.502596 20098 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:14.504211 19512 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:14.486267 20086 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:14.510324 15641 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:14.510815 20054 consensus_queue.cc:260] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.513147 20103 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:14.514856 15641 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:14.516386 20104 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.517478 20103 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:14.518893 15641 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:14.518970 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:14.521224 20103 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:14.521833 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:14.521756 20103 raft_consensus.cc:697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:14.522727 15524 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:14.524295 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:14.525342 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:14.525954 15675 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:14.531922 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:14.532681 15524 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:20:14.533828 15599 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:20:14.531946 15641 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:14.534547 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:14.507452 19895 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:14.538519 19701 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:14.544389 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:14.546708 16233 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:14.545656 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:14.548527 15528 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:14.575850 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:14.583585 15594 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:20:14.557677 16204 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:14.523164 20103 consensus_queue.cc:237] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.533004 19512 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:14.597630 16204 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:14.597525 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 65671.5us
I20260709 13:20:14.595984 15491 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:14.599401 20086 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:14.599895 20068 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:14.601689 16233 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:20:14.587738 15528 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:20:14.604744 15568 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:14.605681 20068 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:14.608305 20098 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:14.608331 20068 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:14.610862 20093 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:14.535535 20056 consensus_queue.cc:1048] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 13:20:14.614737 20123 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:14.615188 20123 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:14.615540 20123 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:14.620159 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:14.621438 20024 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:14.623941 20149 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:14.624585 20149 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.550215 20148 consensus_queue.cc:1048] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 13:20:14.627391 20149 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:14.629642 15642 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:14.630846 16204 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:14.633623 15641 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:14.635166 16231 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:14.606876 19895 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:14.629325 19766 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:14.633150 20098 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.631799 20128 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:14.656585 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:14.657702 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:14.658541 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:14.661929 15671 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
W20260709 13:20:14.665202 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 178052us
I20260709 13:20:14.714478 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:14.715312 15599 raft_consensus.cc:2393] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 9: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:14.641422 20086 raft_consensus.cc:697] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:14.651229 19895 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:14.677142 16235 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:14.678557 20104 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:14.679255 20068 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.646220 20037 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:14.607180 19512 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.650180 20025 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:14.684676 15491 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:14.653090 19787 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:14.652840 20087 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:14.663389 19727 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:14.657675 20051 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:20:14.690565 20125 log.cc:927] Time spent T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.658s	user 0.006s	sys 0.000s
I20260709 13:20:14.680256 20128 raft_consensus.cc:697] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:14.687934 15519 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
W20260709 13:20:14.700635 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 163868us
I20260709 13:20:14.663789 19766 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.667615 20098 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:14.704238 15669 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:20:14.704569 15670 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
I20260709 13:20:14.711063 20132 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.719460 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:14.751830 15668 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:14.752107 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1442) took 1409 ms. Trace:
I20260709 13:20:14.723498 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:14.725903 15599 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:14.754170 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:14.724879 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:14.727504 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:14.732404 16238 raft_consensus.cc:1217] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Deduplicated request from leader. Original: 8.9->[9.10-9.10]   Dedup: 9.10->[]
W20260709 13:20:14.736558 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 51944.5us
I20260709 13:20:14.739156 15601 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:20:14.755949 16238 raft_consensus.cc:1240] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:20:14.739292 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:14.755323 16233 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:20:14.740434 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:14.741560 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:14.751083 20135 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:20:14.758538 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:14.759369 16236 raft_consensus.cc:2410] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:20:14.759794 19749 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:14.761085 16234 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:20:14.752545 15519 rpcz_store.cc:276] 0709 13:20:13.337025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:13.337129 (+   104us) service_pool.cc:224] Handling call
0709 13:20:13.337373 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:14.300380 (+963007us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b5400303ca0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4982dc0 00007f5ce4a0d1c0 00007f5ce4a0e231 00007f5ce4a0ad8f 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c
0709 13:20:14.745695 (+445315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:14.745910 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":312088960}
I20260709 13:20:14.699626 20062 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.746407 15567 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:14.747655 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:14.728334 19895 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:14.743144 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:14.763264 15519 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:14.768980 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:14.744457 19724 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.729568 20024 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.767218 16231 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:14.734468 20154 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.756821 15491 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:14.769831 15598 raft_consensus.cc:2393] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 12: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:14.731096 20107 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:20:14.752743 19888 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:20:14.773447 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:14.754890 20025 raft_consensus.cc:697] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:14.778365 15521 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:20:14.778772 15521 pending_rounds.cc:85] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 10
I20260709 13:20:14.778998 15521 pending_rounds.cc:107] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:20:14.779181 15521 raft_consensus.cc:2889] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:14.778412 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:14.761368 19727 raft_consensus.cc:697] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:14.767926 20087 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:14.817492 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:14.790694 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:14.786273 15598 raft_consensus.cc:2368] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 11. Current term is 12.
I20260709 13:20:14.787187 15519 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:20:14.830377 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:14.831178 15596 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:20:14.832376 16231 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:14.793497 15493 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:14.750613 19787 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.838181 15596 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:20:14.753196 20051 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.819988 20087 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:14.820542 19978 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:14.776075 20024 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.787401 20135 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:14.864125 20055 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:14.826722 19987 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:20:14.775502 19724 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.877432 19727 consensus_queue.cc:237] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.885391 20053 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.886508 19766 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:14.888007 20062 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:14.889287 20098 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:14.892493 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:14.777343 20154 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.904759 16236 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:14.905789 20154 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:14.907158 20132 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:14.874241 20087 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:14.907948 20142 raft_consensus.cc:3055] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:14.908351 20142 raft_consensus.cc:740] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:14.908931 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:14.909601 15673 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:14.903918 19787 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:14.776609 20107 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.898824 20053 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.881328 19978 raft_consensus.cc:697] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:14.910987 20160 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:14.925400 20068 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:14.909226 20142 consensus_queue.cc:260] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:14.839993 20086 consensus_queue.cc:237] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.926889 19512 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:14.928874 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:14.943881 15673 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:14.913561 20131 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.927731 20107 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:14.903802 19724 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:14.933189 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:14.960496 20131 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.967468 20053 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:14.952616 16239 consensus_queue.cc:237] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.969388 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:14.970129 15676 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:20:14.971706 20051 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:14.943090 20067 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.973618 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:14.974022 20067 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.974659 20131 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:14.953567 15668 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:20:14.866722 20128 consensus_queue.cc:237] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.894191 20055 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:14.979524 20055 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:14.959674 20123 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.973120 20087 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:14.957491 20024 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:14.997782 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:15.003571 20050 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:15.004274 20050 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:15.006233 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 212807us
I20260709 13:20:15.006901 20050 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:14.902418 20025 consensus_queue.cc:237] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:14.990598 20055 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:15.001997 20142 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:15.012645 15493 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:15.016197 20136 log.cc:927] Time spent T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.787s	user 0.002s	sys 0.004s
I20260709 13:20:15.009168 20103 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:15.017326 20165 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:15.017771 20123 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:15.007844 20067 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:15.018216 15493 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:15.018739 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:15.056665 15673 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:15.013219 19978 consensus_queue.cc:237] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:15.035043 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:15.035364 20098 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:15.044456 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:15.044472 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:15.039496 16203 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:15.058198 15526 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:20:15.060740 15520 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:15.062547 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:15.064059 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 45957.2us
I20260709 13:20:15.062018 15522 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:20:15.064888 15526 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:20:15.066984 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:15.067684 15526 raft_consensus.cc:2393] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 14: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:15.067727 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:15.068028 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:15.068650 15596 raft_consensus.cc:2393] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 10: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:15.063534 16203 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:15.077896 19210 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:15.078342 19210 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:15.078675 19210 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:15.074355 20024 raft_consensus.cc:2764] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:20:15.017434 19895 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:15.081948 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:15.082613 15522 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:20:15.082877 16237 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:15.078042 16203 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:15.083570 15493 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:15.083564 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:15.084424 16203 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:15.089715 20067 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:15.090162 20067 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:15.090503 20067 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:15.083778 20086 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:20:15.063112 20103 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:15.100180 20103 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:15.101068 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:15.036574 20055 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:15.102075 20123 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:15.102456 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:15.095782 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:15.104941 15523 raft_consensus.cc:2410] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:20:15.113287 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:15.105249 15594 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:20:15.122162 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:15.123104 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:15.123641 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:15.123805 16238 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:15.124722 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:15.123162 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:15.128051 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:15.114037 15526 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:20:15.131584 16239 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:15.131942 15668 raft_consensus.cc:1240] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:15.078569 19512 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:15.085884 20165 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:15.081813 19895 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:15.149617 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:15.131201 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:15.150378 15523 raft_consensus.cc:2368] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11.
I20260709 13:20:15.150440 15677 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:15.096238 20087 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:15.140751 16204 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:15.134642 15641 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:15.142979 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:15.156246 16234 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:15.143853 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:15.157173 15668 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:20:15.147648 15493 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:15.148434 15674 raft_consensus.cc:1240] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:15.109340 20024 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:15.158093 20024 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:15.158496 20024 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:15.159399 20062 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:15.159811 20062 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:15.160176 20062 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:15.103780 20055 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:15.162941 20159 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:15.139612 19210 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:15.078696 20131 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 13:20:15.153437 15520 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:20:15.165009 16232 raft_consensus.cc:2393] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 14: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:15.165907 15668 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:20:15.166028 19210 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:15.166651 19210 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:15.166987 19210 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:15.167315 19210 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:15.167847 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:15.174680 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:15.175712 15668 raft_consensus.cc:2393] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 14: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:15.170240 15677 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:15.143245 20098 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:15.135254 19962 consensus_queue.cc:1048] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:20:15.233137 19210 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:15.155094 20073 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:15.178032 20025 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:20:15.279532 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 145862us
W20260709 13:20:15.283481 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:15.235873 20100 log.cc:927] Time spent T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:20:15.229449 20123 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:15.293560 19210 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:15.152793 20165 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:15.300117 16234 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:15.149355 19970 consensus_queue.cc:1059] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 13:20:15.285300 19787 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:15.314522 20124 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:15.293962 20106 log.cc:927] Time spent T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:20:15.322217 16205 leader_election.cc:400] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 11. Current term is 12.
I20260709 13:20:15.377584 16205 leader_election.cc:403] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:15.335981 20165 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:15.336524 20123 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:15.306469 20025 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:15.337430 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:15.337446 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:15.381798 15595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 2299) took 1507 ms. Trace:
W20260709 13:20:15.338429 20174 log.cc:927] Time spent T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:20:15.341300 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:15.343628 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:15.345436 15676 raft_consensus.cc:1275] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:20:15.387809 15676 pending_rounds.cc:85] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting all ops after (but not including) 6
I20260709 13:20:15.388053 15676 pending_rounds.cc:107] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:20:15.388345 15676 raft_consensus.cc:2889] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:15.346095 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:15.389473 15669 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:15.244853 20098 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:15.346285 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:15.346797 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:15.347036 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:15.392066 15668 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:15.392211 15677 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:15.350878 15671 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:20:15.357282 15673 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
W20260709 13:20:15.371709 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 231087us
I20260709 13:20:15.395052 16204 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:15.306075 20133 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:15.379122 19787 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:15.394058 20093 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:15.399858 20093 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:15.400713 19787 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:15.401214 16204 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:15.382302 15595 rpcz_store.cc:276] 0709 13:20:13.873057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:13.895606 (+ 22549us) service_pool.cc:224] Handling call
0709 13:20:13.913751 (+ 18145us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:14.500686 (+586935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:15.377229 (+876543us) spinlock_profiling.cc:243] Waited 586 ms on lock 0x7b540043b054. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:15.377404 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1348454272}
W20260709 13:20:15.402783 20173 log.cc:927] Time spent T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
W20260709 13:20:15.404784 20175 log.cc:927] Time spent T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:20:15.399260 20121 consensus_queue.cc:1059] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:20:15.406872 15525 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:20:15.408375 20093 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:15.411057 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:15.411633 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:15.386026 19210 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:20:15.396581 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:15.412237 15601 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:15.413450 20055 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:15.414266 20155 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:15.414688 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:15.414791 20155 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:15.415331 15599 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:15.415663 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:15.416231 15600 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:20:15.418249 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:15.418445 20055 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:15.418938 19962 raft_consensus.cc:3055] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:15.419379 19962 raft_consensus.cc:740] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:15.399268 20133 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:15.419960 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:15.385478 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 68540.2us
I20260709 13:20:15.421657 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:15.411832 15522 raft_consensus.cc:2393] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 11: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:15.424995 15669 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:15.412457 15524 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:20:15.426014 15601 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:20:15.409335 20062 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:15.429355 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:15.429610 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:15.432127 15600 raft_consensus.cc:2393] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 6: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:15.384928 20025 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:15.437736 19210 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 11. Current term is 12.
I20260709 13:20:15.441011 16204 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:15.443362 20051 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:15.448225 15597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 1187) took 1529 ms. Trace:
I20260709 13:20:15.455395 20062 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:15.448643 15597 rpcz_store.cc:276] 0709 13:20:13.918766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:13.918919 (+   153us) service_pool.cc:224] Handling call
0709 13:20:13.919263 (+   344us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:20:14.021894 (+102631us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:14.021939 (+    45us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:20:14.023080 (+  1141us) log.cc:844] Serialized 104 byte log entry
0709 13:20:14.106954 (+ 83874us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:14.107273 (+   319us) log.cc:844] Serialized 14 byte log entry
0709 13:20:14.107574 (+   301us) log.cc:844] Serialized 14 byte log entry
0709 13:20:14.107733 (+   159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:14.107802 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:14.989266 (+881464us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:20:15.447973 (+458707us) raft_consensus.cc:1685] finished
0709 13:20:15.448002 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:15.448164 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1665,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":1344,"raft.queue_time_us":113488,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":224,"spinlock_wait_cycles":3438336,"thread_start_us":179309,"threads_started":2,"wal-append.queue_time_us":181025}
I20260709 13:20:15.458341 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:15.430154 15668 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:20:15.460119 15597 raft_consensus.cc:2393] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 10: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:15.460532 16204 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:15.420185 19962 consensus_queue.cc:260] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:15.461423 15603 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
W20260709 13:20:15.492992 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 70121.3us
I20260709 13:20:15.437017 20026 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:20:15.506811 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 115790us
I20260709 13:20:15.383869 20024 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:15.446791 20133 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:15.516477 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:15.453423 20067 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:15.519364 16205 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:15.312675 20073 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:15.456187 15523 raft_consensus.cc:2955] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Committing config change with OpId 5.7: config changed from index 5 to 7, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:15.531319 15522 raft_consensus.cc:2393] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 14: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:20:15.529110 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 68657.8us
I20260709 13:20:15.460377 20103 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:15.530963 15677 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:15.529685 19210 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:15.534868 16205 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:15.536202 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:15.459600 20086 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:15.538863 20181 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:15.532840 15566 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:15.527320 20026 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:15.556623 15566 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:15.394968 20124 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:15.560890 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:15.576602 15597 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:15.512530 20053 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:20:15.534049 20128 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:20:15.589423 16234 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:20:15.586735 16232 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:20:15.587181 16204 leader_election.cc:400] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11.
I20260709 13:20:15.590987 16204 leader_election.cc:403] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:20:15.554203 20181 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:15.590540 15491 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:15.553774 20042 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:15.540385 20060 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:15.610636 20120 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:15.627714 20151 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:15.554997 19962 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:15.632817 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:15.633780 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:15.636225 20169 log.cc:927] Time spent T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.356s	user 0.001s	sys 0.000s
I20260709 13:20:15.638638 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:15.676856 16235 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:15.640750 16203 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:15.571462 20026 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:15.557344 20133 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:20:15.642524 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 64652.3us
I20260709 13:20:15.706750 15491 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:15.649077 15602 raft_consensus.cc:1275] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:20:15.619603 20184 log.cc:927] Time spent T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.176s	user 0.002s	sys 0.000s
I20260709 13:20:15.657481 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:15.680346 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:15.571171 19970 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.080s
I20260709 13:20:15.551576 20103 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:15.651301 19976 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:15.642498 20050 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:15.714789 15597 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:20:15.559747 20086 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:15.632714 20054 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:15.648581 20181 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:15.650910 20037 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:15.726013 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:15.735059 15525 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:15.727102 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:15.736244 16235 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:20:15.731402 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:15.655951 20042 raft_consensus.cc:697] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:15.574110 16237 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:20:15.681578 20055 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:15.724274 19787 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:15.792470 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:15.793708 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 200623us
I20260709 13:20:15.719889 20149 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:20:15.732358 20136 log.cc:927] Time spent T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:20:15.804188 16233 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:15.801883 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:15.815547 16205 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:15.733523 19976 raft_consensus.cc:697] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:15.739986 20154 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:15.706223 20151 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:15.731129 20109 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:15.814196 15525 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:20:15.798002 20123 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:15.841850 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 2103) took 1059 ms. Trace:
I20260709 13:20:15.845279 15566 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:15.816653 20149 raft_consensus.cc:697] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:15.535799 20082 raft_consensus.cc:2955] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Committing config change with OpId 5.7: config changed from index 5 to 7, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:15.842309 15528 rpcz_store.cc:276] 0709 13:20:14.782282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:14.782396 (+   114us) service_pool.cc:224] Handling call
0709 13:20:14.826001 (+ 43605us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:14.873691 (+ 47690us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:14.873726 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:14.878747 (+  5021us) log.cc:844] Serialized 26 byte log entry
0709 13:20:15.003052 (+124305us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:20:15.003099 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:15.003175 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:15.635526 (+632351us) spinlock_profiling.cc:243] Waited 2.69 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:20:15.831402 (+195876us) raft_consensus.cc:1685] finished
0709 13:20:15.831425 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:15.841772 (+ 10347us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":46191,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":3876,"spinlock_wait_cycles":6192384,"thread_start_us":58221,"threads_started":1,"wal-append.queue_time_us":233295}
I20260709 13:20:15.847424 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:15.851342 15677 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:15.823374 20154 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:15.801162 19787 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:15.847970 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 264779us
I20260709 13:20:15.872732 16204 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:15.874090 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:15.875777 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:15.868227 20180 log.cc:927] Time spent T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
I20260709 13:20:15.879606 19976 consensus_queue.cc:237] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:15.880921 15528 raft_consensus.cc:1275] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:20:15.882138 20043 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:15.883464 20043 raft_consensus.cc:2804] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:15.883834 20043 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:15.884147 20043 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:15.869184 20024 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:15.868496 20067 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:15.871914 20129 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:15.890447 20129 raft_consensus.cc:697] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:15.891378 20129 consensus_queue.cc:237] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:15.892601 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:15.873508 20107 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:15.894403 20107 raft_consensus.cc:697] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:15.898986 15567 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:15.821592 20037 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:15.902523 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 223461us
I20260709 13:20:15.859427 20154 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:20:15.919005 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:15.923705 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:15.840400 20109 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:15.920475 20093 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:15.956498 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:15.896044 20073 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:15.923197 20121 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:15.919016 20098 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:15.963764 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:15.959280 15568 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:15.956552 20034 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.009s	sys 0.012s
W20260709 13:20:15.972766 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 157277us
I20260709 13:20:15.977568 15568 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:15.960430 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 88288.2us
I20260709 13:20:15.971901 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:15.986749 15671 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:16.021458 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 43963.4us
I20260709 13:20:16.021896 20128 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:16.024930 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:20:16.029876 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 151307us
I20260709 13:20:16.025309 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:15.989336 20034 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.010s	sys 0.012s
I20260709 13:20:16.035136 15598 raft_consensus.cc:2376] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8. Re-sending same reply.
I20260709 13:20:15.983779 19970 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:15.899945 20042 consensus_queue.cc:237] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:15.972288 20121 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:15.973417 20093 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:16.074388 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 88582.8us
I20260709 13:20:16.080674 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:16.081943 16235 raft_consensus.cc:1240] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:16.030401 20103 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:16.078787 20121 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:16.083361 20189 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 13:20:16.092813 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1480) took 1050 ms. Trace:
I20260709 13:20:16.093726 15642 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:16.094348 15677 raft_consensus.cc:1275] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:20:16.088263 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:16.088889 20034 consensus_queue.cc:1059] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:20:16.089527 20193 log.cc:927] Time spent T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:20:16.084182 20093 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:15.966638 20149 consensus_queue.cc:237] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:16.097684 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 66976.8us
I20260709 13:20:16.074652 20124 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:16.101610 19968 consensus_queue.cc:1059] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:20:16.103108 20132 raft_consensus.cc:3055] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:20:16.114882 15412 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet bd36c2cfd99c45cea9e48096d54bed1a with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 5498ef3e89a941e39c25f68cf64e6b0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:16.108266 15671 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:15.978441 20107 consensus_queue.cc:237] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:16.111075 20199 log.cc:927] Time spent T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:20:16.111745 20034 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:16.113413 20201 log.cc:927] Time spent T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 13:20:16.076973 20086 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:16.116577 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:16.130285 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:16.117373 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:16.135049 20200 log.cc:927] Time spent T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 13:20:16.136022 15676 raft_consensus.cc:1275] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:20:16.137234 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2153) took 1030 ms. Trace:
I20260709 13:20:16.137609 15522 rpcz_store.cc:276] 0709 13:20:15.106678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.559488 (+452810us) service_pool.cc:224] Handling call
0709 13:20:15.585845 (+ 26357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:15.586281 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:15.586309 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:15.586770 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 13:20:15.800195 (+213425us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:15.800246 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:15.800313 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:16.137032 (+336719us) raft_consensus.cc:1685] finished
0709 13:20:16.137052 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:16.137169 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2501,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":50708,"thread_start_us":147861,"threads_started":1,"wal-append.queue_time_us":282630}
W20260709 13:20:16.139381 20207 log.cc:927] Time spent T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:20:16.140251 20086 raft_consensus.cc:3055] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:16.140664 20086 raft_consensus.cc:740] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:16.141486 20086 consensus_queue.cc:260] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:16.144555 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:16.144584 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:16.147533 15597 raft_consensus.cc:1275] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:20:16.148658 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:16.120819 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:16.104393 20109 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:16.124102 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:16.125331 16203 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:16.111624 20037 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:16.090210 20209 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:16.142768 20151 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:16.099614 20024 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
W20260709 13:20:16.200320 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 50257us
W20260709 13:20:16.200513 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 53783.4us
W20260709 13:20:16.200383 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 53962.1us
I20260709 13:20:16.211463 20168 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:16.214298 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:16.126219 20132 raft_consensus.cc:740] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:20:16.209285 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:16.134176 20093 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:16.220077 19976 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:16.291814 19976 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:16.293318 20124 raft_consensus.cc:3055] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:16.293762 20124 raft_consensus.cc:740] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:16.294103 19976 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:16.216274 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 91036.5us
W20260709 13:20:16.149526 20025 consensus_meta.cc:287] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.683s	user 0.009s	sys 0.009s
I20260709 13:20:16.294548 20124 consensus_queue.cc:260] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:16.294939 20025 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:16.295763 20124 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:16.296052 15675 raft_consensus.cc:1240] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:16.297142 20025 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:16.292238 20155 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:16.301250 20128 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:16.215926 19210 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:16.236390 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:16.304282 16231 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:16.302692 19210 raft_consensus.cc:697] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:20:16.301525 20179 log.cc:927] Time spent T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.688s	user 0.002s	sys 0.000s
I20260709 13:20:16.204861 20086 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:16.307294 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:16.309082 15669 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:20:16.322347 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:16.332505 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 113211us
W20260709 13:20:16.217821 20199 log.cc:927] Time spent T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:20:16.336251 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:16.224628 20026 consensus_meta.cc:287] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.506s	user 0.006s	sys 0.001s
I20260709 13:20:16.341339 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:16.289762 20186 log.cc:927] Time spent T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.475s	user 0.004s	sys 0.000s
I20260709 13:20:16.193181 19978 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:20:16.234436 20214 log.cc:927] Time spent T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:20:16.339025 19210 consensus_queue.cc:237] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:16.343937 15521 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:20:16.333396 19962 consensus_meta.cc:287] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.652s	user 0.010s	sys 0.000s
I20260709 13:20:16.343832 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:16.233515 20034 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.352638 15522 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:16.308822 20155 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:16.215605 20123 consensus_queue.cc:1048] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.130s
I20260709 13:20:16.333287 20055 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:16.360553 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:16.221560 20213 consensus_queue.cc:1048] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.140s
I20260709 13:20:16.334009 20168 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:16.336184 20025 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:20:16.338006 20093 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:16.311476 20209 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:16.350745 20053 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:20:16.384100 15642 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:16.348181 20026 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:16.371410 20211 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.390514 20205 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
W20260709 13:20:16.391218 20212 log.cc:927] Time spent T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:20:16.351675 19978 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:16.391888 20215 log.cc:927] Time spent T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:20:16.395476 20213 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:20:16.399477 20142 consensus_meta.cc:287] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 1.362s	user 0.009s	sys 0.003s
I20260709 13:20:16.358336 20034 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:16.402906 20054 consensus_meta.cc:287] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.663s	user 0.007s	sys 0.002s
I20260709 13:20:16.403952 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 2104) took 1621 ms. Trace:
I20260709 13:20:16.427873 15522 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:16.408120 19787 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:16.408835 20168 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:16.409991 15670 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:20:16.411566 16231 raft_consensus.cc:1275] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:20:16.410020 16205 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:16.362509 20155 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:16.428303 15527 rpcz_store.cc:276] 0709 13:20:14.782572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:14.782671 (+    99us) service_pool.cc:224] Handling call
0709 13:20:14.782991 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:14.817459 (+ 34468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:14.817486 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:14.817934 (+   448us) log.cc:844] Serialized 26 byte log entry
0709 13:20:15.172606 (+354672us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b18004058e0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab0d1 00007f5ce49c6a0f
0709 13:20:15.456156 (+283550us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:15.531891 (+ 75735us) log.cc:844] Serialized 14 byte log entry
0709 13:20:15.532102 (+   211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:15.532174 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:15.633590 (+101416us) spinlock_profiling.cc:243] Waited 6.46 ms on lock 0x7b18004058e0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce191e3f3 00007f5ce19207c1 00007f5ce1920779 00007f5ce1920709 00007f5ce19206d1 00007f5ce191f9cd 000000000038f354 000000000038f199 00007f5ce190183a
0709 13:20:16.245091 (+611501us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b4800002958. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce9934aa1 00007f5ce992bb37 00007f5ce992b9ca 00007f5ce9a2326f 00007f5ce9a22f37 00007f5ce9a3755c 00007f5ce9a37508 00007f5ce9a37488 00007f5ce9a37443 00007f5ce9a366ec 00007f5ce8960a96
0709 13:20:16.246805 (+  1714us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:20:16.403736 (+156931us) raft_consensus.cc:1685] finished
0709 13:20:16.403772 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:16.403897 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37233,"prepare.queue_time_us":77250,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":42476,"raft.run_cpu_time_us":2716,"raft.run_wall_time_us":616773,"spinlock_wait_cycles":80252544,"thread_start_us":283392,"threads_started":1,"wal-append.queue_time_us":850186}
I20260709 13:20:16.371520 20219 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.091s
I20260709 13:20:16.423352 16235 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:16.424739 16234 raft_consensus.cc:2368] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11.
W20260709 13:20:16.406320 20043 consensus_meta.cc:287] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.522s	user 0.007s	sys 0.004s
I20260709 13:20:16.432655 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:16.441812 16236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53976 (request call id 992) took 1333 ms. Trace:
I20260709 13:20:16.371909 20055 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:16.363783 20132 consensus_queue.cc:260] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:16.413173 20050 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.008s	sys 0.015s
I20260709 13:20:16.440558 20068 consensus_queue.cc:1048] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:20:16.438891 20042 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:16.448961 20213 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:16.428093 20051 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:16.484309 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:16.452090 20209 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:16.462467 16239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57258 (request call id 957) took 1288 ms. Trace:
I20260709 13:20:16.093309 15523 rpcz_store.cc:276] 0709 13:20:15.042345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.455475 (+413130us) service_pool.cc:224] Handling call
0709 13:20:15.455862 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:15.456081 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:20:15.964037 (+507956us) log.cc:844] Serialized 14 byte log entry
0709 13:20:16.092161 (+128124us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:16.092294 (+   133us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:16.092385 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:16.092569 (+   184us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:16.092753 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1286,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":3818,"thread_start_us":126500,"threads_started":1}
I20260709 13:20:16.485560 15525 raft_consensus.cc:2410] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:20:16.442848 16236 rpcz_store.cc:276] 0709 13:20:15.098737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.098883 (+   146us) service_pool.cc:224] Handling call
0709 13:20:15.099208 (+   325us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:15.099650 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:15.099678 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:15.100297 (+   619us) log.cc:844] Serialized 52 byte log entry
0709 13:20:15.132415 (+ 32118us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:15.235324 (+102909us) log.cc:844] Serialized 14 byte log entry
0709 13:20:15.280336 (+ 45012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:15.280414 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:15.613601 (+333187us) spinlock_profiling.cc:243] Waited 3.24 ms on lock 0x7b180031ac20. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce191e3f3 00007f5ce19207c1 00007f5ce1920779 00007f5ce1920709 00007f5ce19206d1 00007f5ce191f9cd 000000000038f354 000000000038f199 00007f5ce190183a
0709 13:20:16.300904 (+687303us) spinlock_profiling.cc:243] Waited 64.1 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:20:16.431715 (+130811us) raft_consensus.cc:1685] finished
0709 13:20:16.431736 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:16.431880 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":284659,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":62,"raft.queue_time_us":47260,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":9451,"spinlock_wait_cycles":154947328,"thread_start_us":114968,"threads_started":2,"wal-append.queue_time_us":184948}
I20260709 13:20:16.463197 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:16.482892 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:16.487196 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:16.477150 15671 raft_consensus.cc:2393] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 9: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:16.485400 16239 rpcz_store.cc:276] 0709 13:20:15.174088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.174932 (+   844us) service_pool.cc:224] Handling call
0709 13:20:15.175242 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:15.175586 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:15.175615 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:15.616875 (+441260us) spinlock_profiling.cc:243] Waited 5.84 ms on lock 0x7b48002508d8. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce9934aa1 00007f5ce992bb37 00007f5ce992b9ca 00007f5ce9a2326f 00007f5ce9a22f37 00007f5ce9a3755c 00007f5ce9a37508 00007f5ce9a37488 00007f5ce9a37443 00007f5ce9a366ec 00007f5ce8960a96
0709 13:20:16.235496 (+618621us) spinlock_profiling.cc:243] Waited 3.96 ms on lock 0x7b2c00047828. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce3304a11 00007f5ce3314372 00007f5ce4a46a01 00007f5ce4a0b43a 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2
0709 13:20:16.235978 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 13:20:16.412817 (+176839us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:16.413046 (+   229us) log.cc:844] Serialized 14 byte log entry
0709 13:20:16.427856 (+ 14810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:16.427940 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:16.462258 (+ 34318us) raft_consensus.cc:1685] finished
0709 13:20:16.462278 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:16.462408 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14542,"prepare.queue_time_us":2353,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":50,"raft.queue_time_us":29697,"raft.run_cpu_time_us":2677,"raft.run_wall_time_us":436629,"spinlock_wait_cycles":22541056,"thread_start_us":176557,"threads_started":1,"wal-append.queue_time_us":224414}
I20260709 13:20:16.497830 15523 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:16.464491 15527 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:20:16.519245 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:16.518425 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:16.488888 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:16.518888 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:16.534391 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:16.445246 20217 log.cc:927] Time spent T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:20:16.536304 15522 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12. Ops: []
W20260709 13:20:16.461793 20181 consensus_meta.cc:287] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.668s	user 0.009s	sys 0.000s
W20260709 13:20:16.485255 20050 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.009s	sys 0.019s
I20260709 13:20:16.472054 20025 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.463201 20121 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:16.555213 20050 raft_consensus.cc:697] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:16.555194 20181 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:16.556463 16238 raft_consensus.cc:2368] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 14. Current term is 15.
I20260709 13:20:16.556354 20050 consensus_queue.cc:237] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:16.557178 20121 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:16.465144 19978 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:16.558389 20155 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:16.560170 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:16.560940 15522 raft_consensus.cc:2393] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 7: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:16.564034 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:16.564603 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:16.536803 20086 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.428107 20211 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:16.578347 20219 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:16.561028 20198 log.cc:927] Time spent T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.419s	user 0.000s	sys 0.001s
I20260709 13:20:16.475773 20043 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:16.551451 19701 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.606488 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:16.588479 20093 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:16.607695 15597 raft_consensus.cc:1813] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Handling vote request from an unknown peer 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:16.608175 15597 raft_consensus.cc:2368] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8.
I20260709 13:20:16.596751 20211 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:16.541292 20209 consensus_queue.cc:1059] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:20:16.599332 15523 raft_consensus.cc:1275] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:20:16.613030 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:16.600088 16204 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:16.615916 19856 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:16.575940 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:16.604557 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:16.605274 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:16.591647 20181 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:16.606956 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:16.555420 20025 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:16.560218 20034 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:16.623167 15599 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:20:16.623235 15677 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:20:16.628472 16233 raft_consensus.cc:2368] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 7. Current term is 8.
I20260709 13:20:16.626302 16204 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:16.624845 20082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:45438 (request call id 1) took 2745 ms. Trace:
I20260709 13:20:16.539729 20213 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:16.631830 20082 rpcz_store.cc:276] 0709 13:20:13.872933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:13.895685 (+ 22752us) service_pool.cc:224] Handling call
0709 13:20:14.020170 (+124485us) log.cc:844] Serialized 494 byte log entry
0709 13:20:14.618042 (+597872us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b18003abbe0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce191e3f3 00007f5ce19207c1 00007f5ce1920779 00007f5ce1920709 00007f5ce19206d1 00007f5ce191f9cd 000000000038f354 000000000038f199 00007f5ce190183a
0709 13:20:15.610330 (+992288us) spinlock_profiling.cc:243] Waited 577 ms on lock 0x7b58001f0ad0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4975090 00007f5ce496543c 00007f5ce496bd53 00007f5ce496bcd9 00007f5ce496bc69 00007f5ce496bc31 00007f5ce496adfd 000000000038f354 000000000038f199 00007f5ce191e475 00007f5ce19207c1
0709 13:20:16.613754 (+1003424us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1544,"raft.queue_time_us":3318,"raft.run_cpu_time_us":9206,"raft.run_wall_time_us":1074476,"spinlock_wait_cycles":1333319296,"thread_start_us":40815,"threads_started":1,"wal-append.queue_time_us":88640,"wal-append.run_cpu_time_us":8525,"wal-append.run_wall_time_us":1178824}
I20260709 13:20:16.632611 20034 raft_consensus.cc:3055] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:16.583943 20086 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:16.622810 15603 raft_consensus.cc:2955] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Committing config change with OpId 5.7: config changed from index 5 to 7, VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:16.645226 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:16.618440 20211 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:16.646250 20211 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:16.646562 15493 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:16.647459 20093 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:16.647946 20093 raft_consensus.cc:697] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:16.651568 15493 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:16.652980 15599 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 15.
I20260709 13:20:16.653050 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:16.654163 15493 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:16.654011 16205 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:16.641178 20181 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:16.617713 19701 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:16.657703 16237 raft_consensus.cc:606] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:16.659350 20223 raft_consensus.cc:2804] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:20:16.660776 20194 log.cc:927] Time spent T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.623s	user 0.002s	sys 0.000s
I20260709 13:20:16.690913 20213 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:16.662029 16234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57258 (request call id 961) took 1021 ms. Trace:
I20260709 13:20:16.662986 20043 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:16.612620 20219 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:16.692731 15673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2526) took 1347 ms. Trace:
I20260709 13:20:16.694197 20219 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:16.694440 16238 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57244 (request call id 1380) took 1063 ms. Trace:
I20260709 13:20:16.694780 16238 rpcz_store.cc:276] 0709 13:20:15.630811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.633536 (+  2725us) service_pool.cc:224] Handling call
0709 13:20:16.694224 (+1060688us) spinlock_profiling.cc:243] Waited 899 ms on lock 0x7b54001e50d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:16.694355 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2068414208}
I20260709 13:20:16.695423 15598 raft_consensus.cc:1275] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:20:16.694716 20154 consensus_meta.cc:287] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.761s	user 0.010s	sys 0.000s
W20260709 13:20:16.695819 20197 log.cc:927] Time spent T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.611s	user 0.000s	sys 0.002s
I20260709 13:20:16.691942 16234 rpcz_store.cc:276] 0709 13:20:15.640298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.651571 (+ 11273us) service_pool.cc:224] Handling call
0709 13:20:16.661798 (+1010227us) spinlock_profiling.cc:243] Waited 714 ms on lock 0x7b54001e1754. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:16.661968 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1641972480}
I20260709 13:20:16.696066 20154 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:16.697268 16232 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:16.699183 19724 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:16.701195 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:16.693106 15673 rpcz_store.cc:276] 0709 13:20:15.345178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.428172 (+ 82994us) service_pool.cc:224] Handling call
0709 13:20:15.459048 (+ 30876us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:15.462152 (+  3104us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:15.462179 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:15.518084 (+ 55905us) log.cc:844] Serialized 26 byte log entry
0709 13:20:15.792329 (+274245us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:20:15.792367 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:15.792426 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:16.660073 (+867647us) spinlock_profiling.cc:243] Waited 3.33 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:20:16.692423 (+ 32350us) raft_consensus.cc:1685] finished
0709 13:20:16.692444 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:16.692667 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1822,"raft.queue_time_us":2771,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":48068,"spinlock_wait_cycles":7648640,"thread_start_us":227094,"threads_started":1,"wal-append.queue_time_us":391764}
I20260709 13:20:16.666872 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:16.706126 15599 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:20:16.667212 15493 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:16.644155 20192 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.020s	sys 0.022s
I20260709 13:20:16.668490 19856 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:16.669370 19701 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:20:16.670776 20055 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:16.671583 20082 raft_consensus.cc:993] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9: : Instructing follower d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:20:16.672273 20086 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:16.642992 20034 raft_consensus.cc:740] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:16.672827 20093 consensus_queue.cc:237] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:16.695585 20025 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:16.629850 20060 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:20:16.658150 19978 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:16.686245 16240 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
W20260709 13:20:16.688764 20120 consensus_meta.cc:287] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 1.009s	user 0.009s	sys 0.000s
I20260709 13:20:16.689432 20223 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:16.690306 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:16.741519 20120 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11.
I20260709 13:20:16.746749 20219 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.747393 20219 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:16.752719 20168 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:16.741577 20223 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:16.739600 19978 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:16.754319 19978 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:16.758246 20210 raft_consensus.cc:1081] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Signalling peer d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:20:16.761953 20026 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:16.766321 20025 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:16.766983 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:16.766925 20025 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:16.767727 15523 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:16.769106 20025 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:16.771411 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:16.773533 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:16.773854 15527 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:56840
I20260709 13:20:16.774691 15527 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:16.775053 15527 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:16.775341 15412 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0af5649725244ae599dfc6ed7aeac5f1 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:20:16.753307 20168 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:16.786285 15600 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:16.749656 20219 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:16.797946 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:16.798458 15527 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:16.730064 20237 log.cc:927] Time spent T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:20:16.708058 15493 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:16.713156 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:16.714110 15642 leader_election.cc:400] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8.
I20260709 13:20:16.714666 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:16.714954 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:16.800945 15642 leader_election.cc:403] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:16.801159 15596 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:20:16.718245 19787 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:16.718284 19856 raft_consensus.cc:697] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:16.718909 20149 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:16.720640 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:16.726899 15599 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:20:16.726989 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:16.711138 20192 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.020s	sys 0.023s
I20260709 13:20:16.730028 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:16.740033 19445 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.715020 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:16.798645 15669 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:20:16.801327 15594 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:20:16.804694 15523 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 15.
I20260709 13:20:16.805287 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:17.010978 15527 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:17.011112 15602 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
W20260709 13:20:16.992158 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:16.769853 20025 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:17.024456 15674 raft_consensus.cc:1240] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:16.806089 20123 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:16.752338 20154 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:16.801738 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 93757.4us
I20260709 13:20:16.994971 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:16.784564 20168 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:16.805948 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 91987.7us
I20260709 13:20:16.752127 20211 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:16.847719 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 44021.2us
W20260709 13:20:16.999146 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 200539us
I20260709 13:20:17.041584 15568 leader_election.cc:400] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 14. Current term is 15.
I20260709 13:20:17.041958 15568 leader_election.cc:403] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:17.000571 20219 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:20:17.029587 15669 raft_consensus.cc:1275] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:20:17.052714 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:17.053000 15568 leader_election.cc:400] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 7. Current term is 8.
I20260709 13:20:17.053474 15568 leader_election.cc:403] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:17.031240 15524 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:17.056619 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:17.032826 16237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:52820 (request call id 11) took 1466 ms. Trace:
I20260709 13:20:16.992471 20149 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:17.037669 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:17.061851 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:17.018128 15600 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
W20260709 13:20:17.048101 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 53402.3us
I20260709 13:20:17.034278 16205 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:17.058452 16237 rpcz_store.cc:276] 0709 13:20:15.557752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.558356 (+   604us) service_pool.cc:224] Handling call
0709 13:20:17.015334 (+1456978us) spinlock_profiling.cc:243] Waited 918 ms on lock 0x7b54006aac54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a05853 00007f5ce99d360a 00007f5ce2f597d4 00007f5ce2f59762 00007f5ce2f59691 00007f5ce2f5960c 00007f5ce2f588c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:17.015419 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2111217536}
I20260709 13:20:17.065418 15603 raft_consensus.cc:1240] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:17.064265 15673 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:17.067507 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:17.069206 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:17.072335 15642 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:17.028024 20183 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:17.078657 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:17.079432 15669 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:17.033639 20123 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:17.081398 16233 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:17.088423 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:17.081693 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 53789.7us
I20260709 13:20:17.082674 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:17.091646 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 50154.7us
W20260709 13:20:17.092175 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 57997.3us
I20260709 13:20:17.071539 20149 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:17.028672 20055 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:20:17.060185 20142 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 14. Current term is 15.
I20260709 13:20:17.071705 19585 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 13:20:17.026332 19445 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:17.102615 20203 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:17.085886 20086 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:17.108268 15524 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:17.029523 20025 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:17.086169 20110 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 7. Current term is 8.
I20260709 13:20:17.109368 16205 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:17.087589 20183 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:17.088308 20123 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:17.059962 20148 consensus_queue.cc:1059] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:20:17.122016 16203 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:20:17.116751 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 86056.4us
I20260709 13:20:17.110371 19888 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.112908 20202 raft_consensus.cc:2804] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:17.173084 16232 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
W20260709 13:20:17.174384 20190 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.936s	user 0.036s	sys 0.054s
W20260709 13:20:17.177541 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 68249.4us
W20260709 13:20:17.180841 20190 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.041s	user 0.037s	sys 0.054s
I20260709 13:20:17.190304 16237 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:20:17.191609 16237 raft_consensus.cc:606] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:17.092342 19856 consensus_queue.cc:237] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:17.187911 15493 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:17.193252 20110 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:17.200937 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 79011.1us
I20260709 13:20:17.203950 20080 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:17.178687 19888 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:17.208734 19888 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:17.192492 20202 raft_consensus.cc:697] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:17.212128 20223 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:17.212543 20202 consensus_queue.cc:237] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:17.213768 15602 raft_consensus.cc:1275] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:20:17.214879 20245 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:17.216696 15522 raft_consensus.cc:2393] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 8: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:17.218459 15671 raft_consensus.cc:1275] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:20:17.224213 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:17.187425 20067 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:17.228884 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:17.232750 20183 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:17.226989 20110 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:17.237608 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:17.239462 20110 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:17.246236 20239 log.cc:927] Time spent T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.058s	user 0.000s	sys 0.002s
I20260709 13:20:17.247833 19978 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:17.248457 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:17.227461 20151 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:20:17.258334 19888 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:17.258912 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:17.259563 15603 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:17.261255 19210 consensus_queue.cc:1048] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:17.259451 20067 raft_consensus.cc:697] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:17.264801 20067 consensus_queue.cc:237] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:17.270262 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 192308us
I20260709 13:20:17.270968 15641 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:17.272904 15641 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:17.274309 15641 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:17.275137 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:17.275869 15600 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:20:17.275947 15641 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:17.257519 15567 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:17.278873 15567 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:17.228052 20219 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8.
I20260709 13:20:17.287335 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:17.290621 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:17.286329 15567 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:17.286517 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:17.299965 16205 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:17.305603 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:17.306277 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:17.306907 15603 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:17.310253 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:17.310969 16233 raft_consensus.cc:3055] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:17.311275 16233 raft_consensus.cc:740] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:17.316824 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:17.319757 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:17.322166 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:17.323148 16236 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 13:20:17.324239 15493 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.68:39627 timed out after 1.529s (SENT) [suppressed 20 similar messages]
I20260709 13:20:17.325103 15602 raft_consensus.cc:2393] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 8: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:17.287660 20055 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:17.312055 16233 consensus_queue.cc:260] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:17.327797 16233 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:17.288976 20110 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:17.298584 20223 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:17.324769 20051 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:17.322963 19895 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:17.361858 20051 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:17.335520 20195 raft_consensus.cc:2955] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Committing config change with OpId 7.8: config changed from index 5 to 8, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:17.337581 15672 raft_consensus.cc:2955] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Committing config change with OpId 7.8: config changed from index 5 to 8, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:17.362528 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1472) took 2330 ms. Trace:
I20260709 13:20:17.338533 20025 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:17.362908 15519 rpcz_store.cc:276] 0709 13:20:15.032005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.112382 (+ 80377us) service_pool.cc:224] Handling call
0709 13:20:16.239582 (+1127200us) spinlock_profiling.cc:243] Waited 3.19 ms on lock 0x7b5800003698. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99e4fc0 00007f5ce99cf929 00007f5ce99cf3d2 00007f5ce2f46bd4
0709 13:20:16.239795 (+   213us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:16.415210 (+175415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:16.415241 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:16.415277 (+    36us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:16.415305 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:17.362284 (+946979us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b54003ccdd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:17.362331 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:17.362447 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":410321664}
I20260709 13:20:17.258904 19888 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:17.360059 19978 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:17.360498 15670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 2139) took 1237 ms. Trace:
I20260709 13:20:17.350351 16233 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:20:17.369221 15603 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:20:17.383091 15670 rpcz_store.cc:276] 0709 13:20:16.123566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:16.123867 (+   301us) service_pool.cc:224] Handling call
0709 13:20:17.360313 (+1236446us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b540009c9d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:17.360430 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":483568896}
I20260709 13:20:17.362340 20051 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:17.383210 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 284947us
I20260709 13:20:17.385171 20055 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:17.341621 20123 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:17.378536 20236 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:17.362083 19895 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:17.406240 20219 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:20:17.412513 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:17.386013 20025 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:17.410550 15566 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:17.448884 20034 consensus_queue.cc:260] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:17.450057 20034 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:17.464366 20223 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:17.470495 20223 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:17.470978 19888 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:17.472966 20223 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:17.446276 19585 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
W20260709 13:20:17.482918 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 207046us
I20260709 13:20:17.486498 15641 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:17.439415 20110 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:17.438633 20025 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.437883 20190 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.493753 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:17.421875 20247 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:20:17.498365 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 205639us
I20260709 13:20:17.501068 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:17.487427 20037 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:17.504384 16239 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:20:17.442893 15566 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:17.511173 15519 raft_consensus.cc:1240] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:17.512485 15567 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:17.517372 15524 raft_consensus.cc:1275] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:20:17.509616 20190 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:17.525303 20190 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:17.462019 20055 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:20:17.530258 20211 consensus_queue.cc:1048] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:17.525826 20190 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:17.543174 20050 raft_consensus.cc:3055] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:17.543715 20050 raft_consensus.cc:740] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:17.544555 20050 consensus_queue.cc:260] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:17.546610 19895 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:17.533456 20149 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:17.548759 20051 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:17.550565 20149 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:17.553031 20123 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:17.555236 20204 log.cc:927] Time spent T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 1.470s	user 0.000s	sys 0.005s
I20260709 13:20:17.556612 20132 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:17.558286 15524 raft_consensus.cc:1275] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:20:17.562611 20051 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:17.569543 20249 log.cc:927] Time spent T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:20:17.570933 15524 raft_consensus.cc:1275] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:20:17.575619 20244 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.396s	user 0.001s	sys 0.001s
I20260709 13:20:17.590615 19445 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:17.595748 19895 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:17.597911 20154 raft_consensus.cc:993] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9: : Instructing follower d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:20:17.600148 20024 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:17.601652 19445 raft_consensus.cc:1081] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Signalling peer d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:20:17.604643 20051 consensus_queue.cc:1048] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:17.607064 20149 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:17.607673 20149 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:17.609396 20240 log.cc:927] Time spent T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.430s	user 0.001s	sys 0.000s
I20260709 13:20:17.609942 20149 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:17.610428 20142 raft_consensus.cc:3055] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:17.610796 20142 raft_consensus.cc:740] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:17.611676 20142 consensus_queue.cc:260] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:17.612826 20142 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:17.615525 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:17.619004 15519 raft_consensus.cc:1240] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10. Ops: []
W20260709 13:20:17.619786 16240 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57258 (request call id 942) took 2895 ms (client timeout 1529 ms). Trace:
W20260709 13:20:17.620208 16240 rpcz_store.cc:269] 0709 13:20:14.724379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:14.724596 (+   217us) service_pool.cc:224] Handling call
0709 13:20:17.619603 (+2895007us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b540028e6d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:17.619696 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3875231616}
I20260709 13:20:17.622372 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:17.623023 15519 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:17.635835 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:17.636433 15523 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:56840
I20260709 13:20:17.636499 15668 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:17.637171 15523 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:17.637533 15523 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:17.644439 15523 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:17.646853 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 135341us
W20260709 13:20:17.648277 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:17.652741 20105 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:17.652825 20267 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.510281 20025 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:17.545733 20050 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:17.662971 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 338752us
I20260709 13:20:17.666258 15642 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:17.677103 15523 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:17.679164 15675 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:20:17.679832 15675 raft_consensus.cc:606] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:17.680271 15675 raft_consensus.cc:614] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:17.679867 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:17.686046 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:17.687144 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:17.688220 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:17.689177 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:17.689409 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:17.689824 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:17.690743 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:17.692497 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:17.693749 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:17.690444 15525 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:17.697324 15525 raft_consensus.cc:1240] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:17.698119 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:17.698810 15528 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:17.701678 15520 raft_consensus.cc:1240] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:20:17.707165 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:17.711398 15598 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:17.710999 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:17.719471 15521 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
W20260709 13:20:17.721246 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 55077.3us
I20260709 13:20:17.722186 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:17.723455 15642 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:20:17.724785 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 51743.7us
I20260709 13:20:17.725500 16204 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:17.727517 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 284684us
W20260709 13:20:17.729565 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 43941.9us
I20260709 13:20:17.679359 20110 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:17.728289 15566 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:17.738953 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:17.680461 20267 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:17.737011 15493 leader_election.cc:400] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11.
I20260709 13:20:17.740063 15493 leader_election.cc:403] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:20:17.741746 16203 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:17.728963 20247 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:20:17.737567 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:17.746870 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:17.748544 15668 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:20:17.748992 15668 pending_rounds.cc:85] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting all ops after (but not including) 10
I20260709 13:20:17.749233 15668 pending_rounds.cc:107] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:20:17.749464 15668 raft_consensus.cc:2889] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:17.753226 15493 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:17.777882 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:17.780011 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:17.783022 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:17.738206 20098 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:17.791723 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:17.787915 15493 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:17.789288 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 47840.5us
W20260709 13:20:17.795186 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 66443.3us
I20260709 13:20:17.734158 20264 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:20:17.796110 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 54473.1us
I20260709 13:20:17.793853 15493 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:17.797825 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:17.752130 20247 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:17.775334 20051 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:17.754022 20103 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:17.781946 19888 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11.
I20260709 13:20:17.785615 20160 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:17.739533 20110 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:17.738965 19724 consensus_queue.cc:1059] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
W20260709 13:20:17.813201 19856 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.021s	sys 0.017s
I20260709 13:20:17.795866 20098 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:17.794903 20149 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:17.794708 20192 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:17.824105 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:17.824923 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:17.751529 20263 consensus_queue.cc:1059] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 13:20:17.797255 20223 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:17.806057 20068 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:17.810380 20247 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:17.812803 20168 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:17.845152 16232 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
W20260709 13:20:17.826574 19856 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.021s	sys 0.017s
W20260709 13:20:17.846303 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 46915.9us
I20260709 13:20:17.812847 20051 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:17.859712 20051 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:17.815016 19968 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:17.856297 15642 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:17.856848 20168 raft_consensus.cc:697] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:17.816181 20257 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
W20260709 13:20:17.866935 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 61780.6us
I20260709 13:20:17.843506 20098 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:17.819913 20103 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:17.856256 15420 catalog_manager.cc:5697] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 6 to 10, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 10 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:17.892957 20266 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:17.862236 19968 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:17.913277 15595 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:20:17.922582 16205 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:17.926100 20183 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.927289 19895 raft_consensus.cc:3055] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:17.931548 20286 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:17.932946 20168 consensus_queue.cc:237] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:17.961774 15420 catalog_manager.cc:5697] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:17.940713 20051 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:17.942233 20253 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:17.947288 20110 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:17.848258 20068 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:17.949965 20267 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:17.849119 20192 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:17.956837 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:18.020852 15420 catalog_manager.cc:5697] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 7 to 9, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 9 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:18.022042 16232 raft_consensus.cc:2410] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:20:18.023591 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:18.024956 16237 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:20:17.958029 20266 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:18.029827 20266 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:20:18.030190 20269 log.cc:927] Time spent T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 13:20:18.030786 20275 log.cc:927] Time spent T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:20:18.031379 20195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:45438 (request call id 0) took 4158 ms. Trace:
I20260709 13:20:17.969017 20183 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:18.037717 15420 catalog_manager.cc:5697] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 8 to 9, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 9 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:18.035198 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:17.960733 19968 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:18.036594 15595 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34102 (request call id 1222) took 1597 ms (client timeout 1651 ms). Trace:
W20260709 13:20:18.039522 15595 rpcz_store.cc:269] 0709 13:20:16.432253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:16.432405 (+   152us) service_pool.cc:224] Handling call
0709 13:20:18.029109 (+1596704us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b540025ba54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:18.029236 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3384237824}
I20260709 13:20:17.960532 19895 raft_consensus.cc:740] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:18.017401 20026 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:18.022006 20068 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:18.072178 20236 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:18.072860 20236 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:18.076020 20192 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:18.077479 20103 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:18.077843 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:18.078392 20271 raft_consensus.cc:3055] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:18.078832 20271 raft_consensus.cc:740] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:18.075295 20236 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:18.020778 20253 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:18.084961 20253 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:18.086593 20051 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:18.021313 20086 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:18.087384 20086 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:18.088097 20068 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:18.057242 19895 consensus_queue.cc:260] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:18.058238 16237 raft_consensus.cc:606] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:18.100492 16237 raft_consensus.cc:614] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:18.059484 20183 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:18.079720 20271 consensus_queue.cc:260] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:18.046500 15420 catalog_manager.cc:5697] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:18.081517 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:18.089479 20086 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:18.036875 20025 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:18.096033 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 43918.5us
I20260709 13:20:18.073328 20247 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:18.031778 20195 rpcz_store.cc:276] 0709 13:20:13.872639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:13.872769 (+   130us) service_pool.cc:224] Handling call
0709 13:20:14.951758 (+1078989us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b54008a0aa0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4982ec4 00007f5ce4a068d7 00007f5ce4a1203c 00007f5ce4a116b3 00007f5ce4a099fa 00007f5ce99d14c9 00007f5ce2f4dc54 00007f5ce2f4dbe2 00007f5ce2f4db11 00007f5ce2f4da8c
0709 13:20:14.969791 (+ 18033us) log.cc:844] Serialized 494 byte log entry
0709 13:20:15.626434 (+656643us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b54008a0aa0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4984142 00007f5ce4964eaa 00007f5ce496bd53 00007f5ce496bcd9 00007f5ce496bc69 00007f5ce496bc31 00007f5ce496adfd 000000000038f354 000000000038f199 00007f5ce191e475
0709 13:20:16.289490 (+663056us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b54008a0aa0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce498bb26 00007f5ce498549a 00007f5ce4984714 00007f5ce4964eaa 00007f5ce496bd53 00007f5ce496bcd9 00007f5ce496bc69 00007f5ce496bc31 00007f5ce496adfd 000000000038f354
0709 13:20:16.719803 (+430313us) spinlock_profiling.cc:243] Waited 24.9 ms on lock 0x7b4c004c3340. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce2cd7a20 00007f5ce2ce8a29 00007f5ce2ce913f 00007f5ce2cad2ab 00007f5ce2cdaf3c 00007f5ce2cdb3fe 00007f5ce2f6850b 00007f5ce4967381 00007f5ce496526b 00007f5ce496bd53 00007f5ce496bcd9
0709 13:20:18.031308 (+1311505us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23606,"raft.run_cpu_time_us":21657,"raft.run_wall_time_us":2849580,"spinlock_wait_cycles":119020672,"thread_start_us":87861,"threads_started":1,"wal-append.queue_time_us":184248,"wal-append.run_cpu_time_us":2645,"wal-append.run_wall_time_us":1248455}
I20260709 13:20:18.090330 19968 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:18.056347 20160 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:18.102615 16232 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
W20260709 13:20:18.096268 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:18.107108 15420 catalog_manager.cc:5697] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 7 to 14, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 14 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:20:18.108206 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:18.109793 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:18.130133 15671 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:18.130553 15642 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:18.130851 15568 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:18.142046 15420 catalog_manager.cc:5697] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
W20260709 13:20:18.129582 15519 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.010s	sys 0.019s
I20260709 13:20:18.122905 16205 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
W20260709 13:20:18.144986 15519 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.011s	sys 0.019s
I20260709 13:20:18.153303 15412 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b99eaeae24b44aa78cfa8c90caca288e with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:20:18.155016 15420 catalog_manager.cc:5697] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 8 to 9, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 9 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:18.207775 15420 catalog_manager.cc:5697] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:18.142046 20025 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:18.237339 20286 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:18.198776 19724 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:18.228295 15420 catalog_manager.cc:5697] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 8 to 10, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 10 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:18.237891 20286 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:18.256390 19724 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:18.262950 19724 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:18.263598 19968 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:20:18.256132 20286 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:18.278156 16237 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:20:18.278920 16237 raft_consensus.cc:606] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:18.279242 16237 raft_consensus.cc:614] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:18.218634 20025 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:18.266796 20068 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:18.294940 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 483328us
I20260709 13:20:18.298344 15420 catalog_manager.cc:5697] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 7 to 13, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 13 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:20:18.306288 15522 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.004s	sys 0.019s
W20260709 13:20:18.306665 15522 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.004s	sys 0.020s
I20260709 13:20:18.306006 15420 catalog_manager.cc:5697] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:20:18.291276 20273 log.cc:927] Time spent T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:20:18.341115 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:18.305717 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:20:18.352144 15420 catalog_manager.cc:5697] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:18.141625 20160 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:18.362939 20257 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:18.296015 20025 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:18.353734 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:20:18.345850 20050 consensus_meta.cc:287] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.654s	user 0.010s	sys 0.000s
W20260709 13:20:18.379946 19888 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.012s	sys 0.024s
I20260709 13:20:18.303776 20209 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:18.420883 15603 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:20:18.345373 20240 log.cc:927] Time spent T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:20:18.421123 16233 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.003s	sys 0.011s
I20260709 13:20:18.440374 15600 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:20:18.420625 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:18.431879 15523 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:20:18.445190 15599 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:18.354176 20086 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:18.280357 20098 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:18.365972 20255 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.016s	sys 0.019s
W20260709 13:20:18.442975 16233 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.003s	sys 0.011s
W20260709 13:20:18.423543 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 292744us
W20260709 13:20:18.432736 20123 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.008s	sys 0.032s
W20260709 13:20:18.452229 15566 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.006s	sys 0.015s
I20260709 13:20:18.470029 20086 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:18.472088 20274 log.cc:927] Time spent T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.192s	user 0.000s	sys 0.000s
W20260709 13:20:18.485085 16240 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.007s	sys 0.020s
W20260709 13:20:18.486071 16240 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.007s	sys 0.020s
I20260709 13:20:18.414950 15420 catalog_manager.cc:5697] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 5 to 6, term changed from 4 to 6, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68), VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New cstate: current_term: 6 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:20:18.478473 15566 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.007s	sys 0.016s
W20260709 13:20:18.440881 19888 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.012s	sys 0.024s
W20260709 13:20:18.472505 20255 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.016s	sys 0.019s
I20260709 13:20:18.494237 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:18.495153 15521 raft_consensus.cc:1240] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11. Ops: []
W20260709 13:20:18.482132 20123 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.009s	sys 0.033s
I20260709 13:20:18.488811 15523 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:20:18.489434 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 761181us
I20260709 13:20:18.489630 15668 raft_consensus.cc:2368] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12.
I20260709 13:20:18.503753 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:18.504354 15523 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:18.462854 20209 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:18.433534 20051 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:18.496989 20024 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:18.527215 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:18.528472 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:18.525350 15520 raft_consensus.cc:2955] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Committing config change with OpId 7.8: config changed from index 5 to 8, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:18.415648 20257 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:18.493850 15420 catalog_manager.cc:5697] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:20:18.441386 19656 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.004s	sys 0.016s
I20260709 13:20:18.515242 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:18.497786 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:18.490046 16234 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.001s	sys 0.016s
I20260709 13:20:18.433910 20271 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:19.077744 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:20:18.655958 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 127746us
W20260709 13:20:18.714442 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:19.078984 16234 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.216s	user 0.001s	sys 0.020s
I20260709 13:20:18.666496 15599 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
W20260709 13:20:18.990813 15641 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.014s	sys 0.008s
I20260709 13:20:19.080504 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:19.087322 19888 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:18.618944 15523 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:19.081992 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:18.632726 15493 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.826s	user 0.007s	sys 0.007s
W20260709 13:20:19.111637 15493 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.305s	user 0.010s	sys 0.009s
W20260709 13:20:19.108103 15641 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.017s	sys 0.011s
I20260709 13:20:19.111840 15675 raft_consensus.cc:2393] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 12: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:19.111896 15668 raft_consensus.cc:2410] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 13:20:19.112004 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 1.30523e+06us
I20260709 13:20:19.112833 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:19.114071 15673 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.020s	sys 0.016s
I20260709 13:20:19.114657 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:19.117048 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:20:18.813129 20278 log.cc:927] Time spent T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.547s	user 0.008s	sys 0.006s
I20260709 13:20:18.889012 20236 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:19.118831 20236 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.119634 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:20:19.112193 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 744892us
W20260709 13:20:19.120589 19724 consensus_meta.cc:287] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.857s	user 0.006s	sys 0.003s
I20260709 13:20:19.120798 20236 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:19.121253 19738 consensus_queue.cc:1059] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:19.122517 20236 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:19.088785 15420 catalog_manager.cc:5697] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:20:18.786196 20168 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.006s	sys 0.019s
W20260709 13:20:18.706043 20282 log.cc:927] Time spent T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
W20260709 13:20:18.955300 20286 consensus_meta.cc:287] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.689s	user 0.007s	sys 0.004s
W20260709 13:20:18.609696 20279 log.cc:927] Time spent T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
W20260709 13:20:18.912051 20280 log.cc:927] Time spent T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.549s	user 0.000s	sys 0.002s
I20260709 13:20:18.861554 20098 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:19.134698 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:20:18.625387 20252 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.013s	sys 0.038s
I20260709 13:20:18.529647 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:18.519639 20266 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:18.662392 20258 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.730s	user 0.015s	sys 0.002s
W20260709 13:20:19.078625 19656 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.217s	user 0.004s	sys 0.029s
I20260709 13:20:18.529260 20160 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:18.813863 20107 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:18.521073 20247 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:18.862345 20086 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:18.862169 20209 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:18.841147 20060 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.295s
I20260709 13:20:19.137569 15420 catalog_manager.cc:5697] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 7 to 10, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 10 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:19.159368 15677 raft_consensus.cc:1240] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 12. Current term is 13. Ops: []
W20260709 13:20:19.134253 15528 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.008s	sys 0.009s
I20260709 13:20:19.139874 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:20:19.159890 15528 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.009s	sys 0.009s
I20260709 13:20:19.140827 16240 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:19.160472 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:19.164933 15491 leader_election.cc:400] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12.
I20260709 13:20:19.165294 15491 leader_election.cc:403] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:19.013267 20073 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:19.144198 15598 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:20:19.168242 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2175) took 1480 ms. Trace:
I20260709 13:20:19.154073 15641 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:19.147850 15522 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.016s	sys 0.060s
W20260709 13:20:19.131114 20168 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.064s	user 0.006s	sys 0.023s
W20260709 13:20:19.134415 20295 log.cc:927] Time spent T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.345s	user 0.001s	sys 0.001s
I20260709 13:20:18.551234 20024 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.132817 19724 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.174454 16203 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:19.143815 20252 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.281s	user 0.021s	sys 0.067s
W20260709 13:20:19.114938 15673 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.020s	sys 0.016s
I20260709 13:20:18.842695 20292 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.328s
W20260709 13:20:19.147727 20258 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.215s	user 0.022s	sys 0.027s
I20260709 13:20:19.182327 20024 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:19.160287 20287 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:19.152413 20202 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.007s	sys 0.015s
W20260709 13:20:19.169212 15522 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.016s	sys 0.060s
I20260709 13:20:19.168610 15528 rpcz_store.cc:276] 0709 13:20:17.688107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:17.699390 (+ 11283us) service_pool.cc:224] Handling call
0709 13:20:17.699684 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:17.700163 (+   479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:17.700193 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:17.725797 (+ 25604us) log.cc:844] Serialized 26 byte log entry
0709 13:20:18.227630 (+501833us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:19.167155 (+939525us) log.cc:844] Serialized 14 byte log entry
0709 13:20:19.167933 (+   778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.168005 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:19.168049 (+    44us) raft_consensus.cc:1685] finished
0709 13:20:19.168061 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:19.168184 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":494,"prepare.queue_time_us":383852,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":181,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":14975,"thread_start_us":1188689,"threads_started":2,"wal-append.queue_time_us":966269}
I20260709 13:20:18.551273 20051 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.171808 16236 raft_consensus.cc:2368] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12.
I20260709 13:20:19.130056 19888 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.123236 20236 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.154568 20151 raft_consensus.cc:3055] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:19.179330 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:20:19.162851 20025 consensus_meta.cc:287] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.730s	user 0.009s	sys 0.001s
I20260709 13:20:19.170586 20160 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 12
W20260709 13:20:19.154882 16239 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.001s	sys 0.033s
I20260709 13:20:19.191795 15420 catalog_manager.cc:5697] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 6 to 10, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 10 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:19.196895 15524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1526) took 1618 ms. Trace:
I20260709 13:20:19.177883 20098 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:19.177634 20086 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:19.193971 20202 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.008s	sys 0.015s
I20260709 13:20:19.153990 20107 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.221204 20107 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:19.197633 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1536) took 1531 ms. Trace:
I20260709 13:20:19.209697 15524 rpcz_store.cc:276] 0709 13:20:17.578303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:17.579121 (+   818us) service_pool.cc:224] Handling call
0709 13:20:17.579433 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:17.579776 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:17.579805 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:17.580271 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 13:20:18.041683 (+461412us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:18.041726 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:18.041795 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:18.911346 (+869551us) spinlock_profiling.cc:243] Waited 2.6 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:20:19.196593 (+285247us) raft_consensus.cc:1685] finished
0709 13:20:19.196614 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:19.196836 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27034,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":73,"spinlock_wait_cycles":5971712,"thread_start_us":461061,"threads_started":1,"wal-append.queue_time_us":782715}
I20260709 13:20:19.194557 19727 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:19.217905 20098 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:19.199856 20287 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:19.242671 20287 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:19.203804 20151 raft_consensus.cc:740] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:19.226321 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1533) took 1593 ms. Trace:
W20260709 13:20:19.226675 16237 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.017s	sys 0.027s
W20260709 13:20:19.245983 16237 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.017s	sys 0.027s
I20260709 13:20:19.222831 15522 rpcz_store.cc:276] 0709 13:20:17.666434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:17.666544 (+   110us) service_pool.cc:224] Handling call
0709 13:20:17.707272 (+ 40728us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:17.707644 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:17.707672 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:17.708252 (+   580us) log.cc:844] Serialized 52 byte log entry
0709 13:20:17.858836 (+150584us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b4800002958. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce9934aa1 00007f5ce992bb37 00007f5ce992b9ca 00007f5ce9a2326f 00007f5ce9a22f37 00007f5ce9a3755c 00007f5ce9a37508 00007f5ce9a37488 00007f5ce9a37443 00007f5ce9a366ec 00007f5ce8960a96
0709 13:20:18.307061 (+448225us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:19.189774 (+882713us) log.cc:844] Serialized 14 byte log entry
0709 13:20:19.190020 (+   246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.190098 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:19.190153 (+    55us) raft_consensus.cc:1685] finished
0709 13:20:19.197423 (+  7270us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:19.197572 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":844,"prepare.queue_time_us":399091,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":1174,"raft.run_cpu_time_us":6229,"raft.run_wall_time_us":150106,"spinlock_wait_cycles":5111680,"thread_start_us":1288155,"threads_started":2,"wal-append.queue_time_us":963805}
I20260709 13:20:19.230552 15420 catalog_manager.cc:5697] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 7 to 8, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 8 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:20:19.232251 16239 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.001s	sys 0.033s
I20260709 13:20:19.141358 20286 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.245329 15527 rpcz_store.cc:276] 0709 13:20:17.633210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:17.633322 (+   112us) service_pool.cc:224] Handling call
0709 13:20:17.633649 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:17.634001 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:17.634028 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:17.652495 (+ 18467us) log.cc:844] Serialized 26 byte log entry
0709 13:20:18.042317 (+389822us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:18.042357 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:18.042419 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:19.226085 (+1183666us) raft_consensus.cc:1685] finished
0709 13:20:19.226108 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:19.226260 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9532,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":4671,"thread_start_us":384392,"threads_started":1,"wal-append.queue_time_us":823014}
I20260709 13:20:19.176585 20073 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.258093 15420 catalog_manager.cc:5697] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:19.227035 19727 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:19.211758 20025 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.272997 15420 catalog_manager.cc:5697] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 6 to 8, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 8 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:19.223711 20086 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.271467 19727 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:19.287133 15420 catalog_manager.cc:5697] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:19.288794 15420 catalog_manager.cc:5697] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:20:19.272117 20051 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:19.290755 20151 consensus_queue.cc:260] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:19.300859 15553 heartbeater.cc:646] Failed to heartbeat to 127.14.23.126:39251 (3 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.14.23.126:39251 timed out after 15.000s (SENT)
I20260709 13:20:19.281881 19888 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:19.308848 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 223032us
I20260709 13:20:19.312129 16237 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:20:19.315917 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:19.316812 15598 raft_consensus.cc:2393] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 12: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:19.318029 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:19.318723 15599 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:20:19.320358 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:19.320924 15599 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:19.323589 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:19.325475 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:19.326149 15603 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
W20260709 13:20:19.334793 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:19.352277 20287 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:19.352991 20287 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.332914 20252 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:19.354403 20252 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.356721 20252 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:19.363466 20292 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:19.363658 20223 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:19.364163 20292 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:19.364434 20223 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.366607 20223 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:19.363509 15493 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:19.377205 15493 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:19.381150 15599 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:20:19.382369 15420 catalog_manager.cc:5697] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:20:19.306458 20257 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:19.306314 20266 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:19.393637 15493 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:19.395640 15493 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:19.401567 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:19.401774 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:19.402346 15524 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:20:19.404099 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:19.404785 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:19.406136 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:19.404774 15524 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
W20260709 13:20:19.375661 20303 log.cc:927] Time spent T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.159s	user 0.001s	sys 0.001s
I20260709 13:20:19.357292 20252 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:19.408946 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:19.415478 15493 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:19.417608 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:19.447758 15525 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:20:19.385360 19888 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:20:19.469158 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 53772.4us
I20260709 13:20:19.532968 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:19.535077 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:19.386265 19724 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:19.536273 15493 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:19.541872 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:19.537801 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:19.544572 15598 raft_consensus.cc:2393] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 8: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:19.545995 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:19.547845 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:19.548111 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:19.548599 15597 raft_consensus.cc:2368] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12.
I20260709 13:20:19.418543 20291 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:19.419148 20314 log.cc:927] Time spent T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 13:20:19.409139 20266 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:19.408528 20223 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:19.577193 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:19.459640 20257 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:19.577862 15598 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:19.530808 20149 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:19.584379 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:19.592561 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:19.598938 15523 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:19.530941 20252 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.603909 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:19.579759 19888 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:19.611572 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:19.609055 20257 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:19.592042 20266 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
W20260709 13:20:19.609642 20316 log.cc:927] Time spent T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.333s	user 0.000s	sys 0.001s
I20260709 13:20:19.600291 20073 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:19.619627 15521 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:19.605305 20286 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:19.597036 20098 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:19.624639 15566 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.002s	sys 0.011s
W20260709 13:20:19.625054 15566 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.002s	sys 0.011s
I20260709 13:20:19.622049 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:19.613826 20236 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:19.626827 15600 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.012s	user 0.007s	sys 0.007s
W20260709 13:20:19.630519 15600 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.016s	user 0.007s	sys 0.007s
I20260709 13:20:19.605990 20317 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
W20260709 13:20:19.633795 16234 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.023s	sys 0.035s
W20260709 13:20:19.634235 16234 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.023s	sys 0.035s
I20260709 13:20:19.635313 16234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53976 (request call id 1009) took 1806 ms. Trace:
I20260709 13:20:19.635758 16234 rpcz_store.cc:276] 0709 13:20:17.829299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:17.829421 (+   122us) service_pool.cc:224] Handling call
0709 13:20:17.853618 (+ 24197us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:17.853859 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:20:19.109039 (+1255180us) log.cc:844] Serialized 14 byte log entry
0709 13:20:19.634970 (+525931us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:19.635026 (+    56us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:19.635076 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.635136 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:19.635253 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8605,"prepare.queue_time_us":401691,"prepare.run_cpu_time_us":22,"prepare.run_wall_time_us":30,"raft.queue_time_us":116,"thread_start_us":1771576,"threads_started":2}
W20260709 13:20:19.640376 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 477303us
W20260709 13:20:19.641309 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:19.642364 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 488394us
I20260709 13:20:19.615370 20025 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:19.626004 20086 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:19.631733 20286 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:19.644611 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 459318us
I20260709 13:20:19.641528 20287 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:19.645632 16204 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.011s	sys 0.042s
I20260709 13:20:19.623984 20266 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.618244 20257 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.631531 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1551) took 1107 ms. Trace:
W20260709 13:20:19.643126 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 444106us
I20260709 13:20:19.630664 16239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1404) took 1145 ms. Trace:
I20260709 13:20:19.637871 20098 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:19.637595 20236 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:19.639701 20320 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:19.664460 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:19.666564 15520 rpcz_store.cc:276] 0709 13:20:18.524546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:18.524741 (+   195us) service_pool.cc:224] Handling call
0709 13:20:18.525037 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:18.525243 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:20:19.136725 (+611482us) log.cc:844] Serialized 14 byte log entry
0709 13:20:19.631178 (+494453us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:19.631230 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:19.631272 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.631372 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:19.631477 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":84,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":4203,"thread_start_us":494058,"threads_started":1}
I20260709 13:20:19.674734 15525 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:19.672253 20287 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:19.682020 20287 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.684724 20287 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:19.643474 20255 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.012s	sys 0.038s
I20260709 13:20:19.645193 15600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 2329) took 1151 ms. Trace:
I20260709 13:20:19.690006 15600 rpcz_store.cc:276] 0709 13:20:18.493464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:18.558414 (+ 64950us) service_pool.cc:224] Handling call
0709 13:20:18.569590 (+ 11176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:18.569854 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:20:18.614611 (+ 44757us) log.cc:844] Serialized 14 byte log entry
0709 13:20:19.644789 (+1030178us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:19.644849 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:19.644924 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.644996 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:19.645127 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13735,"prepare.queue_time_us":44459,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"thread_start_us":1016057,"threads_started":1}
W20260709 13:20:19.689869 20255 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.012s	sys 0.038s
W20260709 13:20:19.665356 16204 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.012s	sys 0.043s
W20260709 13:20:19.667943 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:19.692039 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 613994us
I20260709 13:20:19.668447 16239 rpcz_store.cc:276] 0709 13:20:18.484820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:18.484988 (+   168us) service_pool.cc:224] Handling call
0709 13:20:18.485321 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:18.485554 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:20:18.511525 (+ 25971us) log.cc:844] Serialized 14 byte log entry
0709 13:20:19.630282 (+1118757us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:19.630349 (+    67us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:19.630402 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.630467 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:19.630599 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9363,"prepare.queue_time_us":32307,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"raft.queue_time_us":44658,"spinlock_wait_cycles":2193280,"thread_start_us":1112267,"threads_started":2,"wal-append.queue_time_us":863904}
W20260709 13:20:19.672488 15676 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.004s	sys 0.013s
W20260709 13:20:19.693231 15676 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.005s	sys 0.013s
I20260709 13:20:19.664460 20025 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:19.664984 19656 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:19.674381 20312 log.cc:927] Time spent T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
W20260709 13:20:19.660655 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:19.664296 20086 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
W20260709 13:20:19.680811 20311 log.cc:927] Time spent T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
W20260709 13:20:19.698035 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:19.746619 15567 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:19.665544 20286 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:19.754462 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:19.755149 15668 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:19.675617 20098 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.779675 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:19.675869 20236 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.741544 19787 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:19.784181 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:19.783712 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 247528us
I20260709 13:20:19.801713 15672 raft_consensus.cc:2393] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 9: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:19.836885 15493 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:19.839471 15676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2549) took 1335 ms. Trace:
I20260709 13:20:19.696803 20320 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:19.712890 19656 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.708649 20025 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.839910 15676 rpcz_store.cc:276] 0709 13:20:18.504080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:18.504194 (+   114us) service_pool.cc:224] Handling call
0709 13:20:18.504472 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:18.504662 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:20:18.934446 (+429784us) log.cc:844] Serialized 14 byte log entry
0709 13:20:19.839125 (+904679us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:19.839180 (+    55us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:19.839225 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.839294 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:19.839414 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":655287,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":2862,"thread_start_us":1312246,"threads_started":3}
I20260709 13:20:19.774535 20317 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:19.842288 20318 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:19.851178 19895 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:19.852370 20299 log.cc:927] Time spent T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.633s	user 0.003s	sys 0.000s
I20260709 13:20:19.858270 20328 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:19.858946 20335 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:19.843112 19888 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:19.864420 20292 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:19.870478 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 177727us
I20260709 13:20:19.810298 20060 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:19.780334 20252 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:19.803295 20334 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:19.749567 20086 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:19.869618 20209 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:19.841619 19787 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:19.896417 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 149875us
W20260709 13:20:19.882534 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 44238us
W20260709 13:20:19.883761 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 184589us
I20260709 13:20:19.952863 20333 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:19.883759 20318 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.953426 20333 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.954886 16203 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:19.954982 20318 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:19.894928 20123 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 13:20:19.955789 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:19.957471 16238 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:20:19.955552 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:19.959525 15493 leader_election.cc:400] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12.
I20260709 13:20:19.872576 20347 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:19.962100 20351 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:19.962632 20351 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:19.962968 20351 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:19.964350 16234 raft_consensus.cc:1240] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:19.965094 20026 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:19.948868 20073 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:19.967844 20073 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:19.976611 20322 log.cc:927] Time spent T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 13:20:19.979856 20151 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:19.981709 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 317299us
W20260709 13:20:19.982926 15491 leader_election.cc:336] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.679s (SENT)
W20260709 13:20:19.892735 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 141060us
I20260709 13:20:19.983456 15491 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:19.961323 15493 leader_election.cc:403] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:20:19.982995 15642 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:19.990062 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:19.991307 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:19.991523 16204 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:19.992569 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:19.885540 20160 consensus_meta.cc:287] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.670s	user 0.002s	sys 0.009s
W20260709 13:20:19.998374 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 47073.3us
I20260709 13:20:19.991938 15668 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:20.001284 15673 raft_consensus.cc:2410] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 15 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
W20260709 13:20:19.951733 20271 consensus_meta.cc:287] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.863s	user 0.003s	sys 0.009s
I20260709 13:20:20.005439 20292 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12.
I20260709 13:20:20.006906 20291 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.007459 20149 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.007942 20223 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:20.010180 20341 log.cc:927] Time spent T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:20:20.011116 20202 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:20.011356 20107 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:20:20.015336 19727 consensus_meta.cc:287] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.715s	user 0.005s	sys 0.005s
I20260709 13:20:20.016651 20320 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:19.953208 20345 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:20.018029 20123 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:20.019784 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.020282 20051 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:20.060514 20343 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.030544 20339 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.106578 20343 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.106771 20339 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.062697 20291 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:19.997712 20333 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:20.109889 20098 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.068174 20257 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.111181 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.038549 19888 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.111516 20025 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.068543 20353 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.112106 19888 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.112715 20353 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.040166 20160 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12.
I20260709 13:20:20.115028 20353 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:19.951334 20334 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.116086 20086 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:20.062906 20149 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.117340 20334 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.118475 20149 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:19.990473 16205 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:20.120015 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 129654us
I20260709 13:20:20.067868 20247 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:20.121608 19656 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:20.122020 20350 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:20.122241 15526 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:20:20.122716 20350 raft_consensus.cc:697] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:20:20.042408 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 58593.5us
W20260709 13:20:20.045135 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 62207.8us
I20260709 13:20:19.898459 20236 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.047801 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:20.126909 16237 raft_consensus.cc:2393] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 12: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:20.052397 15598 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:20:19.959944 20335 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.003568 20347 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.057158 20323 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:19.960366 20060 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.064479 15641 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:20.136808 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 72438.5us
I20260709 13:20:20.138799 15641 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:20.047111 20266 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:20.066457 20319 log.cc:927] Time spent T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.314s	user 0.004s	sys 0.000s
I20260709 13:20:19.952032 20209 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.143522 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.084041 20361 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.085868 20360 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.087218 19701 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.078908 15567 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:20.130868 20255 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:20.160302 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.071979 20271 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.099366 19787 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:20.066277 20073 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:20.100055 20286 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:20.102499 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.124720 15642 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:20.129345 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.109340 20339 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:19.896421 20328 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:20.147838 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:20.152535 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:20.173365 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:20.154493 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 69280.6us
W20260709 13:20:20.157013 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 78202.5us
I20260709 13:20:20.151650 20266 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:20.149734 20343 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.161079 15526 raft_consensus.cc:2393] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 12: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:20.161522 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.162388 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.180848 15524 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:20.181172 15525 raft_consensus.cc:2376] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12. Re-sending same reply.
I20260709 13:20:20.162441 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.181936 20328 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:20.175719 20098 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:20.182834 20098 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:20.183176 20098 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:20.163992 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.069073 20223 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.164929 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.166030 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:20.166018 15493 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:20.168047 16239 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:20.123592 20350 consensus_queue.cc:237] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:20.170321 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 45681.3us
I20260709 13:20:20.182000 15527 raft_consensus.cc:1813] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Handling vote request from an unknown peer 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:20.142652 20236 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:20.081449 19727 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.191175 20223 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:20.191974 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 56912.3us
I20260709 13:20:19.884409 20317 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.153935 16203 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:20.178404 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.163646 15641 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:20.187865 15527 raft_consensus.cc:2410] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:20:20.187973 15493 leader_election.cc:400] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12.
I20260709 13:20:20.190079 20339 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:20.192121 19727 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:20.174619 15567 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:20.192770 16237 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:20:20.187817 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:20.165355 20149 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.165148 20086 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:20.196828 15493 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:20.197043 15641 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:20.154983 20247 consensus_queue.cc:1059] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:20:20.155515 20346 consensus_queue.cc:1059] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:20:20.207163 15642 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:20.168164 20327 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:20.143232 20323 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.222967 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:20.175673 20366 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:20.226024 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.204592 20328 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:20.209491 20331 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:20.160219 19701 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.224628 15642 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:20.171355 20271 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.233661 15642 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:20.234934 16240 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57258 (request call id 978) took 1072 ms. Trace:
I20260709 13:20:20.236429 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:20.236538 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:20.213027 20266 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:20:20.237181 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 226114us
I20260709 13:20:20.228972 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.220427 15641 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:20.242986 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.215708 19888 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.246290 15599 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:20:20.225131 15567 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:20.246960 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.247148 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.247575 15596 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:20:20.247758 15594 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:20:20.224143 20355 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:20.158860 20361 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.163591 20360 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.213189 20252 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:20.222227 20327 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:20.235296 16240 rpcz_store.cc:276] 0709 13:20:19.162452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:19.163693 (+  1241us) service_pool.cc:224] Handling call
0709 13:20:19.180142 (+ 16449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:19.222246 (+ 42104us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:19.222276 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:19.228515 (+  6239us) log.cc:844] Serialized 26 byte log entry
0709 13:20:19.874075 (+645560us) spinlock_profiling.cc:243] Waited 2.53 ms on lock 0x7b180006d1e0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab0d1 00007f5ce49c6a0f
0709 13:20:20.060269 (+186194us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:20.060310 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:20.234616 (+174306us) spinlock_profiling.cc:243] Waited 41.7 ms on lock 0x7b54001cfcd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:20.234667 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:20.234706 (+    39us) raft_consensus.cc:1685] finished
0709 13:20:20.234719 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:20.234874 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":599,"raft.queue_time_us":49487,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":4970,"spinlock_wait_cycles":101748352,"thread_start_us":185720,"threads_started":1,"wal-append.queue_time_us":825702}
I20260709 13:20:20.239763 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.240931 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.241588 15675 raft_consensus.cc:1240] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:20.230958 20331 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:20.216014 20236 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:20.249140 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.249289 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.232635 20366 raft_consensus.cc:697] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:20.222505 19787 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:20.240583 20287 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.251693 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.283039 15599 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:20.258075 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.241945 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:20.259666 15603 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:20.242604 20025 raft_consensus.cc:3055] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:20.230041 20160 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.231151 20328 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:20.230274 20292 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.268797 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.269405 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.322633 15525 raft_consensus.cc:2393] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 14: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:20.270262 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.271000 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.323639 15526 raft_consensus.cc:2393] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 14: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:20.273131 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.324062 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:20.287639 20025 raft_consensus.cc:740] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
W20260709 13:20:20.231593 15412 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 582343936346420b87ecf2eebd00dc25 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 5498ef3e89a941e39c25f68cf64e6b0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:20:20.273909 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 53570.5us
W20260709 13:20:20.325949 15412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 94388us
I20260709 13:20:20.327617 15641 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:20.305018 20328 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:20.330636 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:20.228083 15493 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:20.276305 15523 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:20.242947 20247 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:20.242787 20349 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:20.333039 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.331384 15673 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
I20260709 13:20:20.218165 20086 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:20.245607 20358 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.243573 20266 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:20.285030 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.285770 20287 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.347496 20287 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.250152 20348 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:20.218330 20149 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.352458 20149 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.288776 15672 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:20:20.258822 20346 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:20.353500 20346 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:20.353989 20346 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:20.311480 16205 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:20.256336 20252 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:20.358470 20252 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:20.266525 20331 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:20.316802 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.377220 15603 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 17.
I20260709 13:20:20.307597 15598 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:20:20.307219 20160 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.318286 20060 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:20.336062 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 108053us
I20260709 13:20:20.335240 20292 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.323141 20335 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.253882 20355 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.322764 20051 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:20.382174 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 70797.7us
I20260709 13:20:20.393893 15493 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:20.396517 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:20.336634 20349 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:20.336143 20236 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:20.342631 19888 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:20.376140 20266 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:20.407326 16205 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:20.381021 20209 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.411253 15677 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:20.394189 20291 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:20.421433 19895 consensus_meta.cc:287] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.536s	user 0.009s	sys 0.000s
I20260709 13:20:20.422557 20382 consensus_queue.cc:1059] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:20.424710 20383 raft_consensus.cc:3055] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:20.425146 20383 raft_consensus.cc:740] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:20.426064 20383 consensus_queue.cc:260] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.427510 20383 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:20.404332 20354 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:20.430968 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.438566 15520 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:20:20.403458 20352 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:20.434706 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.478721 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.479782 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.435729 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.433755 16233 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:20.403934 20349 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:20.466058 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.404078 20149 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:20.477111 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.480656 15527 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:20:20.483181 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.490988 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.485236 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.486789 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.493026 15526 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:20:20.487543 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.485630 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.490084 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.484618 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.487761 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.495707 15520 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:20:20.496538 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.497617 15677 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:20:20.498277 15521 raft_consensus.cc:2393] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 10: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:20.403713 20051 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.438592 20333 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:20.438333 20317 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.498914 19888 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.488183 19724 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:20.394922 20366 consensus_queue.cc:237] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:20.508944 20354 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.511153 20323 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.494133 20160 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.464356 20348 raft_consensus.cc:697] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:20.406232 20168 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.086s
I20260709 13:20:20.545624 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.548372 19701 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.481040 20352 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:20.377012 20358 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.571290 15597 raft_consensus.cc:1240] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:20.560856 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:20.563570 20151 consensus_meta.cc:287] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.583s	user 0.004s	sys 0.005s
I20260709 13:20:20.550748 19888 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.565074 20098 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:20.557210 19724 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.566953 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.578096 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.578393 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.579347 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.579555 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.580353 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.581413 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:20.582083 15520 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 4.
I20260709 13:20:20.582744 20358 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.583307 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:20.584407 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:20.567620 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.499383 20360 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.502274 20271 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.568948 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:20.552426 20354 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:20.593248 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.594451 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.595604 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.568908 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.598291 15596 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:20:20.569010 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.409003 20025 consensus_queue.cc:260] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.569238 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.518676 20333 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:20.577708 15675 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 9: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:20.578162 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.550076 20365 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:20.588989 15668 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:20:20.546157 20327 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:20.632845 20025 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:20.553009 20086 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:20.635602 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:20.636286 15520 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:20:20.590481 15568 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:20.637221 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 46815.1us
I20260709 13:20:20.553329 20292 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.569693 20361 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.606689 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.629318 15599 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
I20260709 13:20:20.578033 20371 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:20.586551 20123 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:20.602744 15522 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:20.637310 20323 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:20.639686 20323 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:20.638062 20333 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:20.640120 20323 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:20.600860 16232 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:20:20.642933 20051 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:20.644654 20098 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:20.650738 15493 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:20.652215 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:20.653414 15493 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:20.654631 20346 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:20.656946 20346 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:20.658320 20358 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:20.658957 20358 raft_consensus.cc:697] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:20.654819 20333 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.659870 20358 consensus_queue.cc:237] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.661250 20347 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.662948 19787 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:20.663430 19787 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.664007 19787 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:20.661197 20333 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:20.638005 20381 consensus_queue.cc:1175] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" responder_term: 13 status { last_received { term: 13 index: 12 } last_committed_idx: 12 last_received_current_leader { term: 13 index: 12 } }
I20260709 13:20:20.668715 15493 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:20.619024 15674 raft_consensus.cc:2368] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 6. Current term is 7.
I20260709 13:20:20.603549 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.600415 15642 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:20.675598 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:20.676282 15672 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:20:20.668239 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:20.679188 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 271921us
I20260709 13:20:20.679955 16205 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:20.680086 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:20.586252 20223 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.683609 16239 raft_consensus.cc:1240] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:20.683843 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.684762 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.686677 15641 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:20:20.619674 20372 log.cc:927] Time spent T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:20:20.690249 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.691120 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.663335 20382 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:20.687897 15641 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:20.576372 20352 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:20.688804 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:20.692216 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 91886.4us
I20260709 13:20:20.695243 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.703924 15642 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:20.695255 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:20.704730 15597 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:20.698457 15641 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:20.705138 15642 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:20.619079 20380 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:20.700834 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.699740 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.706764 15642 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:20.708803 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.682502 20123 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:20.726441 15642 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:20.722715 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:20.729655 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.730897 15672 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18.
I20260709 13:20:20.690176 20168 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:20:20.732403 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:20.728174 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:20.724586 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:20:20.724973 15493 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:20.735587 15493 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:20.726228 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:20.737105 15603 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:20.724224 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.614601 20271 consensus_queue.cc:1059] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.098s
I20260709 13:20:20.694777 20327 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:20.696424 20382 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:20.743274 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:20.744571 15493 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:20.746006 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:20.700227 20347 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:20.760056 15598 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:20.695659 20151 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:20.703847 20346 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:20.700209 20067 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:20.709728 20098 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:20.639272 20371 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:20.641638 20348 consensus_queue.cc:237] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:20.687163 20365 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.770668 20346 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.731904 20086 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:20.764004 20382 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:20.739061 20335 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:20.741892 20168 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.757932 20360 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:20.723385 20236 raft_consensus.cc:2804] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:20:20.694998 20223 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.724349 20025 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:20.780771 15598 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:20:20.811460 15642 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:20.781140 15519 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:20:20.812703 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.781601 15493 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:20.772557 20351 consensus_meta.cc:287] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.809s	user 0.011s	sys 0.001s
I20260709 13:20:20.789168 20151 raft_consensus.cc:697] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:20.781510 20123 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:20.770958 20386 raft_consensus.cc:2764] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:20:20.740564 20355 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.770741 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.826515 16238 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
I20260709 13:20:20.759951 20327 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.722455 20361 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.780779 20266 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.777416 20247 raft_consensus.cc:3055] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:20.794929 20346 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:20.813704 16231 raft_consensus.cc:2393] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 12: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:20.728304 20380 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:20.792176 20365 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.822419 20209 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:20.797428 20168 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:20.799643 20236 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:20.829702 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.829818 15603 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18.
I20260709 13:20:20.777510 20067 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.795109 20086 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:20.776297 20349 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.822799 20292 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:20.862375 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.830207 20327 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:20.832448 19888 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:20.832703 20247 raft_consensus.cc:740] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:20.859496 20236 raft_consensus.cc:3060] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:20.875895 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.902536 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.871963 20086 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:20.870085 20209 raft_consensus.cc:697] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:20.823882 20351 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.850841 20361 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:20.961423 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:21.011451 15491 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:21.014890 15491 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:20.963349 20348 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:21.031558 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:21.032472 15491 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:21.023363 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:21.045009 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 135423us
I20260709 13:20:21.045282 16238 raft_consensus.cc:2368] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 17. Current term is 18.
I20260709 13:20:21.046908 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:21.049299 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:21.009778 20342 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:21.022539 20383 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:21.055501 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:21.058677 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:21.059863 15668 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:20:21.060663 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:21.050101 15491 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:21.032009 20123 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:20.964244 20331 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.067765 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:20.962769 20151 consensus_queue.cc:237] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.081871 16231 raft_consensus.cc:1240] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:20:21.104218 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:21.079766 20388 log.cc:927] Time spent T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
I20260709 13:20:21.135893 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:21.137233 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 87205.1us
I20260709 13:20:21.044596 20348 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:21.114512 20395 log.cc:927] Time spent T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:20:21.034366 20247 consensus_queue.cc:260] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.080935 20209 consensus_queue.cc:237] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:21.118757 20389 log.cc:927] Time spent T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.302s	user 0.000s	sys 0.002s
W20260709 13:20:21.134895 20390 log.cc:927] Time spent T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
I20260709 13:20:21.119547 20361 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:21.119886 20351 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:21.120010 20252 consensus_meta.cc:287] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.761s	user 0.011s	sys 0.004s
I20260709 13:20:21.120783 20323 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.122154 20380 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:21.123385 20349 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:21.082784 20371 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:21.141507 20360 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:21.182291 20360 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:21.184092 16232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57258 (request call id 976) took 2044 ms (client timeout 1587 ms). Trace:
W20260709 13:20:21.184547 16232 rpcz_store.cc:269] 0709 13:20:19.139453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:19.139605 (+   152us) service_pool.cc:224] Handling call
0709 13:20:21.183900 (+2044295us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b5400532854. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:21.184022 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3270889216}
I20260709 13:20:21.066764 20342 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.185811 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:20:21.186084 20393 log.cc:927] Time spent T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.134s	user 0.003s	sys 0.000s
I20260709 13:20:21.187664 16232 raft_consensus.cc:2376] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11. Re-sending same reply.
I20260709 13:20:21.053896 20160 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:20:21.051887 20359 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:21.143597 16240 raft_consensus.cc:2368] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12.
W20260709 13:20:21.145660 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:21.148357 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:21.125874 20354 consensus_meta.cc:287] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.508s	user 0.007s	sys 0.000s
I20260709 13:20:21.022297 20292 consensus_queue.cc:1059] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
W20260709 13:20:21.162753 16233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57258 (request call id 975) took 2028 ms (client timeout 2196 ms). Trace:
W20260709 13:20:21.126456 20399 log.cc:927] Time spent T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7: Append to log took a long time: real 0.076s	user 0.002s	sys 0.000s
I20260709 13:20:21.131829 20266 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:21.141978 19888 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:21.172039 15669 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:20:21.134855 20223 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:21.166288 20345 raft_consensus.cc:3055] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:21.178315 20252 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:21.180546 20323 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:21.067031 20328 consensus_meta.cc:287] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.724s	user 0.009s	sys 0.001s
I20260709 13:20:21.198127 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:21.195358 16233 rpcz_store.cc:269] 0709 13:20:19.134266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:19.134441 (+   175us) service_pool.cc:224] Handling call
0709 13:20:21.162582 (+2028141us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54007b0f54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:21.162685 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2951746432}
I20260709 13:20:21.129014 20168 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.130762 20352 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.227224 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:21.227372 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 164452us
I20260709 13:20:21.184384 20360 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:21.191133 20247 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:21.239630 16204 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:21.242702 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:21.251317 15668 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:21.190582 20342 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:21.216516 19888 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:21.259163 16238 raft_consensus.cc:1240] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:21.251945 16204 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:21.215116 20345 raft_consensus.cc:740] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:21.266052 16204 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:21.211228 20354 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.255695 20384 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:21.258872 19888 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:21.239248 20328 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:21.286082 15599 raft_consensus.cc:1240] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:21.287263 16204 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:21.254765 20410 consensus_queue.cc:1059] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.045s
W20260709 13:20:21.293941 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 116508us
I20260709 13:20:21.274946 15668 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:21.278348 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:21.296094 15673 raft_consensus.cc:2393] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 9: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:21.297091 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:21.297619 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:21.298915 15668 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:21.266408 20382 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:21.295698 20339 consensus_meta.cc:287] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 1.084s	user 0.014s	sys 0.001s
I20260709 13:20:21.283994 20333 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:21.344168 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:21.344910 15668 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:21.339864 19787 consensus_meta.cc:287] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.675s	user 0.005s	sys 0.007s
I20260709 13:20:21.356096 20024 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:21.339478 20331 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:21.339643 20348 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:21.343261 20339 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:21.346726 20151 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:21.340688 20346 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.295248 20345 consensus_queue.cc:260] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:21.366811 20409 log.cc:927] Time spent T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:20:21.365554 20086 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:21.374416 20328 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:21.375469 20382 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:21.378113 20252 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:21.379036 20354 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:21.370383 20339 raft_consensus.cc:3055] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:21.395220 20346 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:21.396198 20333 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:21.390599 20339 raft_consensus.cc:740] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:21.386548 20345 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:21.432498 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 136462us
I20260709 13:20:21.353513 15671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 1437) took 1206 ms. Trace:
I20260709 13:20:21.355458 15668 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:21.448029 15671 rpcz_store.cc:276] 0709 13:20:20.147112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.147906 (+   794us) service_pool.cc:224] Handling call
0709 13:20:20.148155 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:20.195477 (+ 47322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:20.195509 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:20.195553 (+    44us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:20.195585 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:21.353263 (+1157678us) spinlock_profiling.cc:243] Waited 47.1 ms on lock 0x7b540042b654. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:21.353309 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:21.353452 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108373760}
I20260709 13:20:21.441249 15493 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:21.456152 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:21.357445 19787 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.491379 20086 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:21.371013 20168 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:21.495205 15522 raft_consensus.cc:1240] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:21.497605 19787 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:21.499200 19787 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:21.500125 20168 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:21.500231 19787 raft_consensus.cc:697] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:21.503266 19787 consensus_queue.cc:237] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.400732 20339 consensus_queue.cc:260] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.535827 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 2143) took 1375 ms. Trace:
I20260709 13:20:21.536311 15523 rpcz_store.cc:276] 0709 13:20:20.160548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.160665 (+   117us) service_pool.cc:224] Handling call
0709 13:20:21.535368 (+1374703us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b5400187c54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:21.535598 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":235218688}
W20260709 13:20:21.527117 20327 consensus_meta.cc:287] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.647s	user 0.010s	sys 0.002s
I20260709 13:20:21.541599 20327 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.542349 20419 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:21.543220 20419 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:21.548733 20236 consensus_meta.cc:287] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.562s	user 0.010s	sys 0.010s
I20260709 13:20:21.549326 20236 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:21.557349 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 850446us
I20260709 13:20:21.558791 15672 raft_consensus.cc:1240] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:20:21.561600 20236 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:21.563858 15674 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 2160) took 1328 ms (client timeout 1679 ms). Trace:
W20260709 13:20:21.563288 15493 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:21.558843 20419 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:21.568090 16237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57258 (request call id 985) took 1420 ms. Trace:
W20260709 13:20:21.564446 15674 rpcz_store.cc:269] 0709 13:20:20.234880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.236186 (+  1306us) service_pool.cc:224] Handling call
0709 13:20:21.562544 (+1326358us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b540009c9d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:21.563026 (+   482us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":850818816}
I20260709 13:20:21.571590 15641 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:21.572685 20327 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:21.568112 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 253753us
I20260709 13:20:21.575716 15669 raft_consensus.cc:1240] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:21.571236 19888 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:21.579156 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:21.582620 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:21.588811 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 271721us
I20260709 13:20:21.584857 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:21.585397 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:21.585552 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:21.594650 15599 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:21.594926 15597 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:20:21.586459 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:21.584030 16237 rpcz_store.cc:276] 0709 13:20:20.145471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.155975 (+ 10504us) service_pool.cc:224] Handling call
0709 13:20:20.156344 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:20.206473 (+ 50129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:21.564086 (+1357613us) spinlock_profiling.cc:243] Waited 36.2 ms on lock 0x7b5400514354. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:21.565748 (+  1662us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":83230464}
I20260709 13:20:21.581867 19724 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:21.599917 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:21.593700 20403 log.cc:927] Time spent T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.464s	user 0.003s	sys 0.000s
I20260709 13:20:21.602602 20359 raft_consensus.cc:3055] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:21.603147 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:21.603982 15597 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:21.605325 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:21.611518 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:21.603106 20359 raft_consensus.cc:740] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:21.615309 20359 consensus_queue.cc:260] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:21.618356 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 47034.5us
I20260709 13:20:21.619540 15641 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:20:21.620782 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:21.620807 20320 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:21.622550 20320 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:21.623093 20320 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:21.625204 20327 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:21.626284 16205 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:21.628476 20414 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:21.628999 20414 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:21.629407 20414 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:21.631544 16205 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:21.625828 20327 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:21.633111 20327 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:21.637164 19888 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:21.641405 20352 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:21.646216 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:20:21.621950 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 1.42901e+06us
W20260709 13:20:21.648382 20421 log.cc:927] Time spent T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.132s	user 0.002s	sys 0.000s
I20260709 13:20:21.648629 15566 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:21.650619 20420 log.cc:927] Time spent T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.249s	user 0.000s	sys 0.002s
I20260709 13:20:21.651260 20252 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:21.651751 20384 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:20:21.653023 20416 log.cc:927] Time spent T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.264s	user 0.000s	sys 0.000s
I20260709 13:20:21.654516 20252 consensus_queue.cc:1059] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:20:21.655669 15677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57710 (request call id 1439) took 1447 ms (client timeout 1907 ms). Trace:
I20260709 13:20:21.656088 20323 raft_consensus.cc:3055] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:21.646347 15491 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:21.656584 20323 raft_consensus.cc:740] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:21.658131 19888 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:21.657737 20323 consensus_queue.cc:260] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.658295 15491 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:21.658790 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 74565.1us
I20260709 13:20:21.659500 20410 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:21.659523 20323 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:20:21.662415 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:21.662729 15670 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:20:21.663882 20410 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:21.664460 15670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49094 (request call id 2165) took 1408 ms. Trace:
I20260709 13:20:21.665134 15673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 2167) took 1269 ms. Trace:
I20260709 13:20:21.665557 15673 rpcz_store.cc:276] 0709 13:20:20.396120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.396265 (+   145us) service_pool.cc:224] Handling call
0709 13:20:21.664713 (+1268448us) spinlock_profiling.cc:243] Waited 899 ms on lock 0x7b540023ad54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:21.664977 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2066593664}
I20260709 13:20:21.666388 19888 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.667613 15491 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:21.664889 15670 rpcz_store.cc:276] 0709 13:20:20.255666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.255808 (+   142us) service_pool.cc:224] Handling call
0709 13:20:21.662136 (+1406328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:21.663221 (+  1085us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:21.664227 (+  1006us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b540026b6d0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:21.664377 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3233335040}
I20260709 13:20:21.668903 19888 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:21.669484 20410 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:21.669981 20410 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:21.670356 20410 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:21.656136 15677 rpcz_store.cc:269] 0709 13:20:20.208342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.208549 (+   207us) service_pool.cc:224] Handling call
0709 13:20:21.655245 (+1446696us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b540026b6d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:21.655441 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":431021440}
I20260709 13:20:21.679518 20339 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:21.685563 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 53004.3us
I20260709 13:20:21.684988 15599 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:21.688520 15567 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:21.693378 20406 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:21.695312 20067 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:21.684805 15642 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:21.702442 15528 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:20:21.703919 15603 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:20:21.704118 15597 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:20:21.711432 20205 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:21.712190 20205 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.682811 20414 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.712754 15567 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:21.718252 20406 raft_consensus.cc:697] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:21.718878 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:21.719513 20406 consensus_queue.cc:237] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.720615 15524 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:21.721513 20327 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.722223 20247 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:21.722855 16240 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 1016) took 1040 ms. Trace:
I20260709 13:20:21.723296 16240 rpcz_store.cc:276] 0709 13:20:20.682817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.684550 (+  1733us) service_pool.cc:224] Handling call
0709 13:20:21.722416 (+1037866us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b5400241b54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:21.722629 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1053455744}
I20260709 13:20:21.724071 20327 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:21.724339 20236 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:21.722869 20247 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.726876 20205 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:21.729333 20317 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:21.729849 20317 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:21.730078 20247 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:21.733458 20414 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:21.734089 20317 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:21.740759 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:21.724095 15642 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:21.742094 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:21.746670 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:21.746532 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:21.750665 16236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1424) took 1058 ms. Trace:
I20260709 13:20:21.725107 20345 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:21.761447 16239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 1017) took 1072 ms. Trace:
I20260709 13:20:21.758390 20252 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:21.761966 16239 rpcz_store.cc:276] 0709 13:20:20.688450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.688564 (+   114us) service_pool.cc:224] Handling call
0709 13:20:21.761055 (+1072491us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b54005370d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:21.761238 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1177702144}
I20260709 13:20:21.753816 16236 rpcz_store.cc:276] 0709 13:20:20.692134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.732078 (+ 39944us) service_pool.cc:224] Handling call
0709 13:20:20.732398 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:20.879714 (+147316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:20.879744 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:20.879788 (+    44us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:20.879824 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:21.750216 (+870392us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54003c9454. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:21.750286 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:21.750478 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":338334464}
I20260709 13:20:21.766135 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:21.768575 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:21.768873 20359 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:21.769302 15528 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:21.764531 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:21.765892 20320 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.776295 20320 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:21.777074 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:21.778445 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:21.780310 16238 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:20:21.781936 20419 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:21.782593 20419 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:21.784747 20419 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:21.786520 15528 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:20:21.786626 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:21.786934 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:21.788501 15526 raft_consensus.cc:1240] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:20:21.790521 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:21.791349 15677 raft_consensus.cc:1240] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:21.792724 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:21.792833 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:21.794854 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:21.795482 15519 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:21.797528 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 89739.6us
I20260709 13:20:21.798492 15567 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:21.799253 20323 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:21.799923 15567 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:21.800743 20323 raft_consensus.cc:2764] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:20:21.791379 15675 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 10: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:21.809706 15523 raft_consensus.cc:2410] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 13:20:21.810366 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 86442.7us
I20260709 13:20:21.811268 15642 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:21.795435 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:21.812294 20320 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:21.813024 20320 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:21.813457 20320 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:21.814934 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:21.816362 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:21.816649 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:21.817257 16236 raft_consensus.cc:2368] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12.
I20260709 13:20:21.765002 20236 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:21.818288 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:21.818608 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:21.819132 16236 raft_consensus.cc:2368] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12.
I20260709 13:20:21.820668 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:21.821390 15669 raft_consensus.cc:1240] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:21.822198 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:21.822005 20320 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.800163 16203 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:21.824641 20320 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:21.825052 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:21.826815 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:21.828722 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:21.829588 15526 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:20:21.829677 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:21.830665 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:21.830922 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:21.831595 15528 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:21.813594 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:21.834789 20320 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:21.837086 15642 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:21.837984 20320 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:21.838891 20060 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:21.839594 20060 raft_consensus.cc:697] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:21.840358 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:21.840600 20060 consensus_queue.cc:237] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:21.844471 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 45375.9us
I20260709 13:20:21.845329 16203 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:21.845999 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:21.846237 20414 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:21.846788 20414 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:21.847190 20414 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:21.850296 15670 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
W20260709 13:20:21.854506 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 89132.7us
I20260709 13:20:21.801520 15567 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:21.850389 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:21.867766 15519 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
W20260709 13:20:21.868734 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:21.880017 15522 raft_consensus.cc:1240] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 9. Current term is 10. Ops: []
W20260709 13:20:21.854038 15566 connection.cc:256] RPC call timeout handler was delayed by -2.5352165300000706s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:20:21.804795 20317 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:21.874539 16203 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:21.875020 20323 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:21.883330 15523 raft_consensus.cc:1240] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:21.884837 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:21.885763 15491 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:21.890491 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:21.891373 15527 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:20:21.894789 16203 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:21.896118 20400 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:21.896675 20400 raft_consensus.cc:697] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:21.898077 15598 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:21.899410 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:21.899747 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:21.900557 20397 consensus_queue.cc:1059] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:21.901456 20331 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:21.902081 20331 raft_consensus.cc:697] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:21.902702 15602 raft_consensus.cc:1240] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:21.903437 20414 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.900066 20026 consensus_queue.cc:1059] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:20:21.904022 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:21.904687 15594 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:21.904911 20026 raft_consensus.cc:3055] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:21.905323 20026 raft_consensus.cc:740] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:21.901995 20397 raft_consensus.cc:3055] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:20:21.906574 20397 raft_consensus.cc:740] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:21.907418 15598 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:20:21.908936 20397 consensus_queue.cc:260] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.902706 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:21.897663 20400 consensus_queue.cc:237] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.873880 19895 consensus_queue.cc:1059] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:20:21.920302 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:21.920338 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 83424.1us
I20260709 13:20:21.905740 20414 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:21.932839 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:21.936098 15597 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:21.942932 20397 raft_consensus.cc:3060] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:21.946528 20317 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:21.947256 20432 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:21.947919 20432 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.949235 20354 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:21.949862 20354 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:21.950855 20432 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:21.951993 20354 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:21.936442 20339 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:21.934103 20423 raft_consensus.cc:3055] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:21.955973 20423 raft_consensus.cc:740] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:21.956066 15597 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:21.957126 20423 consensus_queue.cc:260] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.979547 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:21.959717 20024 consensus_queue.cc:1059] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:21.952309 15602 raft_consensus.cc:1240] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:21.970872 20410 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:21.976354 15528 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:20:21.977082 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:21.956003 16205 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:21.976173 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:20:21.994436 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 226369us
I20260709 13:20:21.995407 15491 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:21.998420 20339 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:21.980866 15594 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:20:21.964205 15603 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:20:21.981832 15669 raft_consensus.cc:1240] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:21.986429 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:21.986586 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:21.986999 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:21.987844 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:22.008921 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 207545us
I20260709 13:20:21.987071 15676 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 10: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:22.007512 15642 leader_election.cc:400] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 17. Current term is 18.
I20260709 13:20:21.903229 20331 consensus_queue.cc:237] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:22.013374 15674 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:20:22.014896 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:22.013790 15566 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:22.022104 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:22.024920 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:22.029562 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:22.030740 15567 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:22.037027 15527 raft_consensus.cc:1240] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:22.038239 15524 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:22.041193 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 62691.4us
I20260709 13:20:22.045400 15528 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:22.047683 20151 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:20:21.989506 20026 consensus_queue.cc:260] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:22.052150 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:22.039695 20423 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:22.057436 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:22.059597 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:22.060277 20354 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:22.060741 20422 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:22.039354 20432 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:20:22.070140 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 114223us
I20260709 13:20:22.074242 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:22.075671 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:22.076418 15598 raft_consensus.cc:2368] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12.
I20260709 13:20:22.045639 20060 consensus_queue.cc:1059] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:20:22.103554 15674 raft_consensus.cc:1240] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:22.110332 15601 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:22.108806 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:22.077796 20060 raft_consensus.cc:3055] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:22.071638 20354 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:22.119946 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:22.132911 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:22.134658 20321 consensus_queue.cc:1059] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.010s
W20260709 13:20:22.138484 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:22.138891 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:22.140935 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 155668us
W20260709 13:20:22.142005 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 83675.5us
I20260709 13:20:22.119098 20354 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:22.144935 15524 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:22.139590 15527 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
W20260709 13:20:22.148144 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 127804us
I20260709 13:20:22.114768 20060 raft_consensus.cc:740] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:22.124886 20026 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:22.152338 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:20:22.162727 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:22.162343 20330 consensus_queue.cc:1059] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.096s
I20260709 13:20:22.175864 20067 consensus_queue.cc:1059] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:22.177059 20067 raft_consensus.cc:3055] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:22.177423 20067 raft_consensus.cc:740] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:22.180511 15567 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:22.183113 15527 raft_consensus.cc:1275] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:20:22.185030 15566 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:22.186789 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:22.187579 15599 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 10: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
W20260709 13:20:22.190603 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 62876.6us
W20260709 13:20:22.195043 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 44497.2us
I20260709 13:20:22.190757 20410 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:22.193293 20432 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:22.193362 20317 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:22.210479 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:22.215277 16205 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:22.201972 20321 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:20:22.222916 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:22.223564 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 68809.3us
I20260709 13:20:22.224332 16205 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:22.224332 15567 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:22.203649 20350 consensus_queue.cc:1059] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:20:22.239379 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:22.234205 20024 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:22.234254 20151 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:22.235004 20432 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:22.262820 15601 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
W20260709 13:20:22.265180 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 104780us
I20260709 13:20:22.276517 15594 raft_consensus.cc:1240] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:22.275630 16204 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
W20260709 13:20:22.268405 20434 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 13:20:22.282469 20350 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:22.287523 15642 leader_election.cc:400] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12.
I20260709 13:20:22.288103 15566 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:22.272639 20151 raft_consensus.cc:697] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:22.260385 20320 consensus_queue.cc:1059] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:20:22.227187 20067 consensus_queue.cc:260] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:22.303805 20414 consensus_queue.cc:1059] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:22.304040 19787 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:20:22.314711 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 90464.4us
I20260709 13:20:22.290382 20350 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:22.271795 20339 consensus_queue.cc:1059] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:20:22.244313 20060 consensus_queue.cc:260] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:22.278647 20425 consensus_queue.cc:1059] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:20:22.324671 20425 raft_consensus.cc:3055] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:22.324914 15642 leader_election.cc:400] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12.
I20260709 13:20:22.325223 15642 leader_election.cc:403] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:20:22.312072 20323 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:22.312675 20432 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:20:22.337114 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 64122.2us
I20260709 13:20:22.320784 20350 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:22.341908 15675 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:20:22.345952 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 119784us
I20260709 13:20:22.346293 20320 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12.
I20260709 13:20:22.347078 20320 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:22.347608 20320 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
W20260709 13:20:22.348255 20442 log.cc:927] Time spent T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:20:22.354244 19724 raft_consensus.cc:3055] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:22.354693 19724 raft_consensus.cc:740] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:22.355651 19724 consensus_queue.cc:260] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:22.357091 19724 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:22.326241 20425 raft_consensus.cc:740] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:22.382887 20425 consensus_queue.cc:260] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:22.384390 20425 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:22.389724 15596 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:22.390259 20236 raft_consensus.cc:3055] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:22.390705 20236 raft_consensus.cc:740] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:22.391549 20236 consensus_queue.cc:260] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:22.392733 20236 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:22.349705 20320 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:22.341001 20067 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:22.407836 20333 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
W20260709 13:20:22.438911 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 134251us
I20260709 13:20:22.410667 15525 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:20:22.444936 20151 consensus_queue.cc:237] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:22.447377 15670 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:20:22.448915 20205 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:22.489264 15676 raft_consensus.cc:1240] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 8. Current term is 9. Ops: []
W20260709 13:20:22.499519 20435 log.cc:927] Time spent T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.465s	user 0.004s	sys 0.000s
I20260709 13:20:22.504383 20354 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:22.479523 20339 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:20:22.519533 20060 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:22.521658 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:22.478780 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:22.535709 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 204686us
W20260709 13:20:22.538745 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 162885us
W20260709 13:20:22.542454 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 217673us
W20260709 13:20:22.548666 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 208084us
I20260709 13:20:22.558127 15526 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:22.563745 15601 raft_consensus.cc:1240] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:22.568217 15567 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:22.548705 20414 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:22.570792 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:22.577343 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:22.577978 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:22.575469 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 96786.2us
I20260709 13:20:22.580871 15596 raft_consensus.cc:1275] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:20:22.581085 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:22.567534 20339 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:22.590627 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:22.591116 15601 raft_consensus.cc:1275] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:20:22.593655 15493 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:22.564616 20350 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:22.597834 20350 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:22.598572 16237 raft_consensus.cc:2368] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13.
I20260709 13:20:22.600521 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:22.601145 15526 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:22.604036 20359 raft_consensus.cc:3055] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:22.604472 20359 raft_consensus.cc:740] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:22.608670 15526 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:20:22.609274 15524 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:20:22.609665 15524 pending_rounds.cc:85] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 9
I20260709 13:20:22.609879 15524 pending_rounds.cc:107] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:20:22.610168 15524 raft_consensus.cc:2889] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:22.611814 16204 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:22.615368 20354 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:22.617256 20423 consensus_meta.cc:287] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.538s	user 0.014s	sys 0.003s
I20260709 13:20:22.618415 20410 raft_consensus.cc:2764] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
W20260709 13:20:22.620399 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 280374us
I20260709 13:20:22.593592 15595 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:20:22.622659 16236 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
I20260709 13:20:22.562278 20446 consensus_queue.cc:1059] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 13:20:22.632243 20051 raft_consensus.cc:3055] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:22.632704 20443 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:20:22.640852 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:22.588231 19724 consensus_queue.cc:1059] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
W20260709 13:20:22.636236 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 55931.2us
I20260709 13:20:22.650148 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:22.623625 20414 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:22.637034 20051 raft_consensus.cc:740] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:22.657202 15642 leader_election.cc:400] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12.
I20260709 13:20:22.639451 20443 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:22.661058 16205 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:22.659391 20339 raft_consensus.cc:697] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:22.665673 20422 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:22.654361 20423 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:22.676939 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:22.677016 20354 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:22.679783 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:22.654367 20026 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:20:22.682243 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:22.682829 20051 consensus_queue.cc:260] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:22.684598 15567 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:22.692375 15493 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:22.694095 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:22.669814 20446 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:22.670060 20443 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:22.701081 15567 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:22.702108 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:22.702464 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:22.705644 15526 raft_consensus.cc:1813] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Handling vote request from an unknown peer 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:22.702960 15525 raft_consensus.cc:1240] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:22.698148 15641 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:22.706200 15526 raft_consensus.cc:2393] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 8: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:20:22.702764 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 79710.6us
I20260709 13:20:22.705790 15524 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:20:22.710512 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:20:22.713341 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 56289.4us
I20260709 13:20:22.714351 15642 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:22.714911 15674 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:22.700192 20354 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:22.716748 20354 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:22.717120 15567 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:22.717542 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:22.718346 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:22.720031 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:22.722072 15527 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 10: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:22.737728 15567 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:22.738489 20317 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:22.738955 15567 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:22.739923 15641 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:22.740412 20451 consensus_queue.cc:1059] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:22.741503 20354 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:22.741882 20354 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:22.742287 20354 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:22.743026 15596 raft_consensus.cc:1240] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:22.703542 19724 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:22.748504 15522 raft_consensus.cc:1275] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:20:22.706614 20460 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:22.711937 20446 raft_consensus.cc:697] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:22.792249 15567 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:22.794094 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 46718.7us
I20260709 13:20:22.793287 15597 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:22.795010 15642 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:22.739071 20317 raft_consensus.cc:697] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:22.698148 20422 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:22.751348 19724 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:22.805992 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 66147.1us
I20260709 13:20:22.811069 15641 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:22.751181 20236 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:22.804906 15642 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:22.767972 20423 raft_consensus.cc:697] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:22.745031 20151 consensus_queue.cc:1059] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 13:20:22.793273 20447 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:20:22.814905 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 73098.3us
I20260709 13:20:22.747478 20449 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 13:20:22.808634 20320 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:22.825407 15566 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:22.810313 19724 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:22.846086 20452 consensus_queue.cc:1059] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:20:22.847306 20452 raft_consensus.cc:3055] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:22.847708 20452 raft_consensus.cc:740] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:22.848495 20452 consensus_queue.cc:260] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:22.850482 20342 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:22.855403 20422 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:22.857991 20448 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:22.858560 20448 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:22.858965 20448 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:22.860999 20466 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:22.830195 20320 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:22.862138 20320 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:22.866693 15674 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:20:22.832376 20236 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:22.805595 20460 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:22.883674 20465 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:22.873214 20452 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:22.863518 20447 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:22.886032 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:22.894143 20423 consensus_queue.cc:237] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:22.904596 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 93601.8us
I20260709 13:20:22.897533 16235 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:20:22.900790 15528 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:20:22.823769 20339 consensus_queue.cc:237] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:22.906934 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 102128us
I20260709 13:20:22.907891 20236 consensus_queue.cc:1059] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:22.908968 20471 raft_consensus.cc:3055] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:22.909363 20471 raft_consensus.cc:740] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:22.910509 20471 consensus_queue.cc:260] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:22.911772 20471 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:22.816974 20446 consensus_queue.cc:237] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:22.921255 20454 log.cc:927] Time spent T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
I20260709 13:20:22.928403 20471 consensus_queue.cc:1048] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:22.929881 20471 consensus_queue.cc:1048] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:20:22.931339 20460 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:22.932197 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 106887us
I20260709 13:20:22.932984 15566 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:22.898934 20051 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:22.847216 20359 consensus_queue.cc:260] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:22.942008 16239 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:20:22.877213 20354 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:22.955044 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:22.961025 15595 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:20:22.948339 20447 raft_consensus.cc:697] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:22.950414 20452 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:22.951644 20359 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:22.967742 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:22.968694 16235 raft_consensus.cc:1240] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:22.951262 20317 consensus_queue.cc:237] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:22.944394 20400 consensus_queue.cc:1059] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:20:22.975615 15595 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:22.984277 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:22.987032 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:22.986841 20447 consensus_queue.cc:237] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:22.987382 20400 raft_consensus.cc:3055] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:22.989212 15641 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:22.990536 15600 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:20:22.986754 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:22.995443 15671 raft_consensus.cc:1240] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:23.001061 15566 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:22.976531 20465 consensus_queue.cc:1059] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:20:23.002351 20464 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:20:23.004635 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:23.005416 15668 raft_consensus.cc:2368] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10.
I20260709 13:20:23.005728 16205 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:23.006021 20457 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.252s	user 0.000s	sys 0.001s
I20260709 13:20:23.009018 20024 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:23.009491 20024 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:23.009848 20024 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:23.015147 20465 raft_consensus.cc:3055] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:23.015683 20465 raft_consensus.cc:740] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:23.016501 20465 consensus_queue.cc:260] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:23.024947 15566 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:23.025599 20469 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:23.025947 20467 raft_consensus.cc:2764] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 13:20:23.009204 20443 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:23.027278 15603 raft_consensus.cc:1240] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:23.027184 20467 consensus_queue.cc:1059] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:20:23.028038 20252 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:23.028234 20467 raft_consensus.cc:3055] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:23.028389 20443 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:23.028709 20467 raft_consensus.cc:740] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:22.991081 20400 raft_consensus.cc:740] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:23.030642 20400 consensus_queue.cc:260] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:23.031787 20400 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:23.035750 20448 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:23.036494 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:20:23.041199 20060 consensus_meta.cc:287] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.521s	user 0.005s	sys 0.005s
I20260709 13:20:23.048341 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:23.054497 19787 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:23.055576 20205 raft_consensus.cc:3055] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:23.056017 20205 raft_consensus.cc:740] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:23.056908 20205 consensus_queue.cc:260] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:23.029513 20467 consensus_queue.cc:260] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:23.060477 15567 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:23.017721 20465 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:23.078035 15669 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:20:23.099028 15672 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:20:23.099975 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:23.091535 15521 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:20:23.109841 15521 pending_rounds.cc:85] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 10
I20260709 13:20:23.110128 15521 pending_rounds.cc:107] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:20:23.072103 16238 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:20:23.110548 15521 raft_consensus.cc:2889] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:20:23.109189 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 48805.6us
W20260709 13:20:23.111250 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:23.061858 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:23.117810 15524 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:23.090485 16203 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:23.136052 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:23.137846 15675 raft_consensus.cc:2393] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 9: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:23.070858 20448 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:23.090396 20205 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:23.090154 20342 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:23.059705 20333 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:23.101529 20414 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:23.102818 20354 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:23.148401 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 57997.2us
I20260709 13:20:23.149849 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:23.135200 20466 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:23.163278 15675 raft_consensus.cc:2393] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 11: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:23.166702 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:23.097055 19724 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:23.169534 15669 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:23.151908 20446 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.169853 15675 raft_consensus.cc:2393] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 13: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:23.171193 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:23.155061 20060 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:23.168227 15493 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:23.175179 15672 raft_consensus.cc:1240] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 13. Current term is 14. Ops: []
W20260709 13:20:23.173691 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 187467us
I20260709 13:20:23.137892 20320 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:23.175974 15641 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:23.188205 16237 raft_consensus.cc:1240] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:23.189414 15597 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:20:23.190201 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:23.198964 15524 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:23.158463 20425 consensus_queue.cc:1059] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:20:23.173421 20467 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:23.205155 15668 raft_consensus.cc:2393] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 9: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:23.164465 20333 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:23.212385 20453 log.cc:927] Time spent T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.513s	user 0.000s	sys 0.002s
I20260709 13:20:23.212937 20333 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:23.213967 20323 raft_consensus.cc:3055] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:23.214403 20323 raft_consensus.cc:740] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:23.214763 19724 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:23.215314 20323 consensus_queue.cc:260] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:23.218614 15671 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:23.221442 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:23.222276 15676 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:20:23.171950 20051 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
W20260709 13:20:23.243305 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 55318.2us
W20260709 13:20:23.244001 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 46472.1us
W20260709 13:20:23.245770 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 69885us
I20260709 13:20:23.246501 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:23.204943 20359 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:23.227105 20447 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:23.179158 20446 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:23.259604 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 91429.7us
W20260709 13:20:23.266142 15491 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.706s (SENT) [suppressed 8 similar messages]
I20260709 13:20:23.266573 20067 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:23.259009 16239 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:20:23.270716 16239 raft_consensus.cc:606] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:23.271098 16239 raft_consensus.cc:614] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:23.271409 15641 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:23.278959 15671 raft_consensus.cc:1217] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 8.8->[9.9-11.10]   Dedup: 9.9->[11.10-11.10]
I20260709 13:20:23.279573 15671 raft_consensus.cc:1240] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12. Ops: [9.9-11.10]
I20260709 13:20:23.281505 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:23.282196 15676 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
W20260709 13:20:23.292745 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 47023.8us
I20260709 13:20:23.269439 20024 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:23.317517 20320 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:23.318292 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:23.321014 15642 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:23.321570 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:23.321734 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:23.323170 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:23.349824 15594 raft_consensus.cc:2393] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 9: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:23.349644 15672 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:23.323377 15641 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:23.325412 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:23.350924 15675 raft_consensus.cc:2393] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 14: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:23.351280 15642 leader_election.cc:400] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13.
I20260709 13:20:23.346376 15669 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:20:23.303022 20448 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:23.349694 15676 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:23.351625 15642 leader_election.cc:403] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:20:23.359485 20447 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:23.364151 20447 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:23.364535 20447 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:23.368017 20024 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:23.368646 20406 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.369187 20406 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:23.371732 20406 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:23.363648 20481 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.382688 20481 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:23.381556 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 62506.9us
I20260709 13:20:23.384138 20422 raft_consensus.cc:3055] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:23.384549 20422 raft_consensus.cc:740] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:23.385105 20481 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:23.388283 20488 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:23.388821 20488 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:23.391080 20488 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:23.385634 16239 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:20:23.385449 20422 consensus_queue.cc:260] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:23.361449 20236 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13.
I20260709 13:20:23.386399 20469 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:23.387346 20448 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:23.393289 16239 raft_consensus.cc:606] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:23.405395 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 138604us
I20260709 13:20:23.405471 16239 raft_consensus.cc:614] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:20:23.410274 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 249857us
I20260709 13:20:23.420393 15675 raft_consensus.cc:1240] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:23.416144 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:23.437016 15669 raft_consensus.cc:2393] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 13: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:20:23.438701 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 115429us
I20260709 13:20:23.439576 15641 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:23.447978 15676 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:20:23.449805 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:23.422758 15567 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:23.451171 15671 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 5.
I20260709 13:20:23.456802 15641 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:23.459160 20450 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:20:23.466877 15641 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:23.464494 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:23.473192 20320 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:23.473773 20320 raft_consensus.cc:697] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:23.474682 20320 consensus_queue.cc:237] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:23.476476 20236 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:23.477725 20472 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:23.478319 20472 raft_consensus.cc:697] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:23.479254 20472 consensus_queue.cc:237] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:23.471967 20446 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:23.464910 20469 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:20:23.413868 19895 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:20:23.490060 20067 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:23.493511 16231 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:20:23.496651 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:23.494282 16231 raft_consensus.cc:606] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:23.498497 16231 raft_consensus.cc:614] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:23.499297 15641 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:23.494151 20481 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.410614 20448 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:23.507486 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:23.508262 15524 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:23.510811 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:23.515965 15521 raft_consensus.cc:2393] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 9: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:23.511318 20475 consensus_queue.cc:1048] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:20:23.517558 15491 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:23.504401 20469 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:20:23.525096 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:23.525518 15677 raft_consensus.cc:1275] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:20:23.528190 20475 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.522476 20452 consensus_queue.cc:1059] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:20:23.504285 20471 consensus_queue.cc:1048] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:20:23.528715 20475 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:23.510735 20236 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:23.512331 20446 consensus_queue.cc:1059] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 13:20:23.530339 20236 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:23.532990 20236 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:23.534776 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:23.536576 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:23.538061 20452 raft_consensus.cc:3055] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:23.538507 20452 raft_consensus.cc:740] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:23.539315 20452 consensus_queue.cc:260] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:23.543246 15642 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:23.544813 20447 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:23.545353 20469 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:23.508986 20481 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:23.547806 16231 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:20:23.548425 16231 raft_consensus.cc:606] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:23.554066 15524 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:20:23.566893 20475 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:23.570735 20359 raft_consensus.cc:697] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:23.571856 20359 consensus_queue.cc:237] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:23.548058 20481 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:23.578982 15493 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:23.583719 15642 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:23.585906 20450 consensus_queue.cc:1048] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:20:23.597877 15668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
mode: GRACEFUL
new_leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
 from {username='slave'} at 127.0.0.1:36980
I20260709 13:20:23.598203 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:23.598474 20448 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:23.598574 15668 raft_consensus.cc:606] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Received request to transfer leadership to TS 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:23.599213 15668 raft_consensus.cc:614] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:23.562408 20459 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:23.598791 16231 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:20:23.604189 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:23.609588 15641 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:23.613909 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:23.616860 20469 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:23.635722 20448 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:23.638024 15641 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:23.638686 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:23.638374 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:23.641211 15521 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
I20260709 13:20:23.639367 20469 raft_consensus.cc:697] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:23.646543 20475 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:23.647002 20475 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:23.647410 20475 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:23.647272 20469 consensus_queue.cc:237] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:23.622007 19724 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.584627 19787 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:23.643349 15641 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:20:23.643688 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 126207us
I20260709 13:20:23.644373 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:23.621130 20488 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:23.651516 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:20:23.636780 20460 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:23.668969 20488 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:23.669196 20460 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:23.670728 15491 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:23.679961 15491 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:23.681371 20446 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:23.682081 20446 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:23.682458 20446 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:23.682824 20446 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:23.684003 15641 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:23.689183 15491 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:23.691566 15491 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:23.692369 20468 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:23.692520 20496 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:23.692935 20468 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:23.693370 20468 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:23.695832 20496 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:23.696290 20496 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:23.692776 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:23.703727 15528 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:23.706291 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:23.707556 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:23.658725 19787 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:23.711778 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:23.676067 20320 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:23.712859 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:23.659826 19724 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:23.713795 15522 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 4.
I20260709 13:20:23.692512 20359 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:23.715408 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:23.718954 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:23.712755 20236 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.715962 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:23.698621 20447 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:23.724354 15670 raft_consensus.cc:2393] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 11: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:23.719650 15527 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:20:23.718351 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:23.710703 20450 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.713711 20460 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:23.739449 15528 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 15.
I20260709 13:20:23.742151 15641 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:23.742687 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:23.743450 15673 raft_consensus.cc:2393] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 12: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:23.724895 20359 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:23.722563 20488 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:23.750654 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:23.727361 20236 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:23.757861 20359 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:23.758754 20447 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:23.814982 16237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57258 (request call id 1011) took 1139 ms. Trace:
I20260709 13:20:23.815442 16237 rpcz_store.cc:276] 0709 13:20:22.675385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.675510 (+   125us) service_pool.cc:224] Handling call
0709 13:20:22.693682 (+ 18172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:23.188655 (+494973us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:23.814727 (+626072us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b54007b0f54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:23.814918 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1136898816}
I20260709 13:20:23.763747 20320 raft_consensus.cc:697] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:23.738031 20450 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:23.832343 20501 log.cc:927] Time spent T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:20:23.847256 20236 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:23.848589 20359 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
W20260709 13:20:23.848516 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:23.851989 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:20:23.848479 20504 log.cc:927] Time spent T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 13:20:23.883879 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:23.889096 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:23.830821 20469 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:23.903861 20502 log.cc:927] Time spent T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
W20260709 13:20:23.905645 20503 log.cc:927] Time spent T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:20:23.886317 20252 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.906322 20252 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:23.907413 20506 log.cc:927] Time spent T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:20:23.908970 20495 log.cc:927] Time spent T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:20:23.914031 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:23.920274 15595 raft_consensus.cc:2393] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 12: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:23.885908 20496 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:23.916463 20469 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:23.924901 20320 consensus_queue.cc:237] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:23.936473 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:23.916599 20447 raft_consensus.cc:697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:23.954341 20475 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:23.956327 15676 raft_consensus.cc:2393] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 15: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:23.926256 19724 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:23.913662 20446 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:23.963515 20485 log.cc:927] Time spent T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.696s	user 0.005s	sys 0.002s
I20260709 13:20:23.965466 20459 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:23.968565 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 2170) took 1210 ms. Trace:
I20260709 13:20:23.969329 20489 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.991755 15493 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:23.978674 15526 rpcz_store.cc:276] 0709 13:20:22.758563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.758693 (+   130us) service_pool.cc:224] Handling call
0709 13:20:23.533201 (+774508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:23.578392 (+ 45191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:23.578428 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:23.597327 (+ 18899us) log.cc:844] Serialized 26 byte log entry
0709 13:20:23.639885 (+ 42558us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:23.639948 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:23.640033 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:23.833452 (+193419us) raft_consensus.cc:1685] finished
0709 13:20:23.833472 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:23.968293 (+134821us) spinlock_profiling.cc:243] Waited 774 ms on lock 0x7b540029bdd0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:23.968478 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82095,"raft.queue_time_us":49687,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":10588,"spinlock_wait_cycles":1780357888,"thread_start_us":42103,"threads_started":1,"wal-append.queue_time_us":101825}
I20260709 13:20:24.000468 15525 raft_consensus.cc:1275] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:20:24.002240 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:23.995028 20400 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.000319 20460 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:23.996443 20406 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.016156 20406 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.010106 15603 raft_consensus.cc:1275] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:20:24.001941 20475 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:24.020855 15491 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:24.018769 20406 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:23.990569 20489 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.042474 20469 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:24.042881 20323 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:24.043905 20460 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:24.012789 20498 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.045372 20498 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.026193 20450 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:24.002264 20468 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:24.047969 15519 raft_consensus.cc:1240] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:24.047371 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:24.058863 15491 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:24.025727 20510 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:24.066690 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:24.067409 16236 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:24.034072 20459 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:24.070825 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:24.073654 15671 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:24.015350 20400 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.058777 19724 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.080262 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:24.065454 20514 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:24.082398 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 52951.9us
I20260709 13:20:24.084184 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:24.084750 15522 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:24.088291 20468 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:24.091078 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:24.091715 16231 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:24.096277 15602 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:20:24.084615 19724 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.096681 20339 consensus_queue.cc:1059] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:24.098489 20498 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:24.099090 20498 raft_consensus.cc:3055] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:24.099423 20498 raft_consensus.cc:740] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
W20260709 13:20:24.100490 20477 log.cc:927] Time spent T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.839s	user 0.008s	sys 0.000s
I20260709 13:20:24.076615 20406 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.104341 20406 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.074995 20510 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.109612 20446 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:24.112000 19724 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:24.113112 20513 log.cc:927] Time spent T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:20:24.113309 16237 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:24.114714 16204 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:24.114856 20489 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:24.115088 20400 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:24.122565 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:24.124310 16231 raft_consensus.cc:2410] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:20:24.122583 15522 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 15.
I20260709 13:20:24.026845 20447 consensus_queue.cc:237] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.125521 16204 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:24.087241 20496 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:24.134334 16204 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:24.109647 20510 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:24.147701 20359 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:24.142647 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:24.149227 16238 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:24.101722 20498 consensus_queue.cc:260] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.133723 19724 raft_consensus.cc:2804] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:24.161176 16231 raft_consensus.cc:1275] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 13:20:24.161619 16231 pending_rounds.cc:85] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9: Aborting all ops after (but not including) 9
I20260709 13:20:24.161844 16231 pending_rounds.cc:107] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:20:24.162089 16231 raft_consensus.cc:2889] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:24.136106 20400 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:24.164443 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:24.165150 16238 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:24.164055 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:24.168047 16240 raft_consensus.cc:2393] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 11: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:24.167901 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:24.168742 15522 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:24.169802 15528 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:20:24.156966 20205 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:24.153904 20496 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.154683 20406 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:24.175446 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:24.176067 15525 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:24.160215 19724 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.163914 20400 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:24.191074 20400 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:24.195659 20514 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:24.184695 19724 raft_consensus.cc:3060] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:24.186117 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:24.188314 20484 log.cc:927] Time spent T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.826s	user 0.003s	sys 0.000s
I20260709 13:20:24.183455 20496 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.178202 20151 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 13:20:24.208729 16238 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 15.
I20260709 13:20:24.235180 15599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 2073) took 1122 ms. Trace:
I20260709 13:20:24.235672 15599 rpcz_store.cc:276] 0709 13:20:23.113116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:23.137454 (+ 24338us) service_pool.cc:224] Handling call
0709 13:20:23.151522 (+ 14068us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:23.154823 (+  3301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:23.154856 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:23.173458 (+ 18602us) log.cc:844] Serialized 26 byte log entry
0709 13:20:23.248509 (+ 75051us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:23.248547 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:23.248616 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:24.171829 (+923213us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:20:24.234831 (+ 63002us) raft_consensus.cc:1685] finished
0709 13:20:24.234854 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:24.235088 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":685,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":13965,"spinlock_wait_cycles":42615680,"thread_start_us":53503,"threads_started":1,"wal-append.queue_time_us":109649}
I20260709 13:20:24.240468 20496 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:24.242949 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 158006us
I20260709 13:20:24.247867 15641 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:24.249245 20481 raft_consensus.cc:3055] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:24.247570 20519 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.250159 20510 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:24.250804 20510 raft_consensus.cc:697] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:24.251761 20510 consensus_queue.cc:237] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.253575 20498 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:24.250419 20519 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:24.256629 20519 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:24.249681 20481 raft_consensus.cc:740] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:24.260113 20481 consensus_queue.cc:260] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.261215 20481 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:24.266955 20422 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:24.272349 20489 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:24.272215 15603 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:20:24.272795 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:20:24.274986 15493 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:24.292557 15524 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:24.294301 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:24.295528 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:24.267285 19724 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:24.298385 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 111547us
I20260709 13:20:24.298990 15526 raft_consensus.cc:2393] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 15: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:24.298344 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:24.302657 15493 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:24.304462 20527 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:24.304888 20527 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:24.305241 20527 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:24.307175 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:24.272830 20467 consensus_queue.cc:1048] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:20:24.316035 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:24.317378 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:24.317945 16238 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:24.318537 16239 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:20:24.320695 15493 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:24.322547 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:24.323076 15669 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:20:24.272928 20489 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.327080 20400 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.296974 20514 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:20:24.359516 20489 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:24.360821 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:24.363291 20400 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:24.365834 15668 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:20:24.367142 15412 catalog_manager.cc:5244] ChangeConfig:REMOVE_PEER RPC for tablet bd36c2cfd99c45cea9e48096d54bed1a with cas_config_opid_index 5: aborting the task: latest config opid_index 6; task opid_index 5
I20260709 13:20:24.332338 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:24.340976 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:24.371443 15599 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:24.345147 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:24.372761 15600 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:24.345897 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:24.346453 20528 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:24.379591 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:24.351116 19724 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:24.381590 16238 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 15.
I20260709 13:20:24.352577 15524 raft_consensus.cc:1275] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:20:24.376269 15491 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:20:24.361482 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:24.387261 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:24.388126 20496 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:24.388672 20496 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.390122 20528 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:24.391054 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:24.391633 19787 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:24.392446 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:24.392638 20252 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:24.393025 15677 raft_consensus.cc:3060] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:24.388551 15595 raft_consensus.cc:1275] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:20:24.394657 19895 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.395371 19895 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.396762 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:24.397373 15598 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
I20260709 13:20:24.397586 19895 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:24.398233 19895 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.398665 19895 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.400872 19895 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:24.401839 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:24.406466 15493 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:24.407307 20488 raft_consensus.cc:2764] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
W20260709 13:20:24.410082 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 79260us
I20260709 13:20:24.410447 16236 raft_consensus.cc:1275] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:20:24.410737 15493 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:24.411975 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:24.412637 15671 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:24.413497 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:24.410550 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:24.414456 15603 raft_consensus.cc:1275] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:24.418216 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:24.419301 20516 log.cc:927] Time spent T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.372s	user 0.003s	sys 0.000s
I20260709 13:20:24.420245 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:24.420924 16239 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:24.413456 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:24.422972 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:24.426255 15566 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:24.427172 15566 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:24.427372 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:24.427945 15526 raft_consensus.cc:3060] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:20:24.428576 20467 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:24.428962 20467 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.429291 20467 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:24.429747 20452 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:24.430308 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:24.431017 15528 raft_consensus.cc:2368] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 14. Current term is 15.
I20260709 13:20:24.432142 20450 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:24.432825 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:24.434391 16238 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
I20260709 13:20:24.433569 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:24.433745 16204 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:24.433312 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:24.440899 15671 raft_consensus.cc:1240] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:24.441174 20425 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:24.441569 20425 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.441895 20425 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:24.432742 16236 raft_consensus.cc:1275] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:20:24.449129 15526 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:20:24.452076 16239 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 15.
I20260709 13:20:24.453140 15568 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:24.420158 20496 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:24.437338 15566 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:24.438280 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:24.439951 20529 log.cc:927] Time spent T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:20:24.606379 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:20:24.594506 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 207312us
I20260709 13:20:24.477456 20450 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:24.474963 20488 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:24.622103 20488 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:24.622574 20488 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:24.623946 20450 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:24.624426 20450 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:24.626874 15420 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49140 (request call id 362) took 18945 ms (client timeout 15000 ms). Trace:
I20260709 13:20:24.491391 20496 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:20:24.600380 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 189711us
I20260709 13:20:24.631953 15493 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:24.479107 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:24.608263 15677 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:20:24.608984 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:24.610046 15668 raft_consensus.cc:1240] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:24.609683 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:24.611691 16203 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:24.618204 20424 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:24.618927 20472 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:24.621300 20400 raft_consensus.cc:993] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9: : Instructing follower d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:20:24.634421 16236 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:24.636068 16205 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:24.646047 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:24.650470 15673 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:24.646776 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:24.646960 15669 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:20:24.647531 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:24.666170 15568 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:24.671799 16204 leader_election.cc:400] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 14. Current term is 15.
I20260709 13:20:24.672768 16204 leader_election.cc:403] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:20:24.673785 20465 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:24.674973 16205 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:24.675894 16205 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:24.676482 20406 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:24.676891 20406 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.677277 20406 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:24.678503 20350 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:24.679548 20467 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:24.680153 16203 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:24.682258 20464 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:24.672660 19895 raft_consensus.cc:1081] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Signalling peer d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:20:24.681885 20351 raft_consensus.cc:2764] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
W20260709 13:20:24.630827 15420 rpcz_store.cc:269] 0709 13:20:05.681852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:05.708762 (+ 26910us) service_pool.cc:224] Handling call
0709 13:20:24.626651 (+18917889us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:19.953713 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:19.957648 (+  3935us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:19.957691 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:20:19.957709 (+    18us) tablet.cc:662] Decoding operations
0709 13:20:20.189514 (+231805us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:20.189581 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:20.189608 (+    27us) tablet.cc:685] Acquiring locks for 24 operations
0709 13:20:20.552936 (+363328us) tablet.cc:701] Row locks acquired
0709 13:20:20.556040 (+  3104us) write_op.cc:260] PREPARE: finished
0709 13:20:20.590090 (+ 34050us) write_op.cc:270] Start()
0709 13:20:20.599670 (+  9580us) write_op.cc:276] Timestamp: P: 1783603220585634 usec, L: 0
0709 13:20:20.602873 (+  3203us) op_driver.cc:348] REPLICATION: starting
0709 13:20:20.722425 (+119552us) log.cc:844] Serialized 9367 byte log entry
0709 13:20:21.405581 (+683156us) op_driver.cc:464] REPLICATION: finished
0709 13:20:21.615241 (+209660us) spinlock_profiling.cc:243] Waited 6.72 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce19010ac 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce8441ef0
0709 13:20:21.983660 (+368419us) write_op.cc:301] APPLY: starting
0709 13:20:22.013713 (+ 30053us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:22.308446 (+294733us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:22.308492 (+    46us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:22.520527 (+212035us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:22.520983 (+   456us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=48,key_file_lookups=48,delta_file_lookups=0,mrs_lookups=0
0709 13:20:22.521033 (+    50us) write_op.cc:312] APPLY: finished
0709 13:20:22.521599 (+   566us) log.cc:844] Serialized 212 byte log entry
0709 13:20:22.626104 (+104505us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:22.661111 (+ 35007us) write_op.cc:454] Released schema lock
0709 13:20:22.661187 (+    76us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:22.666277 (+  5090us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:22.666309 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":131770,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":516760,"apply.run_cpu_time_us":64391,"apply.run_wall_time_us":728583,"cfile_cache_hit":98,"cfile_cache_hit_bytes":104639,"num_ops":24,"prepare.queue_time_us":306461,"prepare.run_cpu_time_us":44628,"prepare.run_wall_time_us":918513,"raft.queue_time_us":190774,"raft.run_cpu_time_us":23080,"raft.run_wall_time_us":432657,"replication_time_us":797776,"spinlock_wait_cycles":15451904,"thread_start_us":1053410,"threads_started":5,"wal-append.queue_time_us":384182,"wal-append.run_cpu_time_us":15796,"wal-append.run_wall_time_us":2531987}]]}
I20260709 13:20:24.687860 20026 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 14. Current term is 15.
I20260709 13:20:24.689533 16204 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:24.693573 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 256299us
I20260709 13:20:24.691818 20425 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.692237 20488 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.694123 15642 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:24.718503 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:24.719080 15676 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:24.718384 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:24.719694 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:24.720259 15602 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:20:24.715950 15673 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 15.
I20260709 13:20:24.669010 20151 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:24.727231 20488 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:24.728582 20339 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.729100 20339 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:24.707846 20519 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:24.730093 20519 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.730506 20519 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:24.731117 20339 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:24.732646 20339 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:24.733134 20339 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.681856 20467 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:24.716748 19895 raft_consensus.cc:2804] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:24.739950 16205 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:24.724184 20151 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.734078 19787 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:24.757162 20510 consensus_queue.cc:1048] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:20:24.749140 20538 log.cc:927] Time spent T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 13:20:24.757791 20527 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.741423 20452 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:24.760604 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 128735us
I20260709 13:20:24.760633 20481 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:24.760787 20514 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.749097 15676 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
W20260709 13:20:24.764432 20534 log.cc:927] Time spent T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:20:24.756902 20463 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:24.666755 20472 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.771749 20252 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.774046 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:24.759852 20425 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:24.724032 20489 consensus_queue.cc:1059] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:20:24.750118 20422 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:24.771507 19895 raft_consensus.cc:697] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:24.782356 15493 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:24.790326 20422 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.792296 20422 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:24.780846 15566 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:24.794485 20520 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:24.796274 19895 consensus_queue.cc:237] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.797391 20465 raft_consensus.cc:3055] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:24.797852 20465 raft_consensus.cc:740] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:24.798944 20342 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:24.799304 20489 consensus_queue.cc:1059] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:24.802294 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:24.772384 20151 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:24.786810 15672 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:24.768627 20496 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:24.799564 20342 raft_consensus.cc:697] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:24.718042 15526 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:56840
I20260709 13:20:24.807010 15526 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:24.807226 16236 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:20:24.807503 15526 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:24.798873 20465 consensus_queue.cc:260] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:24.776175 19787 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.811386 20465 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:24.812625 19787 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:24.815330 20410 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:24.815893 20410 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:24.792234 20488 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:24.819940 20543 consensus_queue.cc:1048] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:20:24.822252 20528 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:24.824008 20528 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.783653 20514 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.826136 20432 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.822562 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:24.828365 20514 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:24.823031 20488 raft_consensus.cc:697] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:24.831708 20488 consensus_queue.cc:237] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.832466 20410 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:24.826680 20432 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.837023 20323 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.780757 20452 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:24.840731 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:24.846673 16238 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:24.848016 15568 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:24.851665 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:24.852325 15522 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:24.854194 15522 raft_consensus.cc:1240] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:24.800650 20463 raft_consensus.cc:697] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:24.857599 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:24.860212 15525 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:24.837570 20323 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:24.785656 20359 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:20:24.787246 20406 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.872958 15519 raft_consensus.cc:1240] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:24.788911 20252 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:24.881742 20450 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.883854 20252 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:24.884480 20252 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:24.884851 20252 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.885191 20252 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:24.846189 20432 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:24.873486 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:24.874131 20528 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:24.826603 20342 consensus_queue.cc:237] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:24.889042 20463 consensus_queue.cc:237] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.876143 20339 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:24.878360 20323 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:24.880182 20452 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:24.901947 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 44719.1us
W20260709 13:20:24.918674 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:24.899238 15526 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:24.919225 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:24.919996 15603 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:24.920727 15521 raft_consensus.cc:1275] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:20:24.922344 15519 raft_consensus.cc:1275] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:20:24.883983 20450 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:24.934044 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:24.933313 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:24.919188 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 69762.9us
I20260709 13:20:24.927903 15525 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:24.929021 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:24.942317 15521 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:24.856547 20489 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:24.930085 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:24.931293 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:24.950513 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:24.945181 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:24.951221 15524 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:24.952813 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:24.955157 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 243637us
I20260709 13:20:24.957165 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:24.950538 15519 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:24.990610 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:24.990628 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:24.956357 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:24.991145 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:25.017442 15674 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:24.932974 20339 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.087s
W20260709 13:20:25.022087 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 95268.9us
I20260709 13:20:25.028997 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:20:25.017984 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 61761.3us
I20260709 13:20:25.023447 15520 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:24.949352 15603 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:20:24.994704 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:24.952225 20024 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.908799 20342 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.962904 15523 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:20:25.058393 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 44176.2us
I20260709 13:20:25.086412 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:25.087224 16236 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:25.047675 20448 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:25.114688 15603 raft_consensus.cc:1240] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:25.115904 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:25.052706 20505 log.cc:927] Time spent T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:20:24.990682 20496 consensus_queue.cc:1048] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.100s
I20260709 13:20:25.115968 15493 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:25.009931 20489 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:25.138002 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:25.143146 15594 raft_consensus.cc:1275] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:20:25.128865 16205 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:25.134574 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:25.138816 15595 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
W20260709 13:20:25.145645 20544 log.cc:927] Time spent T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:20:25.112063 20527 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:25.113726 20051 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.121342 20448 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.089769 20024 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.139760 20514 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:25.085263 20422 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.089435 20541 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:20:25.094561 20342 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:25.154578 20465 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:25.202540 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:25.212383 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:25.213310 15594 raft_consensus.cc:2368] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 12. Current term is 13.
I20260709 13:20:25.190958 20448 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:25.210922 16235 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:20:25.216111 15600 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:25.189563 20514 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:25.193488 20472 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:25.222504 20533 log.cc:927] Time spent T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.918s	user 0.001s	sys 0.003s
W20260709 13:20:25.222694 20545 log.cc:927] Time spent T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:20:25.199602 20400 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:25.238130 15674 raft_consensus.cc:1275] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:20:25.238585 15491 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:25.241173 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:25.217234 20465 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.232674 20514 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:25.190692 20051 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.220638 20406 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:25.267664 15491 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:25.269315 15566 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:25.201152 20551 consensus_queue.cc:1048] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:20:25.276347 15491 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:25.263370 20359 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:25.281867 20024 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:25.271937 20519 consensus_meta.cc:287] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.541s	user 0.011s	sys 0.000s
I20260709 13:20:25.282575 20553 consensus_queue.cc:1048] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:20:25.282665 20519 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.283533 20252 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.284844 15601 raft_consensus.cc:2368] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11.
I20260709 13:20:25.286329 20467 raft_consensus.cc:3055] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:25.286731 20467 raft_consensus.cc:740] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:25.276149 20450 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:25.287225 20450 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.281908 20422 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:25.287593 20467 consensus_queue.cc:260] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:25.287595 20450 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:25.289822 15598 raft_consensus.cc:2393] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 14: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:25.292973 20151 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.294437 16239 raft_consensus.cc:2368] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 12. Current term is 13.
I20260709 13:20:25.279775 15491 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:25.318650 20151 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:25.319643 20425 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.320199 20425 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.322404 20425 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:25.323918 20450 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:25.326620 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:25.327565 16238 raft_consensus.cc:2368] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 13. Current term is 14.
I20260709 13:20:25.330987 16231 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:20:25.337888 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 58258us
I20260709 13:20:25.277753 20481 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:25.344926 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:25.345568 15522 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:25.280263 20541 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:25.282480 15660 tablet_service.cc:1558] Processing DeleteTablet for tablet bd36c2cfd99c45cea9e48096d54bed1a with delete_type TABLET_DATA_TOMBSTONED (TS 3ef186ec12554a1d87814af0a0ace9a4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:49088
I20260709 13:20:25.354764 16240 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:25.295616 20520 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:25.361245 15520 raft_consensus.cc:1240] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:20:25.350158 20359 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:25.350411 20472 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.363096 20472 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.371140 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:25.371924 15528 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:20:25.280046 20548 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.076s
I20260709 13:20:25.279651 20528 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 13:20:25.376933 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:25.385118 15520 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:25.387666 20566 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.388238 20566 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.349747 20488 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:25.390221 20488 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.391268 20051 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:25.392441 20488 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:25.381760 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:25.397469 15528 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:25.365312 20519 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:25.407961 15520 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:20:25.367179 20450 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:25.402549 16204 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:25.431330 16204 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:25.432240 20462 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:25.432657 16204 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:25.432866 20563 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:25.365315 20472 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:25.433297 20563 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.433683 20563 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:25.434391 20406 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:25.434948 20406 raft_consensus.cc:697] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:25.435851 20406 consensus_queue.cc:237] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.412395 20342 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:25.418727 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 201946us
I20260709 13:20:25.423875 20422 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.426868 15522 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:20:25.407358 20024 consensus_queue.cc:1059] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:20:25.432678 20462 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.418167 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:25.462034 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:25.462725 15674 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:20:25.463877 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:25.464522 15672 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:20:25.467537 15669 raft_consensus.cc:1275] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:20:25.468322 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:25.471549 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:25.454773 20566 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:25.472142 15674 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:25.456737 20432 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:25.476011 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:20:25.460054 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 257586us
I20260709 13:20:25.480747 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:25.473037 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:25.481892 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:25.467303 20462 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:25.480306 15673 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:25.493393 15642 leader_election.cc:400] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 13. Current term is 14.
I20260709 13:20:25.494175 15642 leader_election.cc:403] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:25.495476 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:25.496198 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:25.534006 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 44355.1us
I20260709 13:20:25.507258 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:25.506533 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:25.519840 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:25.540036 15674 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:25.541424 15566 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:25.467068 20422 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.547016 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:25.547402 15491 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:25.507977 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:25.544281 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:25.544469 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:25.545531 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 49695.9us
I20260709 13:20:25.550372 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:25.492231 20566 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:25.483497 20432 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.496495 20447 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:25.550133 16237 raft_consensus.cc:1240] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:20:25.551204 15597 raft_consensus.cc:2410] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:20:25.481483 20448 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.560614 20528 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.576547 15669 raft_consensus.cc:2393] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 9: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:25.492759 20519 consensus_queue.cc:1048] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.101s
I20260709 13:20:25.550992 20570 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:25.620946 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:25.556452 20422 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:25.558917 20550 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:25.486335 20400 raft_consensus.cc:697] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:25.584121 15673 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:20:25.637609 20579 tablet_replica.cc:333] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:20:25.640301 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:25.641376 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:25.632462 20488 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:25.596552 20568 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:20:25.543625 20541 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.105s
I20260709 13:20:25.668018 20578 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.641993 20550 raft_consensus.cc:697] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:25.627466 20528 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.710003 20481 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.678038 20447 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:25.714448 20339 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:25.729012 15567 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.034s	sys 0.055s
W20260709 13:20:25.749989 15567 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.037s	sys 0.055s
W20260709 13:20:25.750342 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 662675us
I20260709 13:20:25.744252 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:25.745958 15426 catalog_manager.cc:5697] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:25.740819 20481 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.753470 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:25.542193 20567 consensus_queue.cc:1048] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.170s
I20260709 13:20:25.747850 20563 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.758054 15520 raft_consensus.cc:2393] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 5: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:25.764756 15525 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:25.765147 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:25.766965 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:25.767680 15520 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:25.770511 15567 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:25.771440 15522 raft_consensus.cc:1275] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:20:25.771916 15567 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:25.641712 20570 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:25.778520 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:25.797758 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:25.798660 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:25.799809 16233 raft_consensus.cc:2393] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 14: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:25.754097 20541 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:25.774367 20400 consensus_queue.cc:237] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.777616 20586 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.709842 20578 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:25.806154 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:25.814670 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:25.814858 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:25.820518 15525 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:25.815017 15672 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
W20260709 13:20:25.801468 20514 consensus_meta.cc:287] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.543s	user 0.010s	sys 0.000s
W20260709 13:20:25.801702 20575 log.cc:927] Time spent T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:20:25.805879 15426 catalog_manager.cc:5697] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 5 to 7, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 7 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:25.819541 15528 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:25.799098 20450 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:25.806070 20583 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:20:25.827732 20563 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:25.828817 20583 consensus_queue.cc:1059] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:25.752039 20339 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.830310 20567 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:25.824893 20586 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:25.833206 20581 log.cc:927] Time spent T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:20:25.757469 20447 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.833842 20586 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:25.870484 20450 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.847577 20051 raft_consensus.cc:3055] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:20:25.870961 20450 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:25.818856 20541 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.873605 20541 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:25.822381 20548 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 13. Current term is 14.
I20260709 13:20:25.834470 20510 raft_consensus.cc:3055] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:25.838876 15528 raft_consensus.cc:1240] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:25.842752 20465 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:25.806914 20566 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:20:25.804548 20472 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:25.846392 20252 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:25.857419 20514 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.834990 20448 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:25.876288 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:25.876747 20051 raft_consensus.cc:740] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:25.824250 20550 consensus_queue.cc:237] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:25.841661 20578 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:25.834816 20432 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:25.857189 20481 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:25.845118 20570 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:25.891073 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:25.886793 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:25.896525 20472 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.886987 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:25.883505 15426 catalog_manager.cc:5697] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 5 to 10, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 10 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:25.879706 15525 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
W20260709 13:20:25.846145 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:25.896454 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 302087us
I20260709 13:20:25.901681 15599 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:25.890619 20551 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:25.895092 20586 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.919741 15521 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:25.951562 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:25.954394 20551 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.956539 15526 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:25.906021 20567 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:20:25.957602 20051 consensus_queue.cc:260] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.958746 20051 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:25.963131 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:25.963668 20339 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:25.963864 16234 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:25.947476 20432 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:25.965306 20432 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:25.965768 20462 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:25.967810 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:25.949958 15603 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:25.949986 15426 catalog_manager.cc:5697] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 8 to 12, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 12 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:25.957098 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:56840 (request call id 1647) took 1254 ms. Trace:
I20260709 13:20:25.893764 20510 raft_consensus.cc:740] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:25.921905 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:25.882045 20447 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:25.935705 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:25.967918 16203 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:25.884176 20528 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:25.903185 20582 log.cc:927] Time spent T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.161s	user 0.004s	sys 0.000s
I20260709 13:20:25.963040 20481 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:25.974536 15526 rpcz_store.cc:276] 0709 13:20:24.713702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:24.713857 (+   155us) service_pool.cc:224] Handling call
0709 13:20:25.956037 (+1242180us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b4c003d8280. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce2cd7a20 00007f5ce2ce8a29 00007f5ce2ce913f 00007f5ce2cad2ab 00007f5ce2cdaf3c 00007f5ce2cdb3fe 00007f5ce2f6888b 00007f5ce4967495 00007f5ce49b830f 00007f5ce4a02a55 00007f5ce99d2cd0
0709 13:20:25.957010 (+   973us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28592384}
I20260709 13:20:25.980580 15525 raft_consensus.cc:1240] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:20:25.948136 20472 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:25.942842 20448 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:25.984042 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:25.986289 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:25.996688 15528 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:20:25.944183 15597 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:25.988899 16234 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:25.956756 20551 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:25.980381 15426 catalog_manager.cc:5697] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:25.998433 15568 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:25.979598 15595 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:20:26.000100 15568 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:26.004964 15599 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 15.
I20260709 13:20:26.013875 16233 raft_consensus.cc:2393] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 11: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
W20260709 13:20:26.039426 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 71578.2us
W20260709 13:20:26.040665 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 260307us
I20260709 13:20:26.040995 16203 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:26.042232 15602 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
W20260709 13:20:26.043150 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 50912us
I20260709 13:20:26.045650 15598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34102 (request call id 1369) took 1333 ms. Trace:
W20260709 13:20:26.044258 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 227327us
I20260709 13:20:26.046506 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:26.047705 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:26.047128 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:26.048993 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:26.049583 15677 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 15.
I20260709 13:20:26.050597 15671 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:26.050896 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:26.052068 15642 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:26.046068 15598 rpcz_store.cc:276] 0709 13:20:24.706131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:24.706275 (+   144us) service_pool.cc:224] Handling call
0709 13:20:26.038806 (+1332531us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b540016bf54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:26.038969 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1310257536}
I20260709 13:20:26.058163 15528 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:20:26.035193 20553 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:25.979249 20586 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:26.060218 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:26.045519 20406 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:26.093139 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:26.129235 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:26.092132 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:26.076704 15597 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:20:26.039984 20551 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:20:26.068441 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 290971us
I20260709 13:20:26.116099 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:26.059777 20592 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:26.054975 20432 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:26.123247 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 66464us
I20260709 13:20:26.066885 20562 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:26.015199 20481 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:26.035588 20448 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:26.133958 16238 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:20:26.128672 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:26.136912 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 43851.4us
I20260709 13:20:26.139453 16233 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:26.121316 15674 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:26.136006 16240 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:20:26.144865 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:26.138566 15491 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:26.145502 15597 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:26.102569 20568 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:26.137625 15567 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:26.067346 20510 consensus_queue.cc:260] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:26.079487 20450 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:26.147747 20551 raft_consensus.cc:697] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:26.149381 20067 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
W20260709 13:20:26.152922 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:26.161062 15671 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:26.169430 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:26.166491 15426 catalog_manager.cc:5697] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 4 to 11, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 11 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:26.158311 20514 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:26.179328 15426 catalog_manager.cc:5697] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:26.172461 15597 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:20:26.176198 15567 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:26.161983 20481 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:26.160693 20579 raft_consensus.cc:2243] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:26.143021 20462 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:26.190279 20579 raft_consensus.cc:2272] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:26.193558 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 43764.6us
I20260709 13:20:26.149399 20339 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:26.194710 16204 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:26.159029 20510 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:26.163414 20448 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:26.195786 15521 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:26.175027 20432 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:26.150103 20596 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:26.202770 15567 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:26.163873 20578 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:20:26.177843 20562 raft_consensus.cc:697] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:26.196089 20339 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:26.209476 20339 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:26.212164 16205 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:26.165700 20067 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:26.215437 16203 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:26.159091 20541 consensus_queue.cc:1175] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" responder_term: 10 status { last_received { term: 10 index: 10 } last_committed_idx: 9 last_received_current_leader { term: 10 index: 10 } }
I20260709 13:20:26.159240 20568 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:26.224124 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:26.201262 20514 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.223850 15567 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:26.220985 20465 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.225068 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:26.208689 20565 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:26.227056 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:26.229578 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:26.162909 20551 consensus_queue.cc:237] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:26.231280 20593 log.cc:927] Time spent T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:20:26.218916 19895 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:20:26.387684 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 172324us
I20260709 13:20:26.220328 20596 raft_consensus.cc:697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:26.387589 15600 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:26.255259 20579 ts_tablet_manager.cc:1916] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:26.396343 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:26.222661 20450 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:26.254149 20252 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:26.452222 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:26.646806 15527 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 10
W20260709 13:20:26.404580 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 127350us
W20260709 13:20:26.421558 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:26.453632 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1651) took 1110 ms. Trace:
I20260709 13:20:26.656364 15523 rpcz_store.cc:276] 0709 13:20:25.342989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:25.343125 (+   136us) service_pool.cc:224] Handling call
0709 13:20:25.362585 (+ 19460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:25.391322 (+ 28737us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:25.391359 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:25.391404 (+    45us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:25.391436 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:26.453406 (+1061970us) spinlock_profiling.cc:243] Waited 28.5 ms on lock 0x7b5400262854. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:26.453450 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:26.453575 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65547136}
I20260709 13:20:26.658749 16205 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:26.645303 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 421519us
I20260709 13:20:26.643301 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:26.651762 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:26.660473 15598 raft_consensus.cc:2368] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13.
I20260709 13:20:26.456760 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:26.661295 16239 raft_consensus.cc:2368] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11.
I20260709 13:20:26.455865 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:26.455013 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:26.456408 15596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 2385) took 1341 ms. Trace:
I20260709 13:20:26.223836 20321 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:26.660324 15567 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:26.333858 20489 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:26.420262 20601 log.cc:927] Time spent T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.199s	user 0.000s	sys 0.000s
I20260709 13:20:26.701046 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:26.703935 16238 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:26.655726 20569 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:26.701046 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:26.230036 20448 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.419355 20481 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.702845 15567 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:26.431541 20603 log.cc:927] Time spent T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 13:20:26.684850 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:26.725984 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:26.726612 15671 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:26.422387 20597 log.cc:927] Time spent T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 13:20:26.729266 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:26.729561 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:26.403208 20528 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:26.440383 20586 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:26.441865 20568 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:26.673081 20425 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:26.719930 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:26.732208 15674 raft_consensus.cc:2393] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 15: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:26.433122 19895 raft_consensus.cc:697] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:26.749213 19895 consensus_queue.cc:237] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:26.720690 20448 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:26.752019 20448 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:26.472465 20051 consensus_meta.cc:287] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.513s	user 0.007s	sys 0.001s
I20260709 13:20:26.396937 20452 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.644235 20465 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:26.752527 20528 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:26.754733 20528 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:26.755162 20465 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:26.412986 20576 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.299s
I20260709 13:20:26.756268 20528 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.756832 20528 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:26.758433 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:26.759039 15674 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:20:26.468135 20527 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:26.739609 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:26.760545 20527 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:26.762063 20543 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:26.731089 20481 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:26.705785 20342 raft_consensus.cc:2764] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:20:26.746062 20552 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:26.738052 16233 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
I20260709 13:20:26.757368 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:26.764029 20481 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:26.663050 15596 rpcz_store.cc:276] 0709 13:20:25.115003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:25.115133 (+   130us) service_pool.cc:224] Handling call
0709 13:20:25.124341 (+  9208us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:25.158012 (+ 33671us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:20:25.184140 (+ 26128us) log.cc:844] Serialized 14 byte log entry
0709 13:20:25.748856 (+564716us) spinlock_profiling.cc:243] Waited 4.75 ms on lock 0x7b18002de320. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab23a 00007f5ce4a15a2a
0709 13:20:26.043801 (+294945us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:26.043860 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:26.043916 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:26.454848 (+410932us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x7b18002de320. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce18ec2a6 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419
0709 13:20:26.456157 (+  1309us) spinlock_profiling.cc:243] Waited 33.4 ms on lock 0x7b5400168d54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:26.456207 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:26.456342 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":211,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":81,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":14622,"spinlock_wait_cycles":94050688,"thread_start_us":294722,"threads_started":1,"wal-append.queue_time_us":948371}
I20260709 13:20:26.671564 20578 consensus_queue.cc:1059] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.095s
I20260709 13:20:26.766341 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:26.754635 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:26.763764 20339 consensus_meta.cc:287] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.550s	user 0.008s	sys 0.000s
I20260709 13:20:26.407081 20067 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:26.767421 20339 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:26.769618 20339 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:26.770185 20339 raft_consensus.cc:3055] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:26.762629 20527 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:26.766638 20552 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:26.770355 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:26.748205 15671 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:26.772315 16238 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 15.
I20260709 13:20:26.765424 15566 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:26.775476 15674 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:26.402727 20553 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:26.771067 15642 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:26.771016 20467 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:26.785794 15566 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:26.404857 20572 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.328s
I20260709 13:20:26.797887 15527 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:26.733811 20323 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:26.734629 20489 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.801808 20465 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:26.434057 20562 consensus_queue.cc:237] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:26.796591 20552 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:26.804384 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 351668us
I20260709 13:20:26.777087 20568 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.770547 20339 raft_consensus.cc:740] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:26.662899 20514 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:26.785503 20342 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:26.818969 15493 leader_election.cc:400] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 12. Current term is 13.
I20260709 13:20:26.821902 15493 leader_election.cc:403] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:20:26.819221 16203 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:26.823577 15493 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:26.825165 15493 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:26.827960 15493 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:26.829316 15493 leader_election.cc:400] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11.
W20260709 13:20:26.831466 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 60495us
I20260709 13:20:26.832242 15642 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:26.833288 20586 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:26.815925 20465 raft_consensus.cc:697] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:26.783259 20452 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:26.830531 19724 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:26.827080 20527 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 12. Current term is 13.
I20260709 13:20:26.844604 15642 leader_election.cc:400] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13.
I20260709 13:20:26.805423 20589 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:20:26.848477 15642 leader_election.cc:403] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:26.831169 20481 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:26.765422 20596 consensus_queue.cc:237] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:26.815172 20489 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:26.801381 20553 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:26.846485 20339 consensus_queue.cc:260] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:26.851482 20546 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:26.851725 20481 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:26.860648 19724 raft_consensus.cc:697] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:26.860491 20339 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:26.863041 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:26.853042 20448 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.864655 20448 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:26.854313 20608 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:26.865638 20608 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:26.866024 20608 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:26.820955 20568 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:26.866954 20448 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:26.868155 20568 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:26.852196 20592 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13.
I20260709 13:20:26.861774 19724 consensus_queue.cc:237] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:26.873769 20565 consensus_meta.cc:287] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.513s	user 0.001s	sys 0.006s
I20260709 13:20:26.874301 20546 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:26.876401 20546 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:26.877352 20565 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:20:26.882536 20602 log.cc:927] Time spent T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 13:20:26.886044 20552 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:26.878767 16237 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:20:26.888847 15597 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:20:26.891122 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:26.891922 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:26.892611 15675 raft_consensus.cc:1275] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:20:26.894450 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:26.895076 15595 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
W20260709 13:20:26.888716 20472 consensus_meta.cc:287] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.850s	user 0.010s	sys 0.000s
I20260709 13:20:26.898782 15642 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:26.903894 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:26.904100 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:26.904860 16240 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 15.
I20260709 13:20:26.906502 15527 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:26.852126 20465 consensus_queue.cc:237] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:26.902772 20510 consensus_meta.cc:287] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.698s	user 0.007s	sys 0.002s
I20260709 13:20:26.923218 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:26.927110 15527 raft_consensus.cc:2368] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 15. Current term is 16.
I20260709 13:20:26.913148 20609 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.917515 15521 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:20:26.908708 20528 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:26.908154 20448 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:26.928506 15528 raft_consensus.cc:1275] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:20:26.904699 20339 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:26.900998 20463 consensus_queue.cc:1048] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:20:26.907789 20472 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:26.941160 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:26.943513 16204 leader_election.cc:400] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 15. Current term is 16.
I20260709 13:20:26.948766 16204 leader_election.cc:403] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:26.966241 20510 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:26.966905 20510 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:26.970768 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:26.971550 20510 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:26.972226 20472 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:26.974220 15600 raft_consensus.cc:1240] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:20:26.937896 20452 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:20:26.941860 20448 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:26.984754 20510 consensus_queue.cc:1059] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:26.953171 20321 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:20:27.019269 15602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 2087) took 2108 ms. Trace:
I20260709 13:20:26.948329 20339 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.019776 15602 rpcz_store.cc:276] 0709 13:20:24.911443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:24.911617 (+   174us) service_pool.cc:224] Handling call
0709 13:20:26.042001 (+1130384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:26.042662 (+   661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:27.019009 (+976347us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540016bf50. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:27.019161 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2551213440}
I20260709 13:20:26.994365 20578 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.998371 20592 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:26.938768 20609 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:27.015369 20510 raft_consensus.cc:3055] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:20:27.021657 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 78324.8us
W20260709 13:20:27.037436 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 73438.3us
I20260709 13:20:26.993213 20546 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.009727 20448 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:27.062633 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 232156us
I20260709 13:20:26.983457 20514 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:27.037309 20424 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:20:27.067400 20514 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.067935 20514 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:27.068776 15493 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:27.071249 20339 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:27.072026 15493 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:27.071245 20514 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:27.037104 20252 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:27.073678 20514 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:27.074254 20252 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:27.074645 20514 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:27.075277 20514 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:27.075637 20514 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:27.071954 20339 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.076577 20339 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:27.078001 20432 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:27.078562 20432 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.080456 20424 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 15. Current term is 16.
I20260709 13:20:27.086266 15673 raft_consensus.cc:2393] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 16: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:27.045516 20510 raft_consensus.cc:740] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:27.044261 20604 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:27.079013 20339 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:27.126286 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:27.126363 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 80318.1us
I20260709 13:20:27.127017 16235 raft_consensus.cc:1813] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Handling vote request from an unknown peer 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:27.127575 16235 raft_consensus.cc:2410] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 13:20:27.127556 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 54642.5us
I20260709 13:20:27.045126 20592 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.050390 20578 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:27.130012 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:27.132664 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:27.144557 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:27.145432 15677 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:20:27.148058 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:27.148465 15491 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:27.148790 15528 raft_consensus.cc:2393] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 16: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:27.115715 20553 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:27.151264 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:27.153513 15527 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:27.154981 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:27.155225 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:27.155886 15519 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:20:27.161361 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:27.164053 15519 raft_consensus.cc:2393] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 12: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:20:27.171905 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:27.172184 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 44896.5us
W20260709 13:20:27.174151 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 63063.6us
I20260709 13:20:27.182775 20543 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.183415 20543 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.185596 20543 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:27.133343 20563 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:20:27.158453 20339 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:27.189440 15568 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:27.204654 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:27.205358 15527 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
I20260709 13:20:27.134291 20622 consensus_queue.cc:1059] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:20:27.210698 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:27.168066 19895 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:27.215399 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:27.217681 15522 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
W20260709 13:20:27.202260 20606 log.cc:927] Time spent T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 13:20:27.129770 20604 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.224002 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:27.189106 20067 consensus_queue.cc:1048] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:20:27.202450 20339 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:27.213512 20472 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:27.236860 15493 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:27.220394 20595 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:27.243508 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:27.244469 15527 raft_consensus.cc:2393] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 16: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:27.165488 20510 consensus_queue.cc:260] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:27.247792 15493 leader_election.cc:400] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 12. Current term is 13.
I20260709 13:20:27.248179 15493 leader_election.cc:403] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:20:27.249382 15493 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:27.250269 20425 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:27.251511 20592 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:27.252174 20622 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:27.227388 19895 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.252961 20622 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 12. Current term is 13.
I20260709 13:20:27.234705 20339 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:27.255609 20622 raft_consensus.cc:2764] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:20:27.251399 20604 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:27.258024 20617 log.cc:927] Time spent T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
W20260709 13:20:27.262307 20614 log.cc:927] Time spent T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 13:20:27.265475 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:27.266287 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:27.266987 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:27.269968 15493 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:27.270691 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:27.232100 20610 raft_consensus.cc:3055] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:27.273180 15566 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:27.253410 20252 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:27.288007 15493 leader_election.cc:400] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11.
I20260709 13:20:27.301200 15493 leader_election.cc:403] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:20:27.297169 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:27.302021 15677 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:27.306286 15566 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:27.312058 15493 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:27.284575 20610 raft_consensus.cc:740] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:27.296308 20616 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:27.271225 20546 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:27.285528 20595 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:20:27.332530 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 173424us
I20260709 13:20:27.247916 20472 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.336395 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:27.343127 15641 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:27.352735 15641 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:27.344254 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:27.354908 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:27.347311 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:27.357014 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:27.358172 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:27.333134 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 160296us
I20260709 13:20:27.307802 20465 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:27.362864 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:27.363207 15526 raft_consensus.cc:1275] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:20:27.363162 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:27.367090 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:27.295609 20622 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:20:27.368069 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:27.363407 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:27.368283 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:27.371513 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:27.371659 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:27.378360 15641 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:27.379235 20568 raft_consensus.cc:2804] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:27.379623 20568 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:27.379994 20568 raft_consensus.cc:3060] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:27.362553 20546 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:27.392567 20546 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:27.393007 20546 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:27.302877 20514 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.373809 16233 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:20:27.396633 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:27.427518 16235 raft_consensus.cc:2393] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 11: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:27.429911 16234 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:27.436612 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:27.440205 15526 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:27.357173 20528 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:27.348644 20631 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11.
I20260709 13:20:27.448887 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:27.367713 20586 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:27.366251 20465 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:20:27.452163 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 45940.2us
I20260709 13:20:27.466743 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:27.509672 15527 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:27.505708 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:27.510530 15520 raft_consensus.cc:2368] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 11. Current term is 12.
I20260709 13:20:27.508049 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:27.449291 20593 log.cc:927] Time spent T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:20:27.448504 20467 consensus_meta.cc:287] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.645s	user 0.009s	sys 0.004s
W20260709 13:20:27.520936 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 113685us
I20260709 13:20:27.547202 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:27.553500 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 43805us
I20260709 13:20:27.547290 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:27.556836 15524 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:20:27.428306 20578 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:27.428666 20595 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:27.443060 20496 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:27.563647 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 131536us
W20260709 13:20:27.567248 20652 log.cc:927] Time spent T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:20:27.569116 15595 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:20:27.515007 20465 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:27.465600 20586 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:27.584149 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:27.586621 15596 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:27.592370 20578 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:27.578348 20519 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.443739 20610 consensus_queue.cc:260] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.551396 20448 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:27.589530 20586 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:27.613708 16231 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 16.
W20260709 13:20:27.647207 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 84909.7us
I20260709 13:20:27.584792 20643 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:27.673590 15670 raft_consensus.cc:2393] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 14: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:20:27.612172 20608 consensus_meta.cc:287] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.746s	user 0.011s	sys 0.000s
I20260709 13:20:27.597940 20596 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:27.664705 15527 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:27.597532 20631 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:27.579685 20496 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:27.611666 20610 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:27.695982 20472 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:27.698436 20549 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:27.577652 20595 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:27.730645 20655 log.cc:927] Time spent T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:20:27.731215 20595 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:27.731927 20644 log.cc:927] Time spent T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
W20260709 13:20:27.660526 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 127233us
I20260709 13:20:27.659809 15567 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:27.612457 20452 consensus_queue.cc:1048] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.093s
I20260709 13:20:27.610307 20519 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:27.610569 20578 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.725200 20643 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.728693 20549 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:27.734162 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 74454.8us
I20260709 13:20:27.734158 15493 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:20:27.736352 20635 log.cc:927] Time spent T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
I20260709 13:20:27.737332 20496 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:27.776908 15493 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:27.728071 20649 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:27.710511 20572 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:27.755404 15669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2627) took 1780 ms. Trace:
I20260709 13:20:27.765336 15567 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:27.767647 20638 log.cc:927] Time spent T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
W20260709 13:20:27.768983 20650 log.cc:927] Time spent T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:20:27.770330 20510 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:20:27.771077 20659 log.cc:927] Time spent T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:20:27.773262 20578 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:27.723917 20631 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:27.774650 20645 log.cc:927] Time spent T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:20:27.775187 20448 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:27.737666 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:27.781399 15567 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:27.779392 15669 rpcz_store.cc:276] 0709 13:20:25.975467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:25.976110 (+   643us) service_pool.cc:224] Handling call
0709 13:20:25.989949 (+ 13839us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:26.212494 (+222545us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:26.212526 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:26.217153 (+  4627us) log.cc:844] Serialized 26 byte log entry
0709 13:20:26.821137 (+603984us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:26.821185 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:27.754918 (+933733us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b5400186d54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:27.754980 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:27.755039 (+    59us) raft_consensus.cc:1685] finished
0709 13:20:27.755054 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:27.755324 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6825,"raft.queue_time_us":19495,"raft.run_cpu_time_us":56,"raft.run_wall_time_us":57,"spinlock_wait_cycles":346653568,"thread_start_us":484200,"threads_started":2,"wal-append.queue_time_us":669939}
I20260709 13:20:27.753904 20595 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:27.780843 15493 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:27.786895 20649 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.787964 20585 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.788563 20586 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:27.791509 20549 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:27.791915 20489 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:27.786739 20654 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:20:27.794304 20339 consensus_meta.cc:287] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.540s	user 0.004s	sys 0.005s
I20260709 13:20:27.786753 20568 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.795836 20465 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.796295 20643 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:27.809510 15493 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:27.765956 20514 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:27.779379 20622 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:27.784685 20616 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:27.831063 20651 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:27.832582 20631 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:27.842094 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 133497us
I20260709 13:20:27.865214 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:27.824522 20342 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:27.825701 20472 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.858474 15566 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:27.860256 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 50830.4us
I20260709 13:20:27.875887 15493 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:27.758927 20596 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.868453 15674 raft_consensus.cc:1275] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:20:27.865458 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:27.812047 20578 raft_consensus.cc:2804] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:20:27.760811 20608 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:27.872394 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:27.867806 20448 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:27.871160 20342 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:27.877161 15493 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:27.894021 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:27.894367 15677 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:27.840245 20339 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:27.825382 20585 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.873904 20586 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:27.915699 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:27.911190 15668 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:27.894906 20622 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:27.906761 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:27.917613 15673 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:20:27.913164 15493 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:27.920135 15528 raft_consensus.cc:1240] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:20:27.863190 20653 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:27.923669 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:27.925189 15493 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:27.928627 15641 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:27.932983 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:27.893389 20643 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:27.813355 20572 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:27.946622 20546 consensus_meta.cc:287] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.553s	user 0.011s	sys 0.001s
I20260709 13:20:27.947139 20546 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:27.948402 20572 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:27.897527 20489 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:27.939457 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:27.950223 15603 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:20:27.950675 20568 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:27.951399 20568 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:27.951792 20568 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:27.952157 20568 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:27.942382 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:27.942698 15522 raft_consensus.cc:2368] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13.
I20260709 13:20:27.919286 20631 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:27.949339 20489 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.958005 20649 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:27.958747 20489 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:27.912048 20342 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:27.961634 15677 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:20:27.905498 20578 raft_consensus.cc:697] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:27.963056 15568 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:27.944960 20553 consensus_queue.cc:1048] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:27.938841 20660 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.875114 20651 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.943944 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:27.968219 15599 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:20:27.970152 15568 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:27.879711 20472 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.976807 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:27.977049 20578 consensus_queue.cc:237] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.978240 15603 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:27.978824 20643 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:27.979133 15599 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:20:27.980345 20660 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.981644 15525 raft_consensus.cc:1275] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:20:27.981823 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:27.982438 15527 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:27.983180 16233 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:27.943113 20481 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:27.987321 20481 raft_consensus.cc:697] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:27.988229 20481 consensus_queue.cc:237] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.983238 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:28.029135 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:28.029901 16240 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:27.911834 20448 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:27.949362 20546 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:28.051141 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 137106us
I20260709 13:20:28.054479 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:28.054505 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:28.027684 20550 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.275s	user 0.028s	sys 0.025s
I20260709 13:20:28.061769 15673 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
W20260709 13:20:28.266420 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 296353us
I20260709 13:20:28.183416 16233 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:28.268613 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:28.030053 20595 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:20:28.063360 20550 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.310s	user 0.028s	sys 0.025s
I20260709 13:20:28.271797 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:28.271708 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:28.033155 20649 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:28.291437 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:20:28.283792 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 355244us
I20260709 13:20:28.286744 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:28.150346 20622 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.297111 16239 raft_consensus.cc:2393] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 11: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:28.264159 20465 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:28.273594 20596 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:28.281332 20339 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:28.296815 20551 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.548s	user 0.009s	sys 0.007s
I20260709 13:20:28.306414 15568 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:28.149464 20342 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:28.292296 20660 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:28.322784 20671 log.cc:927] Time spent T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:20:28.314857 20677 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.340219 20342 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:20:28.303329 20546 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:28.340571 20677 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:28.341320 20546 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:28.343200 20252 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:28.301726 20622 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:28.349794 20448 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:28.352161 20519 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:28.361166 20448 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:28.361742 20448 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:28.362360 15597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53324 (request call id 2097) took 2140 ms (client timeout 1576 ms). Trace:
W20260709 13:20:28.362820 15597 rpcz_store.cc:269] 0709 13:20:26.222312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.222439 (+   127us) service_pool.cc:224] Handling call
0709 13:20:28.362121 (+2139682us) spinlock_profiling.cc:243] Waited 654 ms on lock 0x7b540043c954. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:28.362241 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1503848704}
I20260709 13:20:28.365938 15493 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:28.366789 20579 ts_tablet_manager.cc:1929] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.5
I20260709 13:20:28.367329 20579 log.cc:1199] T bd36c2cfd99c45cea9e48096d54bed1a P 3ef186ec12554a1d87814af0a0ace9a4: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-2-root/wals/bd36c2cfd99c45cea9e48096d54bed1a
I20260709 13:20:28.367484 15670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49100 (request call id 2637) took 1105 ms. Trace:
I20260709 13:20:28.367861 15670 rpcz_store.cc:276] 0709 13:20:27.262457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.266750 (+  4293us) service_pool.cc:224] Handling call
0709 13:20:28.367267 (+1100517us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b5400186854. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:28.367429 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":917844864}
I20260709 13:20:28.369390 20579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49088 (request call id 91) took 3244 ms. Trace:
I20260709 13:20:28.336570 20651 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:28.332171 20609 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:28.395272 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:28.366706 20339 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:28.395855 20339 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:28.396376 20339 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:28.398458 16234 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:28.399550 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:28.400804 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:28.401968 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:28.373845 20604 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:28.402688 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:28.402829 20604 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:28.404337 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:28.397881 16236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53976 (request call id 1062) took 1735 ms. Trace:
I20260709 13:20:28.404824 20339 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:28.345734 20622 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:28.316131 20596 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:28.343446 20546 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:28.378757 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:28.442251 16231 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:20:28.379828 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:28.443321 16239 raft_consensus.cc:2393] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 16: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:28.380849 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:28.444701 20351 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:28.445263 20351 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:28.375365 20448 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:28.381865 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:28.297613 20649 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:28.387895 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:28.457295 15642 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:28.388859 20608 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:28.390172 20510 consensus_meta.cc:287] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.572s	user 0.003s	sys 0.007s
W20260709 13:20:28.392483 20647 log.cc:927] Time spent T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.700s	user 0.001s	sys 0.000s
I20260709 13:20:28.393530 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 2210) took 1046 ms. Trace:
I20260709 13:20:28.395918 15602 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:20:28.462028 15602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53324 (request call id 2109) took 1106 ms. Trace:
I20260709 13:20:28.462386 15602 rpcz_store.cc:276] 0709 13:20:27.356216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.395008 (+1038792us) service_pool.cc:224] Handling call
0709 13:20:28.461959 (+ 66951us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:28.402231 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:28.463413 15597 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:20:28.374429 15493 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:28.461156 15520 rpcz_store.cc:276] 0709 13:20:27.346618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.347058 (+   440us) service_pool.cc:224] Handling call
0709 13:20:28.393308 (+1046250us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b5400023d54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:28.393468 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":355350272}
I20260709 13:20:28.463405 15491 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:28.465890 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:28.467097 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:28.467720 15520 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:20:28.419255 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:28.482359 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:28.482425 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:28.483641 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 109293us
I20260709 13:20:28.484390 15493 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:28.426093 20670 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:28.427238 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:28.406903 20051 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:28.428148 20604 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:28.512650 15601 raft_consensus.cc:2393] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 14: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:28.429126 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:28.429606 20472 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:28.307368 20551 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.563s	user 0.009s	sys 0.007s
I20260709 13:20:28.430450 20432 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:28.527498 20351 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:28.528667 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:28.529047 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:28.430666 20654 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:28.530146 20654 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:28.530587 20654 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:28.420660 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:28.531185 15527 raft_consensus.cc:3060] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:28.501895 20527 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:28.430814 20677 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:28.398226 20579 rpcz_store.cc:276] 0709 13:20:25.190158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:25.239450 (+ 49292us) service_pool.cc:224] Handling call
0709 13:20:25.637192 (+397742us) ts_tablet_manager.cc:1151] Deleting tablet bd36c2cfd99c45cea9e48096d54bed1a
0709 13:20:25.637287 (+    95us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:20:27.164122 (+1526835us) tablet_metadata.cc:665] Metadata flushed
0709 13:20:28.366652 (+1202530us) tablet_metadata.cc:665] Metadata flushed
0709 13:20:28.369257 (+  2605us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":338296,"tablet-delete.queue_time_us":276158,"thread_start_us":197773,"threads_started":1}
I20260709 13:20:28.423060 20568 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:28.520390 20321 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 13:20:28.486893 20546 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:28.534372 20546 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:28.534605 20568 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:28.296132 20550 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.535606 20610 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:28.536031 20610 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:28.405619 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:28.405282 16236 rpcz_store.cc:276] 0709 13:20:26.662152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.672592 (+ 10440us) service_pool.cc:224] Handling call
0709 13:20:26.701365 (+ 28773us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:28.063417 (+1362052us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:28.063450 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:28.063941 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 13:20:28.291021 (+227080us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:28.291051 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.397490 (+106439us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54006405d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:28.397537 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:28.397581 (+    44us) raft_consensus.cc:1685] finished
0709 13:20:28.397593 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:28.397823 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":146616,"raft.queue_time_us":4728,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":21007,"spinlock_wait_cycles":3122926464,"thread_start_us":130855,"threads_started":1,"wal-append.queue_time_us":176502}
I20260709 13:20:28.440883 20662 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:28.441570 20609 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:28.435554 20643 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.451889 20585 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:28.458906 20496 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:28.536958 20609 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:28.537160 20643 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:28.381626 15595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53310 (request call id 2399) took 1411 ms. Trace:
I20260709 13:20:28.537319 20496 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:28.538070 15595 rpcz_store.cc:276] 0709 13:20:26.970310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.970478 (+   168us) service_pool.cc:224] Handling call
0709 13:20:28.381459 (+1410981us) spinlock_profiling.cc:243] Waited 598 ms on lock 0x7b540009e554. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:28.381568 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1374380416}
I20260709 13:20:28.490221 15671 raft_consensus.cc:2410] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:20:28.500347 15642 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:28.539268 20643 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:28.502417 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:28.539659 20339 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:28.509968 20628 consensus_queue.cc:1048] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:28.512161 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:28.540576 20585 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:28.540983 20585 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:28.540715 15520 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:28.541316 20585 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:28.513062 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:28.513657 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:28.539665 15642 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:28.544479 16204 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:28.544175 15600 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:20:28.536393 20610 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:28.540097 15668 raft_consensus.cc:1240] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:28.535377 20550 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:28.545570 15642 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:28.554036 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:28.503147 20627 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:28.554296 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:28.558972 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:28.559744 15674 raft_consensus.cc:2393] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 14: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:28.560598 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:28.561292 15676 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:20:28.506681 15567 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:28.563205 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 56620us
I20260709 13:20:28.566753 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:28.572780 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2240) took 1438 ms. Trace:
I20260709 13:20:28.574333 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:20:28.575915 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 55261.8us
I20260709 13:20:28.576756 15674 raft_consensus.cc:1240] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:28.574626 15523 rpcz_store.cc:276] 0709 13:20:27.142666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.143181 (+   515us) service_pool.cc:224] Handling call
0709 13:20:27.162545 (+ 19364us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:27.176581 (+ 14036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:27.176613 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:27.214096 (+ 37483us) log.cc:844] Serialized 52 byte log entry
0709 13:20:27.329215 (+115119us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:27.360183 (+ 30968us) log.cc:844] Serialized 14 byte log entry
0709 13:20:28.273134 (+912951us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b4800002958. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce9934aa1 00007f5ce992bb37 00007f5ce992b9ca 00007f5ce9a2326f 00007f5ce9a22f37 00007f5ce9a3755c 00007f5ce9a37508 00007f5ce9a37488 00007f5ce9a37443 00007f5ce9a366ec 00007f5ce8960a96
0709 13:20:28.358610 (+ 85476us) spinlock_profiling.cc:243] Waited 4.08 ms on lock 0x7b180031ee20. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191b455 00007f5ce4a03060 00007f5ce4a15716 00007f5ce4a26287 00007f5ce4a26208
0709 13:20:28.572407 (+213797us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.572495 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:28.572544 (+    49us) raft_consensus.cc:1685] finished
0709 13:20:28.572559 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:28.572710 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10668,"prepare.queue_time_us":30626,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":26541,"raft.queue_time_us":233526,"raft.run_cpu_time_us":2233,"raft.run_wall_time_us":1086380,"spinlock_wait_cycles":15596800,"thread_start_us":301641,"threads_started":2,"wal-append.queue_time_us":427071}
I20260709 13:20:28.575924 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:28.580655 15670 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:28.584718 15670 raft_consensus.cc:1240] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:28.587108 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:28.587952 15676 raft_consensus.cc:2393] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 14: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:28.598788 15527 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 7.
I20260709 13:20:28.540031 20496 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:28.600983 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:28.605052 15641 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:28.613282 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:28.613874 15641 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:28.503515 20583 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:20:28.616673 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:28.607053 15567 leader_election.cc:400] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 11. Current term is 12.
I20260709 13:20:28.617828 15601 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
I20260709 13:20:28.611315 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:28.612666 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:28.605469 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:28.550896 20546 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:28.550551 20448 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:28.616073 15600 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 11: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:28.555954 20321 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:28.550639 20527 raft_consensus.cc:697] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:28.626194 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:28.498937 20596 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:28.610239 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:28.619436 15676 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:20:28.620077 15596 raft_consensus.cc:2393] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 14: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:28.619539 15598 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:20:28.618408 15567 leader_election.cc:403] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:28.600939 20604 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:28.624356 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2248) took 1455 ms. Trace:
I20260709 13:20:28.649861 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2254) took 1429 ms. Trace:
I20260709 13:20:28.536865 20662 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:28.601440 20628 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:28.514845 20670 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:28.652433 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 45521.1us
I20260709 13:20:28.652369 15566 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:28.650321 15522 rpcz_store.cc:276] 0709 13:20:27.220798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.223725 (+  2927us) service_pool.cc:224] Handling call
0709 13:20:28.649656 (+1425931us) spinlock_profiling.cc:243] Waited 714 ms on lock 0x7b540029d1d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:28.649802 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1642980736}
I20260709 13:20:28.650177 15519 rpcz_store.cc:276] 0709 13:20:27.168503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.169457 (+   954us) service_pool.cc:224] Handling call
0709 13:20:27.172369 (+  2912us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:27.346870 (+174501us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:27.347119 (+   249us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:27.360297 (+ 13178us) log.cc:844] Serialized 26 byte log entry
0709 13:20:27.361680 (+  1383us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:27.361715 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:27.361778 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:28.391917 (+1030139us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:20:28.620167 (+228250us) raft_consensus.cc:1685] finished
0709 13:20:28.620191 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:28.624287 (+  4096us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":40857,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":9561,"spinlock_wait_cycles":25821440,"thread_start_us":174863,"threads_started":2,"wal-append.queue_time_us":288104}
I20260709 13:20:28.707794 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:28.627303 20321 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:28.619511 20586 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:28.627888 20496 raft_consensus.cc:2804] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:28.714107 15670 raft_consensus.cc:2393] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 14: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:20:28.708847 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 50009.3us
I20260709 13:20:28.722440 15676 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:28.707540 20643 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 11. Current term is 12.
I20260709 13:20:28.734246 20684 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.743088 20684 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:28.745436 20684 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:28.746433 20683 log.cc:927] Time spent T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.242s	user 0.000s	sys 0.000s
I20260709 13:20:28.747407 20496 raft_consensus.cc:697] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:28.748443 20496 consensus_queue.cc:237] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:28.726776 20321 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:28.750568 20609 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:28.751947 20596 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:28.753368 20527 consensus_queue.cc:237] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:28.755110 20670 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:28.756089 20649 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:28.658527 20690 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:28.756939 20690 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:28.759857 20550 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:28.735929 15522 raft_consensus.cc:1240] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:20:28.763455 15522 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:20:28.765662 15522 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:20:28.733311 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 78416.8us
I20260709 13:20:28.656644 20628 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:28.773372 20628 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:28.778895 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:28.779743 15602 raft_consensus.cc:2393] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 10: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:28.734537 20342 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:28.772572 19895 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:28.779466 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:28.761322 20546 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:28.782712 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:28.789884 16236 raft_consensus.cc:2410] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:20:28.786710 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:28.790699 15525 raft_consensus.cc:2393] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 14: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:28.790823 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:28.788409 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:28.791656 20690 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:28.794982 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:28.795760 15603 raft_consensus.cc:2393] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 12: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:28.798024 20342 raft_consensus.cc:697] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:28.799213 16205 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:28.799044 20342 consensus_queue.cc:237] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:28.800007 20569 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:28.800420 16205 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:28.770442 15566 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:28.802030 15594 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:20:28.728127 20586 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:28.815676 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:28.816471 15525 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:28.818097 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:28.819607 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:28.836882 15599 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:20:28.839071 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 54436.2us
I20260709 13:20:28.801234 20051 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:20:28.848022 20661 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:28.839929 20549 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:28.833513 20586 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:28.877316 15670 raft_consensus.cc:1240] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 15. Current term is 16. Ops: []
W20260709 13:20:28.886679 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 97185.4us
W20260709 13:20:28.889173 20697 log.cc:927] Time spent T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:20:28.833228 20472 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:28.889686 20472 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:28.890120 20472 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:28.896170 15601 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:20:28.904541 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 134168us
I20260709 13:20:28.898043 20546 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:28.905992 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:28.898427 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:28.909314 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:28.789713 20576 consensus_queue.cc:1059] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.098s
W20260709 13:20:28.905313 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 104963us
I20260709 13:20:28.915798 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:28.921547 15519 raft_consensus.cc:2410] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:20:28.915932 15641 leader_election.cc:400] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13.
I20260709 13:20:28.923099 15641 leader_election.cc:403] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:28.916463 15525 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:28.924427 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:28.925110 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:28.925873 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:28.928453 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:28.928946 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:28.929879 15522 raft_consensus.cc:2393] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 14: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:28.935683 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:28.934604 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 312748us
I20260709 13:20:28.932969 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:28.937830 16204 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:28.938547 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:28.939057 15525 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 13: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:28.941668 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:28.931499 15566 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:28.943645 15642 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:28.942791 15522 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:20:28.945982 20481 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:28.944597 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:28.943957 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:28.972675 15519 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:28.951948 16233 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:20:28.930274 20546 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13.
I20260709 13:20:28.967350 15642 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:28.946378 15674 raft_consensus.cc:1240] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:28.975265 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:28.934211 20690 consensus_queue.cc:1059] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:20:28.976810 15642 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:28.930238 20585 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:28.972846 15493 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:28.978721 15566 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:28.952121 20467 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:28.958400 20576 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:28.952000 20577 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:28.994167 15642 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:28.993268 15641 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:28.952857 20631 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:28.994585 15567 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:29.001622 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:28.969391 20688 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:28.976110 20546 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:28.971936 20649 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:29.009542 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:28.971546 20496 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:28.977109 20702 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.979628 20550 consensus_queue.cc:1059] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:20:28.998049 20670 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:29.022800 20583 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:29.002974 20703 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:29.003005 20631 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:28.985852 20628 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:29.024329 20552 raft_consensus.cc:3055] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:28.981187 20481 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:29.158691 15566 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.669s (SENT) [suppressed 4 similar messages]
I20260709 13:20:29.003064 20568 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:28.943100 20654 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:28.998242 20693 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:29.027082 20651 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:29.039873 15671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49100 (request call id 2638) took 1777 ms (client timeout 1669 ms). Trace:
I20260709 13:20:28.994457 20467 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.033847 20472 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:29.112965 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 111415us
I20260709 13:20:29.210619 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:29.002727 20576 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:29.039826 20672 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:20:29.112993 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 118474us
I20260709 13:20:29.264453 15567 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:29.113374 20321 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:29.163306 20610 consensus_meta.cc:287] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.608s	user 0.010s	sys 0.000s
I20260709 13:20:29.253216 15642 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:29.125334 20583 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:29.158726 20552 raft_consensus.cc:740] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:29.276674 20568 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:29.313504 20552 consensus_queue.cc:260] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:29.313838 20610 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:29.314591 20552 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:29.314947 15524 raft_consensus.cc:1240] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:29.316898 20610 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:29.317534 20610 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:29.277870 20661 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.318481 20568 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:29.320971 20568 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:29.321694 20610 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:29.322865 20481 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:29.278309 20550 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:29.324060 20610 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:29.265873 15567 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:29.305363 15520 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.052s	sys 0.114s
W20260709 13:20:29.190303 20675 log.cc:927] Time spent T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:20:29.306592 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 53458.7us
I20260709 13:20:29.025017 20688 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:29.115410 20649 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:29.331534 20688 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:29.249166 20467 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:29.332154 20649 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:29.124780 20339 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.085s
I20260709 13:20:29.334507 20585 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:29.123483 20702 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:29.327207 15520 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.052s	sys 0.114s
I20260709 13:20:29.194434 20662 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:29.336812 20702 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:29.332093 20583 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:29.337998 20631 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:29.334797 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:29.161692 20703 raft_consensus.cc:697] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:29.340051 20673 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:29.340554 20673 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:29.340943 20673 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:29.342705 16235 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:29.353176 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:29.359241 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:29.365967 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:29.366215 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:29.367727 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:29.370664 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:29.252610 20693 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.127118 20586 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:29.393370 15642 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:29.277791 20496 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:20:29.223114 15671 rpcz_store.cc:269] 0709 13:20:27.262724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.265210 (+  2486us) service_pool.cc:224] Handling call
0709 13:20:29.039670 (+1774460us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b540026c0d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:29.039810 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2806362880}
I20260709 13:20:29.398576 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:29.381458 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 115662us
I20260709 13:20:29.415390 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:29.415519 15567 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:29.403576 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:29.408691 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:29.416669 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:29.417119 16231 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:29.395664 15642 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:29.277426 20546 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:29.308754 20612 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.419322 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:29.422608 15668 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 4.
I20260709 13:20:29.422827 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2244) took 2260 ms. Trace:
I20260709 13:20:29.414860 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:29.421814 15670 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:20:29.423280 15528 rpcz_store.cc:276] 0709 13:20:27.150508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.154183 (+  3675us) service_pool.cc:224] Handling call
0709 13:20:27.154499 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:27.921884 (+767385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.410678 (+1488794us) spinlock_profiling.cc:243] Waited 765 ms on lock 0x7b54003d9fd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:29.410902 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1760253696}
I20260709 13:20:29.321023 20321 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:29.422849 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:29.431131 15410 catalog_manager.cc:5028] TS 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): tablet bd36c2cfd99c45cea9e48096d54bed1a (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:20:29.432327 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:29.419966 20664 consensus_queue.cc:1059] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:20:29.430400 15567 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:29.441875 15528 raft_consensus.cc:1240] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:29.386087 20667 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.448024 15642 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:29.400453 20693 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:29.396472 20568 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:29.318428 20661 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:29.470359 15602 raft_consensus.cc:1240] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10. Ops: []
W20260709 13:20:29.471987 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 459246us
I20260709 13:20:29.429610 20510 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:29.450579 16231 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:20:29.451042 15567 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:29.474860 15567 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:29.435112 20612 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:29.459266 20604 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:29.461709 20703 consensus_queue.cc:237] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:29.463053 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:29.490568 15596 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:20:29.419713 20672 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:29.466941 20654 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:29.467320 20667 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:29.468930 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:29.449484 20550 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:29.485368 15567 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:29.464821 20467 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:29.491585 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:29.496693 15675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 1578) took 1023 ms. Trace:
I20260709 13:20:29.458065 20595 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:29.428901 20546 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:29.509654 15602 raft_consensus.cc:2393] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 10: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:29.471450 20632 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.515321 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:29.509395 15675 rpcz_store.cc:276] 0709 13:20:28.473633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.473772 (+   139us) service_pool.cc:224] Handling call
0709 13:20:28.474091 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:28.728313 (+254222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:28.728346 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:28.728388 (+    42us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:20:28.728420 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.496429 (+768009us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b540042bb54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:29.496482 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:29.496621 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":584211840}
I20260709 13:20:29.489688 20432 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:29.497578 20651 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:29.525444 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:29.527860 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:29.499258 20568 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:29.646559 15491 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:29.647095 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:29.658047 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:29.658808 15596 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:29.522233 20551 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:29.523113 20684 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.526772 20651 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.675055 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:29.676915 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:29.680564 15602 raft_consensus.cc:1240] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:29.697948 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:29.524497 20632 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:29.706743 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 548061us
I20260709 13:20:29.708529 15675 raft_consensus.cc:1240] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10. Ops: []
W20260709 13:20:29.709992 15676 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.004s	sys 0.027s
I20260709 13:20:29.710346 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:29.711026 15668 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:29.709851 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:20:29.711884 15676 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.004s	sys 0.027s
I20260709 13:20:29.678957 20651 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:29.720885 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:20:29.729024 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:29.729883 15493 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:29.730968 15641 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:29.731755 20342 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.732344 20342 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } attrs { replace: true } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:29.735378 20342 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:29.737280 20702 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:29.737784 20702 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:29.738201 20702 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:29.737133 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:29.739454 15671 raft_consensus.cc:1813] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Handling vote request from an unknown peer 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:29.740074 15671 raft_consensus.cc:2410] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:20:29.740845 20604 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:20:29.795943 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 65198.6us
I20260709 13:20:29.796573 15567 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:29.799643 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:29.706617 20713 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:29.802027 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:20:29.811005 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 164518us
I20260709 13:20:29.812484 15491 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:29.813747 15491 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:29.700050 20684 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:29.817103 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:29.819988 15491 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:29.742065 20631 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:29.727028 20576 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:29.822551 16204 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:29.803462 20342 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:29.845777 15595 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:29.846350 15671 raft_consensus.cc:1240] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:29.823133 20610 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:29.822575 20622 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:29.795238 20510 consensus_queue.cc:1059] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:20:29.823482 20586 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:29.863721 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:29.829504 20631 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:29.873936 15671 raft_consensus.cc:1240] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:29.856001 20713 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:29.826732 20711 consensus_queue.cc:1059] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:20:29.865542 20688 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:29.870689 20631 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:29.877496 20715 log.cc:927] Time spent T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:20:29.889024 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:29.869848 20661 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:29.900341 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 2236) took 1009 ms. Trace:
I20260709 13:20:29.902155 15671 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 13: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:29.872164 20612 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:29.882231 20546 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:29.894292 20632 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:29.877936 20467 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:29.895352 20704 log.cc:927] Time spent T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 13:20:29.906263 15668 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:29.900781 15523 rpcz_store.cc:276] 0709 13:20:28.891084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.980321 (+ 89237us) service_pool.cc:224] Handling call
0709 13:20:29.007638 (+ 27317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:29.008003 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:29.008034 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:29.124582 (+116548us) log.cc:844] Serialized 26 byte log entry
0709 13:20:29.515882 (+391300us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:29.515921 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.515980 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:29.900138 (+384158us) raft_consensus.cc:1685] finished
0709 13:20:29.900156 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:29.900282 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83301,"raft.queue_time_us":2975,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":23671,"thread_start_us":326861,"threads_started":1,"wal-append.queue_time_us":528590}
I20260709 13:20:29.910600 20634 raft_consensus.cc:3055] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:20:29.907094 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 87197.8us
I20260709 13:20:29.882405 20667 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:29.926749 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:29.927686 15523 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 13: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:20:29.929272 20714 log.cc:927] Time spent T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:20:29.918493 20465 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:29.926973 15491 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:29.931021 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 2232) took 1040 ms. Trace:
I20260709 13:20:29.931386 15522 rpcz_store.cc:276] 0709 13:20:28.890110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.978839 (+ 88729us) service_pool.cc:224] Handling call
0709 13:20:28.979639 (+   800us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:28.979856 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:28.979930 (+    74us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:28.998684 (+ 18754us) log.cc:844] Serialized 26 byte log entry
0709 13:20:29.532378 (+533694us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:29.532423 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.532497 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:29.930777 (+398280us) raft_consensus.cc:1685] finished
0709 13:20:29.930799 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:29.930959 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":390477,"threads_started":1,"wal-append.queue_time_us":785279}
I20260709 13:20:29.932915 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:29.934271 15673 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
I20260709 13:20:29.934687 20351 raft_consensus.cc:3055] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:29.935056 20351 raft_consensus.cc:740] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:29.935865 20351 consensus_queue.cc:260] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:29.936990 20351 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:29.937435 20465 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:29.938421 16235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57244 (request call id 1483) took 1385 ms. Trace:
I20260709 13:20:29.938771 16235 rpcz_store.cc:276] 0709 13:20:28.553453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.553568 (+   115us) service_pool.cc:224] Handling call
0709 13:20:29.938259 (+1384691us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b540051c554. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:29.938365 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1628402048}
I20260709 13:20:29.939672 20465 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:29.942668 20472 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:29.942879 20728 log.cc:927] Time spent T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:20:29.944181 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:29.944356 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1705) took 1035 ms. Trace:
I20260709 13:20:29.944803 15528 rpcz_store.cc:276] 0709 13:20:28.908698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.442984 (+534286us) service_pool.cc:224] Handling call
0709 13:20:29.443385 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:29.443753 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:29.443779 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:29.444246 (+   467us) log.cc:844] Serialized 26 byte log entry
0709 13:20:29.730505 (+286259us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:29.730541 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.730605 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:29.944103 (+213498us) raft_consensus.cc:1685] finished
0709 13:20:29.944124 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:29.944293 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":5784,"thread_start_us":245429,"threads_started":1,"wal-append.queue_time_us":380519}
W20260709 13:20:29.949728 20716 log.cc:927] Time spent T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:20:29.916855 20632 raft_consensus.cc:3055] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:29.955731 20632 raft_consensus.cc:740] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:29.956737 20632 consensus_queue.cc:260] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:29.951598 20634 raft_consensus.cc:740] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:29.958217 20634 consensus_queue.cc:260] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:29.959281 20634 raft_consensus.cc:3060] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:29.959919 20684 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:29.964563 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:29.900547 20688 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:29.968094 15525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 2231) took 1078 ms. Trace:
I20260709 13:20:29.975452 20342 consensus_queue.cc:1059] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:20:30.036072 15525 rpcz_store.cc:276] 0709 13:20:28.889830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.959464 (+ 69634us) service_pool.cc:224] Handling call
0709 13:20:28.959940 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.257723 (+297783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:29.257759 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:29.257806 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:29.257839 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.967858 (+710019us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b54003cdcd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:29.967908 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:29.968038 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":684243712}
I20260709 13:20:29.966207 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1684) took 1975 ms. Trace:
W20260709 13:20:29.990343 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:29.975734 20690 consensus_queue.cc:1059] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.064s
W20260709 13:20:29.982986 20583 consensus_meta.cc:287] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.645s	user 0.009s	sys 0.000s
I20260709 13:20:29.989315 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:29.981537 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 2234) took 1091 ms. Trace:
I20260709 13:20:29.982260 15599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 1420) took 1186 ms. Trace:
I20260709 13:20:30.072384 15599 rpcz_store.cc:276] 0709 13:20:28.795852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.851601 (+ 55749us) service_pool.cc:224] Handling call
0709 13:20:28.888500 (+ 36899us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.433587 (+545087us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:20:29.712910 (+279323us) log.cc:844] Serialized 14 byte log entry
0709 13:20:29.868905 (+155995us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:29.868964 (+    59us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:29.869017 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.982044 (+113027us) spinlock_profiling.cc:243] Waited 545 ms on lock 0x7b5400439754. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:29.982088 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:29.982206 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":519447,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":228,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":6481,"spinlock_wait_cycles":1253090560,"thread_start_us":420892,"threads_started":2}
W20260709 13:20:30.075197 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 148319us
I20260709 13:20:30.034435 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:20:29.964994 20673 consensus_meta.cc:287] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.624s	user 0.006s	sys 0.002s
W20260709 13:20:30.061789 15642 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.010s	sys 0.013s
I20260709 13:20:30.003057 20551 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.965709 15568 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:29.898680 20693 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:29.995020 20719 log.cc:927] Time spent T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.168s	user 0.002s	sys 0.000s
I20260709 13:20:30.038645 15520 rpcz_store.cc:276] 0709 13:20:27.990495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.541866 (+551371us) service_pool.cc:224] Handling call
0709 13:20:28.542176 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:28.542463 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:28.542495 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:28.542969 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 13:20:29.335487 (+792518us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:29.335526 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.335593 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:29.965965 (+630372us) raft_consensus.cc:1685] finished
0709 13:20:29.965989 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:29.966128 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":58223,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":12838,"thread_start_us":792148,"threads_started":1,"wal-append.queue_time_us":982217}
I20260709 13:20:30.076339 15668 raft_consensus.cc:2410] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 13:20:30.077207 15642 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.010s	sys 0.013s
W20260709 13:20:30.098062 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 650106us
I20260709 13:20:30.064749 15523 raft_consensus.cc:1275] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:20:30.077509 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:30.101935 16240 raft_consensus.cc:1813] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Handling vote request from an unknown peer 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:29.988862 20585 raft_consensus.cc:3055] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:30.069609 15519 rpcz_store.cc:276] 0709 13:20:28.890592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.973595 (+ 83003us) service_pool.cc:224] Handling call
0709 13:20:28.973969 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.333802 (+359833us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:20:29.334124 (+   322us) log.cc:844] Serialized 14 byte log entry
0709 13:20:29.645318 (+311194us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:29.645375 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:29.645424 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.981305 (+335881us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b54002ddcd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:29.981348 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:29.981473 (+   125us) inbound_call.cc:177] Queueing success response
0709 13:20:29.994405 (+ 12932us) spinlock_profiling.cc:243] Waited 8.13 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
Metrics: {"mutex_wait_us":164368,"prepare.queue_time_us":1321,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":119538,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":12619,"spinlock_wait_cycles":845709952,"thread_start_us":192792,"threads_started":1,"wal-append.queue_time_us":390762}
I20260709 13:20:30.079264 20609 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:30.061754 20632 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:30.111728 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 249192us
I20260709 13:20:29.971091 20467 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:30.080281 20673 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.085433 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:30.084057 20684 raft_consensus.cc:3055] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:30.078984 15491 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:30.123524 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:30.104240 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 138601us
I20260709 13:20:30.109951 16240 raft_consensus.cc:2410] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:20:30.109320 15566 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:30.126907 20595 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:30.127061 15568 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:30.113853 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 291395us
I20260709 13:20:30.114887 15521 raft_consensus.cc:1240] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:20:30.119386 15567 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:30.084471 20551 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:30.124349 15594 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
W20260709 13:20:30.131768 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 53382.1us
I20260709 13:20:30.134579 15491 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:30.132761 15567 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:30.138008 15599 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:20:30.138252 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:30.135812 20616 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:30.145500 20718 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:30.146430 20550 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:30.146970 20610 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:30.130999 20609 raft_consensus.cc:697] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:30.147634 20628 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.148456 20609 consensus_queue.cc:237] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:30.146142 20718 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:30.150516 15642 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:30.150797 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.153960 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:30.116214 20690 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:30.078555 20583 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.038089 20631 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.110347 20585 raft_consensus.cc:740] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:30.003558 20651 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.156927 15676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2664) took 1182 ms. Trace:
I20260709 13:20:30.131309 20568 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:30.128459 20684 raft_consensus.cc:740] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:30.167295 15642 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:30.136335 20595 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:30.128238 20496 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:30.186578 15676 rpcz_store.cc:276] 0709 13:20:28.974865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.976339 (+  1474us) service_pool.cc:224] Handling call
0709 13:20:28.976640 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:29.124032 (+147392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:29.124068 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:29.125034 (+   966us) log.cc:844] Serialized 26 byte log entry
0709 13:20:29.713357 (+588323us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:29.713401 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:30.151688 (+438287us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b540023ad54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:30.151750 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:30.151802 (+    52us) raft_consensus.cc:1685] finished
0709 13:20:30.151816 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:30.156853 (+  5037us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":338332416,"thread_start_us":523165,"threads_started":1,"wal-append.queue_time_us":687310}
I20260709 13:20:30.149449 20339 raft_consensus.cc:2804] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:20:30.135859 20546 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:30.133594 20467 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:30.143214 20709 consensus_queue.cc:1059] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.049s
W20260709 13:20:30.255474 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 66916us
I20260709 13:20:30.144352 20342 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:30.156433 20676 log.cc:927] Time spent T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 1.638s	user 0.013s	sys 0.030s
I20260709 13:20:30.262166 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:30.153051 20701 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:30.192525 20596 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:20:30.347409 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 85329.2us
I20260709 13:20:30.195385 20339 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:30.193264 20448 consensus_queue.cc:1048] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.157s
W20260709 13:20:30.350430 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 217769us
I20260709 13:20:30.359638 20608 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.018s
W20260709 13:20:30.378891 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 211689us
I20260709 13:20:30.402364 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:30.403122 15668 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:20:30.404296 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:30.405511 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:30.374713 20595 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:30.410009 20595 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.365756 20703 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:30.412559 20703 raft_consensus.cc:697] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:30.413512 20703 consensus_queue.cc:237] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.348156 20740 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:30.416801 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2275) took 1416 ms. Trace:
I20260709 13:20:30.348802 20701 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:30.417237 20740 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.360669 20585 consensus_queue.cc:260] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:30.421017 20585 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:30.195192 20568 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.427839 20568 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.371824 20342 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:30.429026 20342 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:30.430400 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 285674us
I20260709 13:20:30.360903 20684 consensus_queue.cc:260] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.197899 20496 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.338762 20546 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:30.395695 20702 consensus_meta.cc:287] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.657s	user 0.009s	sys 0.003s
I20260709 13:20:30.397234 20718 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:30.447412 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:30.359448 20627 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:30.406774 15525 raft_consensus.cc:1275] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:30.406702 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:30.407541 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:30.382422 20732 log.cc:927] Time spent T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
I20260709 13:20:30.430753 15672 raft_consensus.cc:2410] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:20:30.431155 15673 raft_consensus.cc:2393] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 10: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:30.392891 20673 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.431833 15642 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:30.417310 15522 rpcz_store.cc:276] 0709 13:20:29.000176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.946994 (+946818us) service_pool.cc:224] Handling call
0709 13:20:29.947254 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:30.069109 (+121855us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:30.069137 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:30.069177 (+    40us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:30.069211 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:30.416512 (+347301us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54003db3d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:30.416568 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:30.416730 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279787264}
I20260709 13:20:30.435057 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:30.465314 15520 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:20:30.436721 15523 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:30.357014 20339 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:30.405977 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:30.449371 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:30.448529 15568 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:30.359845 20628 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:30.461513 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:30.470451 15594 raft_consensus.cc:1240] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:30.471290 15668 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 18.
I20260709 13:20:30.436381 20578 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:30.454450 20627 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:30.442334 20672 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:30.451151 20616 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:30.471983 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2274) took 1472 ms. Trace:
I20260709 13:20:30.453197 20690 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:30.485668 15528 rpcz_store.cc:276] 0709 13:20:28.999926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.946293 (+946367us) service_pool.cc:224] Handling call
0709 13:20:29.946583 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.946822 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:20:30.349286 (+402464us) log.cc:844] Serialized 14 byte log entry
0709 13:20:30.471646 (+122360us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:30.471688 (+    42us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:20:30.471731 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:30.471806 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:30.471924 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":446591,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":917,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":6466,"thread_start_us":520556,"threads_started":2}
I20260709 13:20:30.493683 20467 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:30.499842 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1692) took 2067 ms. Trace:
I20260709 13:20:30.500385 15527 rpcz_store.cc:276] 0709 13:20:28.432910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.734009 (+301099us) service_pool.cc:224] Handling call
0709 13:20:28.787764 (+ 53755us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:28.918072 (+130308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:28.918107 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:28.932422 (+ 14315us) log.cc:844] Serialized 26 byte log entry
0709 13:20:29.958105 (+1025683us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b1800437e60. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab0d1 00007f5ce49c6a0f
0709 13:20:30.332313 (+374208us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:30.332358 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:30.332425 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:30.499450 (+167025us) raft_consensus.cc:1685] finished
0709 13:20:30.499476 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:30.499757 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":131384,"raft.queue_time_us":132798,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":7216,"spinlock_wait_cycles":3372416,"thread_start_us":374001,"threads_started":1,"wal-append.queue_time_us":1442675}
I20260709 13:20:30.482789 20750 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:30.503285 20750 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:30.503698 20750 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:30.483326 20627 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:30.461298 20702 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:30.483585 20604 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:30.483032 20578 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:30.538352 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 89919.2us
I20260709 13:20:30.484213 20586 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:20:30.522133 20685 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:30.542999 20323 raft_consensus.cc:3055] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:30.481935 20628 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.570191 20323 raft_consensus.cc:740] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:30.571163 20323 consensus_queue.cc:260] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:30.572660 20323 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:30.546033 20546 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.515952 20690 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.575560 20690 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.548348 20496 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.550635 20702 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:30.561180 15491 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:30.566399 20651 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.576871 20702 raft_consensus.cc:2804] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:30.489223 20672 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.577286 20702 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:30.577699 15595 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:20:30.577682 20702 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:30.578801 20672 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.539851 20583 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.540781 20604 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.581871 20604 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.491003 20616 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.584086 20616 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.585580 20702 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.587689 20702 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.588033 20578 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.592736 20628 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.594306 20604 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:30.594937 20604 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.597863 20604 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.607010 15491 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:20:30.607997 20351 consensus_meta.cc:287] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.671s	user 0.011s	sys 0.001s
I20260709 13:20:30.609498 20578 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:30.610845 20627 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:30.613186 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:30.617208 20643 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.012s	sys 0.036s
W20260709 13:20:30.617184 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 154389us
I20260709 13:20:30.618458 20750 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:30.618729 15567 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:30.651010 15675 raft_consensus.cc:1240] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:30.651201 20550 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:30.651942 20649 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:30.652455 20649 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:30.616868 20585 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:30.654006 20585 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.656036 20585 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.621840 20627 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.614162 20551 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.622978 20604 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:30.673902 20604 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.676116 20604 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.597613 20339 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.624439 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:20:30.625967 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 164897us
I20260709 13:20:30.677317 15668 raft_consensus.cc:2393] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 10: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:30.626318 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:30.678208 15669 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:30.678776 15566 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:30.679669 20722 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:30.628266 15673 raft_consensus.cc:1240] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:30.680922 20750 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:30.628739 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:30.681830 15677 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:30.683717 20701 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:30.628813 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:30.684458 15676 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:30.685376 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:30.685942 20595 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.686362 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:30.631628 20688 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.634276 20693 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.644310 20736 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:30.687973 20736 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.690201 20736 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.691010 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:30.692447 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:30.693138 20684 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:30.691653 15595 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:20:30.695250 15668 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:30.700824 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:30.701646 15595 raft_consensus.cc:2393] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 14: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:30.648981 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:30.661883 20649 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:30.663050 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.665957 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1706) took 1757 ms. Trace:
I20260709 13:20:30.667232 15524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1685) took 2676 ms. Trace:
W20260709 13:20:30.667884 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.671125 20342 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.673146 20631 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.655952 20610 raft_consensus.cc:3055] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:30.710749 20610 raft_consensus.cc:740] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:30.703305 15521 rpcz_store.cc:276] 0709 13:20:28.908962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.447198 (+538236us) service_pool.cc:224] Handling call
0709 13:20:29.447505 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:30.129001 (+681496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:30.665665 (+536664us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b54003cfad4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:30.665878 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1534293376}
I20260709 13:20:30.711840 20339 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.704018 15524 rpcz_store.cc:276] 0709 13:20:27.990752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.557332 (+566580us) service_pool.cc:224] Handling call
0709 13:20:28.557603 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.315345 (+757742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:30.666984 (+1351639us) spinlock_profiling.cc:243] Waited 757 ms on lock 0x7b5400263254. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:30.667171 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1741380864}
I20260709 13:20:30.711625 20610 consensus_queue.cc:260] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.712586 20610 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:30.712864 20551 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.717965 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:30.686493 20595 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:30.629885 20740 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:30.706784 15677 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:20:30.708473 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:30.720561 15594 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:20:30.720954 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 72049.6us
I20260709 13:20:30.709444 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:30.725267 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:30.725812 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:30.727391 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:30.727658 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:30.728312 15668 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:20:30.729269 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 597467us
I20260709 13:20:30.722684 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:30.732719 15675 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 10: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:30.736150 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:30.731503 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:30.735132 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:30.734175 15491 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:30.742722 16239 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:20:30.745172 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:30.743417 15491 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:30.742946 15566 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:30.744424 15642 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:30.747275 15491 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:30.754640 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:30.759341 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:30.760165 15596 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 10: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:30.760625 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:30.761356 15599 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:30.762400 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:30.762843 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:30.763238 15601 raft_consensus.cc:2393] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 15: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:30.763928 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:30.764658 15525 raft_consensus.cc:1240] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:20:30.760965 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:30.754359 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:30.776880 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:30.777467 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:30.768476 15521 raft_consensus.cc:2393] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 15: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:30.771373 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:30.778704 15525 raft_consensus.cc:2393] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 15: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:30.729171 20595 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:30.734535 20562 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:30.738608 20750 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:30.766759 15596 raft_consensus.cc:2393] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 13: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:30.788720 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:30.784484 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:30.782521 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:30.750276 20740 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:30.791404 15599 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:30.793298 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:30.802230 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:30.796203 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:30.797999 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:30.798077 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:30.738451 20604 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:30.804921 15527 raft_consensus.cc:2393] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 10: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:30.804848 16231 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:20:30.734754 20339 consensus_queue.cc:1059] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:20:30.806262 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:30.784907 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:30.809474 15668 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:20:30.810258 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:30.822386 15601 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:30.796792 15594 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 15.
I20260709 13:20:30.791538 20562 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.813508 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:30.778189 20756 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:30.835726 20562 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:30.836522 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:30.836683 20665 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:30.767546 20734 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:30.766700 20631 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:30.819252 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:30.819489 15567 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:30.775542 20608 raft_consensus.cc:2804] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:30.778358 20703 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:30.825219 15642 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:30.813511 15493 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:30.738451 20342 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:30.837378 20665 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:30.828941 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:30.777026 20586 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:30.782696 20595 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:30.845063 16238 raft_consensus.cc:1275] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:30.851428 15599 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 15.
I20260709 13:20:30.813491 15603 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:30.915041 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:30.914290 15520 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:30.852943 16236 raft_consensus.cc:1240] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:30.857252 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:30.857654 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:30.776695 20546 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:20:30.884040 15596 raft_consensus.cc:1240] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 13. Current term is 14. Ops: []
W20260709 13:20:30.923533 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 104109us
I20260709 13:20:30.924249 16234 raft_consensus.cc:1275] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:20:30.818253 20740 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:20:30.927760 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 102627us
I20260709 13:20:30.922950 20631 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:30.828094 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:30.816115 20604 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:30.800233 20750 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:30.922341 20715 log.cc:927] Time spent T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:20:30.990762 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:20:31.037283 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 223862us
W20260709 13:20:30.996546 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 62574.3us
I20260709 13:20:31.040970 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:30.930563 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:31.059445 15493 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:30.921015 20703 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:31.042568 15601 raft_consensus.cc:2393] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 15: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:31.044234 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:31.044337 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:30.843360 20756 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:30.931870 20722 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:30.935926 20551 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:31.037107 20608 raft_consensus.cc:697] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:30.964617 20568 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:31.057440 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:31.064508 15493 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:31.069895 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:31.130215 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:31.131105 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 66684us
I20260709 13:20:31.131484 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:31.042295 20740 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:31.067939 15520 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:31.054806 20604 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:31.068742 20425 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:20:31.073212 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:31.068673 20452 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:31.044536 20649 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:31.086282 20489 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:31.038070 20565 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:31.068907 20702 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:31.092034 20750 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:31.093861 20665 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:31.063634 20550 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:31.054847 20747 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:31.134853 20578 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:31.093950 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:31.190367 15600 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:30.994040 20677 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:31.194218 20677 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:31.196425 20677 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:31.102603 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:31.045535 20631 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:31.103550 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:31.197765 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:31.198915 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:31.186900 20660 consensus_queue.cc:1059] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:20:31.200030 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:31.201087 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:31.042198 15491 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:31.120270 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:31.123088 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 60056.6us
I20260709 13:20:31.202806 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:31.203375 16236 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:31.123934 16204 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:30.934060 20734 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:31.207823 20734 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:31.208447 20747 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:31.147662 20350 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:20:31.150578 20565 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:31.210435 16240 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 15.
I20260709 13:20:31.210754 20747 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:31.141749 20568 raft_consensus.cc:697] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:30.963815 20342 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:31.039567 20586 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:31.140446 16232 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:31.071955 20703 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:31.146615 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:31.220680 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:31.132853 20756 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:31.099722 20026 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:31.150566 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:31.201617 15568 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:31.189499 20546 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:31.226155 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 102252us
I20260709 13:20:31.226984 20586 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:31.198920 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:31.192317 20550 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:31.131896 15493 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:20:31.234555 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 102746us
I20260709 13:20:31.235383 15493 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
W20260709 13:20:31.213366 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 171240us
I20260709 13:20:31.213371 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:31.214044 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:31.129940 20722 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:31.219122 20627 raft_consensus.cc:2804] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:20:31.150008 20702 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:31.150420 20489 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:31.212913 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:31.236436 15671 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:31.148075 20452 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:31.246255 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:31.236896 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:31.237109 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.228607 15642 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:31.221834 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:31.239012 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:31.241084 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:31.231559 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:31.242702 20702 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:31.293292 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:31.294304 16235 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:31.294658 16234 raft_consensus.cc:1275] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:20:31.296037 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 67537.4us
I20260709 13:20:31.247056 15566 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:31.296535 15601 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
W20260709 13:20:31.297314 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 58418.3us
I20260709 13:20:31.307966 15670 raft_consensus.cc:1240] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:31.239315 20565 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:31.232455 20432 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:31.228720 20568 consensus_queue.cc:237] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:31.366048 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 119068us
W20260709 13:20:31.367148 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 45312.6us
W20260709 13:20:31.381716 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.379586 16236 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:31.246301 20627 raft_consensus.cc:697] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
W20260709 13:20:31.429140 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 197931us
I20260709 13:20:31.240568 20578 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:20:31.247386 20765 log.cc:927] Time spent T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:20:31.292405 20448 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:31.403594 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 176030us
I20260709 13:20:31.241490 20649 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:31.293303 20701 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.366858 20622 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:31.364360 20632 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:31.240042 20576 consensus_queue.cc:1059] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.132s
I20260709 13:20:31.295547 20750 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:31.364907 20749 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:31.233628 20546 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:31.238464 20608 consensus_queue.cc:237] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:31.480114 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 49889.9us
I20260709 13:20:31.404985 20594 raft_consensus.cc:2764] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:20:31.434522 20578 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:31.376852 20450 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:31.431811 20677 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.438268 20577 consensus_queue.cc:1059] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:20:31.376570 20693 consensus_queue.cc:1059] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.100s
I20260709 13:20:31.487625 16203 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:31.491593 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 51572.2us
I20260709 13:20:31.480576 20749 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:31.561417 15671 raft_consensus.cc:1275] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:20:31.487053 20736 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:31.486635 20632 raft_consensus.cc:697] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:31.440871 20448 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:31.527201 20677 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:31.523293 20578 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:31.475210 20701 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:31.493474 20450 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:31.482107 20750 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:31.572862 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 85324.4us
W20260709 13:20:31.570983 20595 consensus_meta.cc:287] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.530s	user 0.007s	sys 0.003s
I20260709 13:20:31.577713 20595 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:31.578466 20677 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:20:31.579841 20595 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:31.580191 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:31.580827 15669 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:20:31.581104 15670 raft_consensus.cc:1240] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:31.563589 20749 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:31.582818 15566 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:20:31.569123 20684 consensus_meta.cc:287] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.873s	user 0.004s	sys 0.006s
I20260709 13:20:31.587116 15642 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:31.587723 15676 raft_consensus.cc:1240] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:20:31.588980 15642 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:31.590636 15597 raft_consensus.cc:1275] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:20:31.583554 15677 raft_consensus.cc:1240] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:31.583426 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:31.585577 15673 raft_consensus.cc:1240] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:31.591720 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:31.589246 20586 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:31.591925 15597 pending_rounds.cc:85] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting all ops after (but not including) 10
I20260709 13:20:31.591952 15670 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:31.592387 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:31.593159 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:31.593211 15597 pending_rounds.cc:107] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:20:31.594570 15597 raft_consensus.cc:2889] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:31.582069 20448 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:31.593581 15674 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:31.572703 20342 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:31.598451 15674 raft_consensus.cc:1240] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:31.598140 15566 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:31.582271 20350 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:31.575285 20450 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:31.576505 20693 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:31.496563 20452 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:31.590845 20718 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:31.593387 20595 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:31.636960 15670 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 19.
I20260709 13:20:31.638487 15566 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:31.480230 20622 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:31.592736 20585 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:31.572494 20736 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:31.599721 20448 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:31.642894 16203 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:31.572278 20632 consensus_queue.cc:237] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:31.621905 20684 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:31.622187 20350 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:31.572494 20627 consensus_queue.cc:237] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:31.657531 16232 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 15.
I20260709 13:20:31.634493 20595 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:31.626011 20693 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:31.678793 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:31.656567 20562 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:31.641547 20660 consensus_queue.cc:1059] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:20:31.656706 20350 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:31.656955 20684 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:31.646898 20738 consensus_queue.cc:1059] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:20:31.646252 20739 consensus_queue.cc:1059] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:20:31.661754 20576 consensus_queue.cc:1059] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:20:31.694047 16234 raft_consensus.cc:1275] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:31.695441 20771 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:31.696091 20771 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:31.698493 20770 log.cc:927] Time spent T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:20:31.708396 20664 raft_consensus.cc:3055] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:31.708935 20664 raft_consensus.cc:740] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:31.724651 20585 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:31.728039 20321 raft_consensus.cc:3055] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:31.728566 20321 raft_consensus.cc:740] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:31.729714 20734 raft_consensus.cc:3055] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:31.730190 20734 raft_consensus.cc:740] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:31.731040 20734 consensus_queue.cc:260] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:31.735472 20760 log.cc:927] Time spent T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.609s	user 0.011s	sys 0.000s
I20260709 13:20:31.739940 20688 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:31.740581 20688 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:31.746938 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:31.749408 20610 consensus_meta.cc:287] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 1.036s	user 0.007s	sys 0.003s
I20260709 13:20:31.665902 20550 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:31.671713 20722 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:31.725317 20585 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:31.671702 20595 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:31.757695 20734 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:31.681124 20546 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:31.759012 20684 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:31.776791 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 585771us
I20260709 13:20:31.782286 20750 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:31.646144 20452 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:31.793686 20489 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:31.802083 20631 consensus_meta.cc:287] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.605s	user 0.010s	sys 0.000s
I20260709 13:20:31.797230 20702 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:31.803583 20452 raft_consensus.cc:697] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:31.806201 20771 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:31.802637 20649 raft_consensus.cc:697] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
W20260709 13:20:31.781800 20740 consensus_meta.cc:287] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.638s	user 0.006s	sys 0.000s
I20260709 13:20:31.819648 20631 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:31.829545 20701 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:31.830175 20688 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:31.856804 20649 consensus_queue.cc:237] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:31.859189 20749 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:31.854941 20622 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:31.835450 20736 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:31.848814 20609 raft_consensus.cc:3055] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:20:31.891845 20756 consensus_meta.cc:287] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.659s	user 0.009s	sys 0.000s
I20260709 13:20:31.874629 20693 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:31.896040 20702 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:31.900203 20756 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:31.898084 20703 consensus_meta.cc:287] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.668s	user 0.005s	sys 0.005s
I20260709 13:20:31.906961 20703 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:31.909354 20703 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:31.897622 20749 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:31.850342 20740 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:31.912206 20740 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:31.906160 20756 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:31.952461 15525 raft_consensus.cc:2393] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 14: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:20:31.952822 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 490361us
W20260709 13:20:31.958281 15494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.003s	sys 0.007s
I20260709 13:20:31.972604 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:31.974150 15596 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:20:31.975765 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:31.976733 15596 raft_consensus.cc:2393] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 13: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:31.978333 15596 raft_consensus.cc:1240] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:20:31.980485 15596 raft_consensus.cc:1240] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:20:31.982161 15596 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:31.983296 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:32.050472 15601 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:31.983903 15598 raft_consensus.cc:1275] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:20:31.985209 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:32.053277 15600 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:20:31.986454 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:31.989250 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:32.054435 15602 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:31.990543 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:32.056233 15520 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:31.991086 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:31.720140 15641 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:32.060695 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:32.012907 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 421142us
I20260709 13:20:32.012738 15493 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:32.019145 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:32.020459 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:32.038290 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:32.063771 15670 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:32.040656 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:32.046938 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:32.049204 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:32.065518 15566 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:32.068600 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:32.070696 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 58283.7us
W20260709 13:20:32.074419 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 354207us
I20260709 13:20:32.078236 15493 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:32.060711 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:32.082984 15642 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:32.092892 20339 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:32.119460 15602 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:20:32.093670 20350 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.096977 20432 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:32.143682 20432 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:32.144276 20432 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:32.098913 20452 consensus_queue.cc:237] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.100471 20546 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.903602 20684 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:31.872740 20609 raft_consensus.cc:740] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:32.131204 20688 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:32.133496 20585 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:32.163575 20673 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:32.166699 20610 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:32.167626 20610 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.175931 20609 consensus_queue.cc:260] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:32.176453 20546 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.177415 20609 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:32.178377 20779 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:32.178624 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 95899.2us
I20260709 13:20:32.179431 20546 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:32.179673 15642 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:32.181047 20627 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:32.182197 20772 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:32.179169 20779 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:32.175709 20586 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:32.198568 15675 raft_consensus.cc:1240] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 LEADER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:32.201978 20749 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:32.206236 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:32.207419 15673 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:32.214239 20777 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:32.215237 20777 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:32.216027 20739 consensus_queue.cc:1059] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:32.163827 20702 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:20:32.219012 20784 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:32.219890 20784 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.222055 20718 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.222955 20718 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:32.219914 20610 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:32.225081 20703 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.225245 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:32.225456 20684 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:32.225950 20703 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.226455 20546 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:32.230384 20758 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.231226 20758 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.234068 20688 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.236905 20688 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:32.163647 20693 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:32.238001 20693 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:32.240526 20693 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:32.243119 20775 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.243959 20775 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.245460 15597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 2428) took 1167 ms. Trace:
I20260709 13:20:32.245898 15597 rpcz_store.cc:276] 0709 13:20:31.077683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:31.077845 (+   162us) service_pool.cc:224] Handling call
0709 13:20:31.078155 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:31.595109 (+516954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:32.244987 (+649878us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b540025ba54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:32.245265 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1177819392}
I20260709 13:20:32.250460 20631 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:32.233855 20758 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:32.256106 15491 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.007s	sys 0.058s
I20260709 13:20:32.260666 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:32.170076 20339 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:32.266300 15491 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.008s	sys 0.058s
W20260709 13:20:32.269578 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 90120.9us
I20260709 13:20:32.270884 15642 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:32.271425 20779 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:32.210052 20771 consensus_queue.cc:1048] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:20:32.274358 20736 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:32.275115 20736 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:32.275871 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:32.276608 15520 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:32.277496 20736 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:32.278067 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:32.278160 15523 raft_consensus.cc:1240] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:32.279428 15670 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:20:32.280315 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:32.281067 15524 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 15.
I20260709 13:20:32.281378 15566 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:32.266537 16238 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:20:32.282337 20758 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:32.282783 20758 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.283183 20758 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:32.284397 20784 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:32.285135 20775 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:32.288460 15673 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:32.267535 20350 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:32.290786 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:32.294099 15522 raft_consensus.cc:1240] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:32.295619 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1718) took 1531 ms. Trace:
I20260709 13:20:32.295979 15522 rpcz_store.cc:276] 0709 13:20:30.764595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:30.766443 (+  1848us) service_pool.cc:224] Handling call
0709 13:20:30.766691 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:32.294641 (+1527950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:32.295328 (+   687us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54001748d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:32.295544 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3497250048}
I20260709 13:20:32.296751 15527 raft_consensus.cc:1240] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10. Ops: []
W20260709 13:20:32.297230 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.298028 20791 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:32.298202 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:32.298897 20791 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.299943 20608 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:32.300668 20608 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:32.301470 20791 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:32.302773 20450 consensus_meta.cc:287] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.672s	user 0.012s	sys 0.008s
I20260709 13:20:32.303122 20608 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:32.303333 20450 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.307003 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:32.282575 16238 raft_consensus.cc:606] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:32.309720 20793 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:32.309768 20649 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:32.311204 20546 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:32.310858 20793 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.311903 20546 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:32.313993 20339 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:32.314307 20546 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:32.315030 20784 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.315788 20784 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.314739 16238 raft_consensus.cc:614] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:32.319924 15599 raft_consensus.cc:1240] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:32.320358 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:32.321398 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:32.325106 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:32.325994 15520 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:32.326298 16240 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:20:32.322280 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:32.327117 20339 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:32.327205 15520 raft_consensus.cc:1240] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:32.327903 20339 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:32.327849 15599 raft_consensus.cc:2393] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 12: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:32.332597 20684 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:32.332255 20321 consensus_queue.cc:260] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.333523 20321 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:32.333240 20684 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:32.326946 16240 raft_consensus.cc:606] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:32.332178 20450 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:32.337213 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:32.333268 20784 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:32.339282 20684 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:32.326956 15594 raft_consensus.cc:1240] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:32.341403 15599 raft_consensus.cc:1240] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:32.342518 15527 raft_consensus.cc:1240] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:20:32.343398 20729 raft_consensus.cc:3055] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:20:32.343832 20729 raft_consensus.cc:740] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:32.344579 16238 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:20:32.345340 16238 raft_consensus.cc:606] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:32.345490 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:32.345774 16238 raft_consensus.cc:614] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:32.346725 15597 raft_consensus.cc:2410] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 13 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
W20260709 13:20:32.348037 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 77314.1us
I20260709 13:20:32.344274 20432 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.349092 15642 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:32.352658 20793 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:32.353538 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:32.354182 15674 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:32.352690 20432 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:32.356633 20595 consensus_meta.cc:287] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.581s	user 0.010s	sys 0.003s
I20260709 13:20:32.331605 20339 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:32.359373 15601 raft_consensus.cc:1240] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:32.360307 16235 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:20:32.333336 20586 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:32.365404 15674 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:32.341488 20664 consensus_queue.cc:260] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:32.369340 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 291387us
I20260709 13:20:32.344965 20729 consensus_queue.cc:260] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.371806 20729 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:32.371258 15493 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:32.378741 15566 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:32.378854 20339 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:32.379393 20339 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:32.379900 20339 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:32.381637 15493 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:32.382539 16239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53976 (request call id 1105) took 1159 ms. Trace:
I20260709 13:20:32.382987 16239 rpcz_store.cc:276] 0709 13:20:31.104324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:31.232421 (+128097us) service_pool.cc:224] Handling call
0709 13:20:31.232709 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:31.232913 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:20:31.246619 (+ 13706us) log.cc:844] Serialized 14 byte log entry
0709 13:20:31.743238 (+496619us) spinlock_profiling.cc:243] Waited 1.29 ms on lock 0x7b18004bd840. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab23a 00007f5ce4a15a2a
0709 13:20:32.246515 (+503277us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:32.246607 (+    92us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:32.246710 (+   103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:32.253284 (+  6574us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:32.263725 (+ 10441us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12835,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":102,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":122,"spinlock_wait_cycles":95783552,"thread_start_us":428906,"threads_started":1,"wal-append.queue_time_us":967429}
I20260709 13:20:32.385977 15603 raft_consensus.cc:1240] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10. Ops: []
W20260709 13:20:32.386194 15426 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49132 (request call id 340) took 28119 ms (client timeout 15000 ms). Trace:
I20260709 13:20:32.373255 20546 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:32.376648 20664 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:32.394603 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.377180 20595 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:32.404737 15566 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:32.405316 16205 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:32.407114 20684 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:32.407781 20684 raft_consensus.cc:697] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:20:32.408720 20790 log.cc:927] Time spent T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.242s	user 0.002s	sys 0.000s
I20260709 13:20:32.409298 20577 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:32.408942 20684 consensus_queue.cc:237] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.414016 20747 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:32.414860 20747 raft_consensus.cc:697] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:32.416038 20747 consensus_queue.cc:237] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.420359 16235 raft_consensus.cc:606] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:32.420849 16235 raft_consensus.cc:614] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:32.425464 15671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49094 (request call id 2269) took 1124 ms. Trace:
I20260709 13:20:32.420579 20718 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:32.425861 15671 rpcz_store.cc:276] 0709 13:20:31.301299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:31.306183 (+  4884us) service_pool.cc:224] Handling call
0709 13:20:31.306487 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:31.561971 (+255484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:32.424998 (+863027us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b540042a254. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:32.425289 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":507701120}
W20260709 13:20:32.427361 20787 log.cc:927] Time spent T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 13:20:32.401480 20608 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:32.401600 16204 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:32.386688 15426 rpcz_store.cc:269] 0709 13:20:04.266435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.268320 (+  1885us) service_pool.cc:224] Handling call
0709 13:20:32.376984 (+28108664us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:26.925717 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:27.168303 (+242586us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:27.189242 (+ 20939us) write_op.cc:435] Acquired schema lock
0709 13:20:27.200454 (+ 11212us) tablet.cc:662] Decoding operations
0709 13:20:27.451402 (+250948us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:27.550387 (+ 98985us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:27.568257 (+ 17870us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:20:27.775032 (+206775us) tablet.cc:701] Row locks acquired
0709 13:20:27.775083 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:20:27.775248 (+   165us) write_op.cc:270] Start()
0709 13:20:27.775393 (+   145us) write_op.cc:276] Timestamp: P: 1783603227775189 usec, L: 0
0709 13:20:27.775420 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:20:27.855044 (+ 79624us) log.cc:844] Serialized 2814 byte log entry
0709 13:20:28.526232 (+671188us) op_driver.cc:464] REPLICATION: finished
0709 13:20:29.189459 (+663227us) write_op.cc:301] APPLY: starting
0709 13:20:29.189567 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:29.455929 (+266362us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:29.456047 (+   118us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:29.503578 (+ 47531us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:29.503825 (+   247us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=0,mrs_lookups=0
0709 13:20:29.503866 (+    41us) write_op.cc:312] APPLY: finished
0709 13:20:29.504131 (+   265us) log.cc:844] Serialized 73 byte log entry
0709 13:20:29.505048 (+   917us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:29.505160 (+   112us) write_op.cc:454] Released schema lock
0709 13:20:29.505200 (+    40us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:29.507505 (+  2305us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:29.507535 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1866106,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":662949,"apply.run_cpu_time_us":11046,"apply.run_wall_time_us":318007,"cfile_cache_hit":30,"cfile_cache_hit_bytes":33443,"num_ops":7,"prepare.queue_time_us":495263,"prepare.run_cpu_time_us":33616,"prepare.run_wall_time_us":1031100,"raft.queue_time_us":185841,"raft.run_cpu_time_us":42522,"raft.run_wall_time_us":434251,"replication_time_us":750637,"thread_start_us":1038631,"threads_started":4,"wal-append.queue_time_us":188077,"wal-append.run_cpu_time_us":10104,"wal-append.run_wall_time_us":2446030}]]}
I20260709 13:20:32.408859 16205 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:32.434839 16204 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:32.311136 20649 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:32.429857 20604 consensus_meta.cc:287] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 1.283s	user 0.003s	sys 0.013s
I20260709 13:20:32.431321 20552 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:32.455348 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 106466us
I20260709 13:20:32.461717 20718 consensus_queue.cc:1048] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:32.463537 20703 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:32.464179 20703 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:32.464570 20594 raft_consensus.cc:3055] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:32.465114 20594 raft_consensus.cc:740] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:20:32.470721 20795 log.cc:927] Time spent T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:20:32.475839 20595 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
W20260709 13:20:32.477334 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 95761.5us
I20260709 13:20:32.481240 20569 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:32.432657 15425 catalog_manager.cc:5697] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:32.431396 20450 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:32.481747 20569 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:32.431105 20489 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.431258 20756 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:20:32.488885 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 160383us
W20260709 13:20:32.493224 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 58538.3us
I20260709 13:20:32.500242 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:32.499894 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:32.498733 20489 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:32.490996 20569 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:32.504436 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:32.505834 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:32.481419 20546 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:32.503564 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:32.506131 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:32.508055 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:32.513175 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:32.502357 20350 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:32.518026 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:32.525066 20649 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:32.526338 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:32.526912 20780 log.cc:927] Time spent T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.312s	user 0.002s	sys 0.000s
I20260709 13:20:32.527495 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:32.531401 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:32.532577 15597 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:32.533627 15599 raft_consensus.cc:2410] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:20:32.536352 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:32.537060 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:32.538025 20729 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:32.538460 20729 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:32.538833 20729 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:32.542059 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:32.542375 20546 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:32.543443 15670 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:32.545275 20546 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:32.548194 20777 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:32.549494 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:32.488799 20604 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.536455 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:32.502368 20450 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:32.493019 20552 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.529810 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:32.558584 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:20:32.529847 20578 consensus_meta.cc:287] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.946s	user 0.013s	sys 0.002s
I20260709 13:20:32.550580 20585 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:32.517335 20594 consensus_queue.cc:260] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.565680 15677 raft_consensus.cc:1240] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:32.569816 15528 raft_consensus.cc:1240] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:32.569907 15676 raft_consensus.cc:2393] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 13: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:20:32.566218 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 157502us
I20260709 13:20:32.568361 15668 raft_consensus.cc:2393] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 14: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:20:32.569460 15525 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2286) took 1763 ms (client timeout 1709 ms). Trace:
I20260709 13:20:32.530985 20779 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:32.578414 20578 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.580268 20779 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:32.581339 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:32.581689 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:32.566238 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:32.531865 15425 catalog_manager.cc:5697] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 7 to 11, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 11 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:20:32.562474 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:32.578418 15525 rpcz_store.cc:269] 0709 13:20:30.805886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:30.806019 (+   133us) service_pool.cc:224] Handling call
0709 13:20:32.569191 (+1763172us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540029c7d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:32.569372 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2597434112}
I20260709 13:20:32.587194 20489 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:32.587534 16233 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:32.554766 20756 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.589622 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:32.596180 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:32.597049 15673 raft_consensus.cc:2368] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 13. Current term is 14.
I20260709 13:20:32.598897 20739 consensus_queue.cc:1059] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:32.599598 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:32.600383 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:32.600334 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:32.602169 15601 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:32.567946 20604 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:32.587379 20633 raft_consensus.cc:3055] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:32.605489 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:32.605752 16204 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:32.607990 15670 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:32.614133 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:32.614903 15672 raft_consensus.cc:2393] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 11: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:32.599886 20739 raft_consensus.cc:3055] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:32.616496 16232 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:32.574448 20552 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:32.622234 15523 raft_consensus.cc:1240] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:32.622869 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:32.623009 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:32.622881 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:32.623971 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:32.626173 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 70452.7us
I20260709 13:20:32.626592 15671 raft_consensus.cc:1240] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:32.579370 20758 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.570986 20609 consensus_queue.cc:1048] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:20:32.565809 20585 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:32.570966 20350 consensus_queue.cc:1059] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:20:32.634361 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:32.616577 20633 raft_consensus.cc:740] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:20:32.635653 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 200482us
I20260709 13:20:32.642311 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:32.621680 20739 raft_consensus.cc:740] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:32.628604 20321 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:32.639634 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:32.634262 20693 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:32.650494 15602 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 LEADER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:32.646368 15567 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:32.656631 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:32.659860 15672 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:32.661208 15669 raft_consensus.cc:1240] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:32.662521 15673 raft_consensus.cc:1240] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:32.664626 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:32.666110 20350 consensus_queue.cc:1243] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer d3c4f5bd76e0445795bad5f6dd93ff69 log is divergent from this leader: its last log entry 8.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:20:32.667214 20739 consensus_queue.cc:260] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.668673 20739 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:32.648962 20779 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:32.674258 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:32.675516 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:20:32.676977 20789 log.cc:927] Time spent T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.508s	user 0.002s	sys 0.000s
I20260709 13:20:32.636969 20595 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.656845 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:32.628820 20664 consensus_queue.cc:1059] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:20:32.642346 15425 catalog_manager.cc:5697] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
W20260709 13:20:32.878298 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.926926 15672 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:20:32.895749 16233 raft_consensus.cc:1275] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:20:32.814440 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 163014us
I20260709 13:20:32.930682 15493 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:32.930909 15677 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 16.
W20260709 13:20:32.699973 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:32.798372 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 55561.9us
W20260709 13:20:32.813771 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 174961us
I20260709 13:20:32.680608 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:32.929291 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 280485us
W20260709 13:20:32.794086 20774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.018s	sys 0.042s
I20260709 13:20:32.673607 20569 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:32.945684 20774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.024s	sys 0.076s
I20260709 13:20:32.945760 20595 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:32.958302 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:32.742616 20802 log.cc:927] Time spent T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
W20260709 13:20:32.924698 20339 consensus_meta.cc:287] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.544s	user 0.007s	sys 0.003s
I20260709 13:20:32.959554 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:32.679132 20791 raft_consensus.cc:3055] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:32.960187 20791 raft_consensus.cc:740] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:32.959919 20339 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.961230 20791 consensus_queue.cc:260] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.961737 20569 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:32.962136 20339 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:32.962381 20791 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:32.677845 20729 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.964339 20729 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:32.659953 20633 consensus_queue.cc:260] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.965869 20633 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:32.968456 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:32.969738 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:32.741786 20756 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:32.970752 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:32.971462 20756 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:32.971973 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.972044 20756 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:32.929047 20585 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:32.975606 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:32.976771 15566 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:32.977083 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:32.977872 15595 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:20:32.978495 20578 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:32.978950 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:32.979923 15674 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:32.980870 15668 raft_consensus.cc:2410] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:20:32.648378 20321 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:32.955615 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:32.969818 15601 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:32.983358 15671 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
W20260709 13:20:32.734750 20803 log.cc:927] Time spent T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:20:32.927588 20758 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:32.683682 20693 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:32.992717 15596 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:32.973755 20807 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
W20260709 13:20:32.993003 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 48896.2us
I20260709 13:20:32.994520 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:32.996011 15594 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:32.996632 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:32.997422 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:32.995239 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:32.993510 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 53168.9us
I20260709 13:20:32.998834 15598 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:20:32.959569 20649 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:32.984995 20452 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.960206 20595 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:33.008977 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 48869us
I20260709 13:20:32.985036 20450 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:33.005300 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:32.986543 20321 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:33.011602 15671 raft_consensus.cc:1240] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:20:33.015472 15673 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:20:33.016630 20569 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.016778 20801 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:32.994513 20693 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:33.010157 15425 catalog_manager.cc:5697] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:32.977999 20660 consensus_queue.cc:1059] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.103s
I20260709 13:20:33.023921 20775 consensus_queue.cc:1048] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:33.017441 20801 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.026340 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:33.027079 15598 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:33.027837 20608 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.028497 20608 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.030007 20339 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:33.030911 20608 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:33.030627 20339 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:33.017213 20569 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.031960 15425 catalog_manager.cc:5697] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 8 to 13. New cstate: current_term: 13 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:33.033813 15425 catalog_manager.cc:5697] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 13 to 14, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 14 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:33.026466 20801 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:33.022102 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 89607us
I20260709 13:20:33.019168 15493 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:33.038784 15641 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:33.039479 15493 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:33.040581 20604 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:33.041026 20604 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:33.041369 20604 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:33.023321 20489 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:33.045420 20799 log.cc:927] Time spent T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.479s	user 0.002s	sys 0.000s
I20260709 13:20:33.045250 15425 catalog_manager.cc:5697] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 8 to 9, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 9 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:33.046352 20622 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:33.046787 20622 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:33.047138 20622 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:33.050432 20578 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.051009 20578 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.053148 20604 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:33.054136 20577 raft_consensus.cc:3055] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:33.054533 20577 raft_consensus.cc:740] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:33.054813 20452 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.013171 20684 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:33.056020 20684 raft_consensus.cc:697] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:33.056977 20684 consensus_queue.cc:237] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.057992 20807 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.059093 15425 catalog_manager.cc:5697] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 8 to 10, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 10 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:33.060191 20807 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:33.060843 15425 catalog_manager.cc:5697] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 6 to 9, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 9 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:20:33.061617 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 428878us
I20260709 13:20:33.064265 15567 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:33.066241 15668 raft_consensus.cc:1240] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 12. Current term is 13. Ops: []
W20260709 13:20:33.073889 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 52256.8us
I20260709 13:20:33.067088 15493 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:33.065757 15567 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:33.080761 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:33.006407 20649 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:33.110719 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 45063us
W20260709 13:20:33.111773 20809 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:20:33.047251 20793 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:33.108645 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:33.120276 15669 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:33.069061 20673 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.053776 20578 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:33.117055 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:33.126353 15677 raft_consensus.cc:2393] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 14: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:20:33.127158 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 110742us
I20260709 13:20:33.126533 20432 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:33.132041 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:33.117628 20608 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:33.157219 15671 raft_consensus.cc:1240] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:33.156901 15425 catalog_manager.cc:5697] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: config changed from index 5 to 8, VOTER 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) evicted. New cstate: current_term: 7 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:33.108318 20807 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:33.117545 20649 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:33.055500 20604 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:33.137405 20594 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:33.162976 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:33.163789 15673 raft_consensus.cc:2393] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 10: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:33.164315 20693 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.166519 15677 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:33.166694 15425 catalog_manager.cc:5697] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 7 to 9, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 9 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
W20260709 13:20:33.179919 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 53195.2us
I20260709 13:20:33.138778 15568 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:33.185335 20569 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:33.185621 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.138552 20432 raft_consensus.cc:697] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:33.131853 20452 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:33.127877 20664 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.206221 20664 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.208323 20664 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:33.210233 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:33.148262 20793 raft_consensus.cc:697] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:33.194641 20643 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:33.132480 20673 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.197213 20604 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:20:33.202455 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 122131us
I20260709 13:20:33.204567 15425 catalog_manager.cc:5697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 6 to 13. New cstate: current_term: 13 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:33.203809 15669 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:20:33.055430 20577 consensus_queue.cc:260] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.156909 20608 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:20:33.218379 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 214124us
I20260709 13:20:33.154510 20811 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.088s
I20260709 13:20:33.220822 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.225231 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:33.227032 15672 raft_consensus.cc:1275] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:20:33.229332 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:33.232813 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.218432 20489 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:33.221526 20608 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:33.237478 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:33.238765 15491 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:33.238899 15669 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:33.241209 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:33.242652 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:33.243305 15675 raft_consensus.cc:1275] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:20:33.248730 16231 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:20:33.248870 15596 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:20:33.243683 20784 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:33.249948 20693 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:33.250385 20784 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.251540 15568 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:33.252607 20784 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:33.240414 20578 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.254106 20452 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:33.262246 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:33.263478 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:33.265287 20425 consensus_queue.cc:1059] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:20:33.266666 15491 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:33.268286 15491 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:33.268442 20577 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:33.270586 15674 raft_consensus.cc:1240] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 13. Current term is 15. Ops: []
I20260709 13:20:33.270243 15425 catalog_manager.cc:5697] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 9 to 16. New cstate: current_term: 16 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:33.239848 20569 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:33.278571 16238 raft_consensus.cc:1275] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:20:33.282516 15425 catalog_manager.cc:5697] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:33.255247 20758 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.284516 15674 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:20:33.287487 15594 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
I20260709 13:20:33.289052 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.289762 15676 raft_consensus.cc:1240] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:33.294737 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:33.295512 15599 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 15.
I20260709 13:20:33.275187 20774 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:33.274688 20771 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:33.307349 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:33.295130 20813 log.cc:927] Time spent T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:20:33.299337 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:33.299604 15598 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:33.312247 15525 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
W20260709 13:20:33.313052 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 56761.1us
I20260709 13:20:33.314193 15673 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:33.313683 15425 catalog_manager.cc:5697] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:33.254735 20578 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:33.259512 20784 consensus_queue.cc:1059] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:20:33.253907 20321 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.298918 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:33.307474 20569 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:33.301301 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:33.307840 15675 raft_consensus.cc:1275] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:20:33.302024 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.333774 15521 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:20:33.274835 20693 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:33.301865 20722 consensus_queue.cc:1059] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 13:20:33.317996 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.301625 20594 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:33.335835 15526 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:33.336076 20594 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.338303 20594 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:33.330058 20793 consensus_queue.cc:237] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.301944 20734 consensus_queue.cc:1059] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.107s
I20260709 13:20:33.293524 20758 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:33.346969 20339 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:33.318984 20673 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:33.311478 20432 consensus_queue.cc:237] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.355418 20633 raft_consensus.cc:3055] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:33.325773 20350 raft_consensus.cc:3055] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:33.328011 20578 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:33.248906 20622 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:33.282117 20425 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
W20260709 13:20:33.350458 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:33.351168 15491 leader_election.cc:400] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 13. Current term is 14.
I20260709 13:20:33.368908 15491 leader_election.cc:403] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:20:33.350368 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:33.369762 15523 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:33.323446 20756 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:33.367043 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 53299.4us
I20260709 13:20:33.372695 20722 raft_consensus.cc:3055] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:33.373143 20722 raft_consensus.cc:740] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:33.374082 20722 consensus_queue.cc:260] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:33.375427 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:33.375104 20660 consensus_queue.cc:1059] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:33.375896 20734 consensus_queue.cc:1059] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:20:33.376259 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 66890.4us
I20260709 13:20:33.377482 16205 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:33.346514 20649 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.378333 20791 raft_consensus.cc:3055] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:33.378620 20758 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:33.378904 20664 raft_consensus.cc:3055] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:33.379272 20664 raft_consensus.cc:740] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:33.380018 20649 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:33.378270 20660 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:33.381074 20660 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.381946 20791 raft_consensus.cc:740] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:33.383276 20660 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:33.383836 20673 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.378123 20578 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:33.378064 20729 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:33.384394 16231 raft_consensus.cc:1240] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:33.384930 20729 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.362234 20350 raft_consensus.cc:740] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:33.387055 20729 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:33.387930 16238 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:20:33.388535 16238 raft_consensus.cc:606] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:33.388875 16238 raft_consensus.cc:614] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:33.380097 20664 consensus_queue.cc:260] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.390554 15491 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:33.393244 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:33.394318 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:33.394675 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:33.395221 16231 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:33.381352 20819 raft_consensus.cc:3055] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:33.370069 20569 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:33.392624 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:33.396704 20819 raft_consensus.cc:740] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:33.397074 16239 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:20:33.398085 20425 consensus_queue.cc:1243] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer 5498ef3e89a941e39c25f68cf64e6b0f log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:20:33.398638 20425 raft_consensus.cc:3055] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:33.398967 20425 raft_consensus.cc:740] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:33.399948 20425 consensus_queue.cc:260] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.401185 20425 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:33.402299 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:33.402438 20819 consensus_queue.cc:260] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.382906 20791 consensus_queue.cc:260] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.394251 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:33.403910 20791 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:33.405117 15642 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:33.406064 20585 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.406523 20585 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.396530 15523 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:20:33.408706 20585 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:33.409003 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:33.410389 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:33.384346 20673 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.410629 20604 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:33.411988 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.409202 20585 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:33.412636 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.402966 16238 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:33.413317 15526 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:33.410307 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:33.414354 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:33.414759 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.414500 16233 raft_consensus.cc:2410] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
W20260709 13:20:33.415863 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:33.416433 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.416600 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:33.417236 15594 raft_consensus.cc:2393] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 14: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:33.416445 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:33.417896 15599 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:33.411357 15642 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:33.419876 15642 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:33.420634 20718 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:33.422232 20718 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:33.422408 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:20:33.398880 20578 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 13. Current term is 14.
I20260709 13:20:33.423287 15642 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:33.423460 20594 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:33.423925 20594 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:33.413276 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:33.425302 15523 raft_consensus.cc:2410] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:20:33.426059 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.423615 20578 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:33.426887 20578 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:33.428030 20578 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:33.428817 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:33.391383 20622 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:33.368803 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 124368us
I20260709 13:20:33.430892 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:33.386699 20350 consensus_queue.cc:260] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.298837 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:33.434458 15595 raft_consensus.cc:2393] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 12: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
W20260709 13:20:33.435442 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 60666.3us
I20260709 13:20:33.362528 20633 raft_consensus.cc:740] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:33.425215 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:33.424384 20594 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:33.439188 16231 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:20:33.439486 15568 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:33.375363 20722 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:33.451802 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:33.423839 20718 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:33.459342 20718 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:33.459355 15594 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:20:33.463304 20819 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:33.412716 20673 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:33.466689 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:33.467456 16238 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:33.394127 15491 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:33.445125 15568 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:33.446779 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:33.470867 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 76819.4us
I20260709 13:20:33.471444 15672 raft_consensus.cc:2393] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 16: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:33.440474 20660 consensus_queue.cc:1059] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 13:20:33.402998 20321 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:33.456149 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:33.475803 15568 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:33.457078 20633 consensus_queue.cc:260] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.458084 20664 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:33.449844 20758 consensus_queue.cc:1059] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:20:33.458642 20585 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:33.479521 16205 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:33.480159 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:33.482215 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:33.486752 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:33.488314 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:33.492625 20784 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:33.493176 20784 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:33.493630 20784 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:33.494560 20633 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:33.487517 16238 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 15.
I20260709 13:20:33.458814 15642 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:33.497630 20585 raft_consensus.cc:697] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:33.475001 20649 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:33.471669 15491 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:33.498270 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.476450 20821 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:33.503979 20550 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:33.503898 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:33.507462 15491 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:33.511286 15491 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:20:33.511749 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 53029.2us
I20260709 13:20:33.512451 15491 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:33.513592 15491 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:33.444585 15599 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:20:33.530296 15425 catalog_manager.cc:5697] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 6 to 10, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 10 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:20:33.531016 15642 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:33.504032 20821 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:33.535174 20821 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:33.536688 20608 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:33.528838 20771 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:33.538391 20771 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:33.506109 20550 raft_consensus.cc:697] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:33.538748 20693 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:33.539661 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.539479 20550 consensus_queue.cc:237] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.522131 20339 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:33.546983 20774 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:33.547909 20673 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:33.549018 15528 raft_consensus.cc:1240] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:33.504743 20622 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:33.553296 15566 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:33.528273 20649 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:33.564494 15425 catalog_manager.cc:5697] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 5 to 9, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 9 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:20:33.538790 20771 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:33.576535 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:33.577891 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:33.533347 20747 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:20:33.580140 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 49203.7us
I20260709 13:20:33.581979 16231 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:33.588697 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:33.589663 15671 raft_consensus.cc:2393] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 15: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:33.588887 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:33.557426 20777 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:33.593070 15523 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
I20260709 13:20:33.554075 20622 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:33.594527 16203 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:33.583006 15566 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:33.602262 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:33.604624 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.605613 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.583567 20452 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:33.606631 20777 raft_consensus.cc:697] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:33.606989 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.608064 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.609371 15527 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:33.600562 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:33.610385 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.611101 15524 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 15.
I20260709 13:20:33.565908 20774 consensus_queue.cc:1048] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:20:33.598284 20747 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:33.605697 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.620007 15523 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:20:33.625051 20777 consensus_queue.cc:237] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:33.591651 20350 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:33.632575 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:33.582772 20604 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:33.636344 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:33.603518 20622 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:33.601683 20756 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:20:33.643909 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 61004.5us
I20260709 13:20:33.617348 20585 consensus_queue.cc:237] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:33.578737 20339 consensus_queue.cc:1059] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:20:33.583318 20693 consensus_queue.cc:1048] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
W20260709 13:20:33.647209 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 52805.2us
I20260709 13:20:33.658885 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.664729 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.670919 15425 catalog_manager.cc:5697] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 7 to 11, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 11 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:20:33.628598 20452 raft_consensus.cc:697] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:33.633548 20673 raft_consensus.cc:697] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:33.642807 20489 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:33.675954 16232 raft_consensus.cc:1240] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:33.649238 20684 raft_consensus.cc:3055] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:33.647971 20801 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.687017 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:33.650167 20722 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:33.686517 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.695817 15520 raft_consensus.cc:2368] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 12. Current term is 13.
I20260709 13:20:33.699658 16231 raft_consensus.cc:1275] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:20:33.694991 15519 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:33.696312 20660 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:33.703974 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.703876 20660 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.703600 16232 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:33.705379 20452 consensus_queue.cc:237] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:33.706135 20660 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:33.683924 20684 raft_consensus.cc:740] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:33.724905 20660 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:33.679466 20432 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 13:20:33.733363 20321 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.742748 16232 raft_consensus.cc:1240] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:33.711208 20608 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:33.745232 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.747707 15525 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
W20260709 13:20:33.750900 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 80991.7us
I20260709 13:20:33.751719 15566 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:33.757414 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:33.767467 15566 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:33.768795 16232 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:20:33.709858 20801 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.773720 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:33.774547 15525 raft_consensus.cc:2393] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 14: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:33.779690 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:33.758183 20807 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:20:33.742730 20660 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.783509 20660 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:33.776314 20321 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.786046 20321 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:33.787968 20594 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.788343 20684 consensus_queue.cc:260] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:33.755273 20718 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:33.791718 20718 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:33.791960 16237 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:20:33.745045 20673 consensus_queue.cc:237] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:33.795118 20829 log.cc:927] Time spent T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:20:33.780365 16235 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:20:33.769608 20552 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:20:33.794498 20824 log.cc:927] Time spent T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:20:33.812590 20594 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:33.813897 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:33.814643 15601 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:33.820109 20844 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:33.841456 15601 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
I20260709 13:20:33.842902 16205 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:33.843838 20594 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:33.844398 20594 raft_consensus.cc:697] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:33.845396 20594 consensus_queue.cc:237] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:33.853745 20843 log.cc:927] Time spent T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:20:33.855172 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:33.857870 20826 log.cc:927] Time spent T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:20:33.859220 20756 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:33.859709 20756 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:33.866484 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:33.867182 16239 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:20:33.868126 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:33.792133 20820 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
W20260709 13:20:33.878228 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 98227.8us
W20260709 13:20:33.868391 20832 log.cc:927] Time spent T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:20:33.928550 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 283644us
I20260709 13:20:33.807060 20552 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:33.936254 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:33.941530 15525 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:33.941226 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:33.942621 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:33.943289 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:33.941196 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:33.936708 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:33.947090 15600 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:33.939057 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:33.947788 16239 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
W20260709 13:20:33.952832 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:33.953127 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:33.966393 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 198999us
W20260709 13:20:33.942214 20839 log.cc:927] Time spent T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:20:33.943540 20552 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:33.974838 15674 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:20:33.973575 15642 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:33.979128 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:33.980340 15642 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:33.983909 16203 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:33.986258 16232 raft_consensus.cc:1275] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:20:33.988090 15528 raft_consensus.cc:1240] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:33.994657 16203 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:33.990339 20585 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:34.005869 20585 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:34.008010 20585 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:33.984989 20673 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:34.008735 20673 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:34.009089 20673 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:34.011310 16203 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:34.010123 20569 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:34.005935 20550 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:20:34.015058 20718 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:34.015695 20718 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:34.016340 15669 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:20:34.018083 20660 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:34.019433 20660 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:34.020229 20841 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:34.021271 20756 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:34.017846 20853 consensus_queue.cc:1059] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:34.021428 20821 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:34.022678 20747 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:34.020244 20718 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:34.025756 16203 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:34.026165 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:34.027011 15523 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:34.029210 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:34.031033 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:34.034626 15525 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:20:34.038769 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:34.039484 15519 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:34.042232 15520 raft_consensus.cc:2393] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 11: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:34.053122 15523 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
W20260709 13:20:34.069332 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 75627.3us
I20260709 13:20:34.055627 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:34.056279 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:34.056303 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:34.058545 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:34.022050 20774 raft_consensus.cc:3055] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:34.061755 15599 raft_consensus.cc:1240] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13. Ops: []
W20260709 13:20:34.063719 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 110918us
I20260709 13:20:34.021979 20820 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:20:34.039052 20569 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:20:34.071820 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 70899.7us
W20260709 13:20:34.059478 20851 log.cc:927] Time spent T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:20:34.068867 15519 raft_consensus.cc:1240] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:34.054100 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:34.071194 15600 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:20:34.070912 15668 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
W20260709 13:20:34.067306 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 178565us
I20260709 13:20:34.083892 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:34.107674 20756 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:34.088999 20834 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:20:34.110224 15642 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:34.126276 20834 raft_consensus.cc:2804] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:34.126679 20834 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:34.127018 20834 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:34.127532 20569 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:34.128069 20569 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:34.130057 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:34.130244 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:34.130700 15521 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:34.130841 15525 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:34.132889 16204 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:34.134298 20604 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:34.110833 20859 consensus_queue.cc:1059] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:34.072383 20774 raft_consensus.cc:740] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:34.118357 20684 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:34.149309 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:34.150364 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:34.150815 20859 raft_consensus.cc:3055] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:34.151252 20859 raft_consensus.cc:740] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:34.151926 20569 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:34.152097 20859 consensus_queue.cc:260] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:34.153432 20859 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:34.153855 20791 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:34.154485 20774 consensus_queue.cc:260] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:34.155490 20774 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:34.156142 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:34.157187 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:34.123273 20750 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:20:34.124868 20771 consensus_meta.cc:287] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.529s	user 0.009s	sys 0.000s
I20260709 13:20:34.086093 20425 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:34.134589 20801 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:34.137163 20834 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:34.110697 20718 raft_consensus.cc:3055] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:34.138301 16204 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:34.159220 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:34.167953 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:34.159289 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:34.154441 20791 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:34.165412 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:34.165717 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:34.169204 15523 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:20:34.168558 16235 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:34.174131 16204 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:34.175340 16204 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:34.171720 16240 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:20:34.147173 20594 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:34.147014 20604 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:34.162407 20750 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:34.107697 20821 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:34.173625 15567 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.615s (SENT) [suppressed 11 similar messages]
I20260709 13:20:34.203603 15567 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:34.211522 16232 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:20:34.196426 16204 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:34.171599 20718 raft_consensus.cc:740] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:34.178870 20569 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:34.198524 15493 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:34.162964 20622 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:34.217383 15567 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:34.219002 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:34.219278 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 93364us
I20260709 13:20:34.186573 20633 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:34.221661 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:34.222352 16235 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:34.223670 16204 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:34.187764 20321 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:34.226064 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:34.200310 20750 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:34.168699 20771 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:34.225315 15642 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:34.236498 16204 leader_election.cc:400] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 12. Current term is 13.
I20260709 13:20:34.172281 20425 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:34.239995 16204 leader_election.cc:403] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:20:34.228921 20569 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:34.243432 20673 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:34.248780 20834 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:34.251209 20693 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:34.252365 20833 raft_consensus.cc:2804] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:34.252810 20833 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:34.253147 20833 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:34.214259 20841 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:34.259730 20622 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:34.261085 16203 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:34.261541 20791 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:34.263247 20578 consensus_meta.cc:287] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.834s	user 0.009s	sys 0.001s
I20260709 13:20:34.263646 20578 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:34.264354 20425 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:34.264946 20718 consensus_queue.cc:260] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:34.261148 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:34.266229 15526 raft_consensus.cc:2393] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 17: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:34.267472 16203 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:34.268113 20664 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:34.269524 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 71079.8us
W20260709 13:20:34.270222 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 52933.1us
I20260709 13:20:34.270313 15493 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:34.271539 15567 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:34.276925 16232 raft_consensus.cc:1275] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:20:34.199813 20722 consensus_queue.cc:1059] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:20:34.278664 15521 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:20:34.245707 20673 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:34.265717 20578 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:34.238480 20594 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:34.291059 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:34.271175 20821 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:34.290202 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:34.304261 15596 raft_consensus.cc:1240] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:34.315217 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:34.283298 20608 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:34.316174 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:34.315217 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:34.316766 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:34.319139 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 93914.5us
I20260709 13:20:34.291344 20844 raft_consensus.cc:2764] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:20:34.323927 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:34.331918 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:20:34.334671 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 124868us
I20260709 13:20:34.301941 20660 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:34.337810 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:34.340588 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:34.341141 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:34.341894 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:34.345235 16204 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:34.347229 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:34.342001 15642 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:34.348511 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:34.349076 15600 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:34.347736 15595 raft_consensus.cc:2368] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12.
W20260709 13:20:34.333081 20784 consensus_meta.cc:287] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.834s	user 0.004s	sys 0.007s
I20260709 13:20:34.350189 15672 raft_consensus.cc:1240] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9. Ops: [8.10-8.10]
W20260709 13:20:34.351015 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 66985.3us
I20260709 13:20:34.355902 15567 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:34.351704 16204 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:34.357425 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:34.365964 15668 raft_consensus.cc:1240] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:20:34.318065 20684 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:20:34.368348 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:34.377787 15603 raft_consensus.cc:1275] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:20:34.378643 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 108045us
I20260709 13:20:34.374394 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:34.379451 16240 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:34.380501 15600 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:34.356161 20722 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:20:34.383749 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 113535us
I20260709 13:20:34.359957 20569 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:34.397555 15493 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:34.398780 15493 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:34.399919 15493 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:34.387852 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:34.402485 15668 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:34.403879 15567 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:34.396189 20452 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:34.419050 20452 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:34.419471 20452 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:34.423321 20777 consensus_queue.cc:1059] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:34.423883 15567 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:34.425094 15567 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:34.426088 20756 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:34.427032 20866 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:34.427512 20866 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:34.427655 20585 raft_consensus.cc:3055] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:34.428083 20585 raft_consensus.cc:740] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:34.428213 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:34.432737 15567 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:34.356647 20784 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:34.466611 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:20:34.481171 20840 log.cc:927] Time spent T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.673s	user 0.006s	sys 0.000s
I20260709 13:20:34.486112 20660 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:34.486707 20660 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:34.406152 20722 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:34.497545 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:34.428094 20844 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:34.427860 20866 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:34.505362 15675 raft_consensus.cc:2393] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 17: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:34.429703 20693 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:34.520423 20693 consensus_queue.cc:1059] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:34.507370 15603 raft_consensus.cc:1240] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:34.522104 20771 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:34.444226 20550 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:34.429508 20489 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.084s
I20260709 13:20:34.507586 16234 raft_consensus.cc:1240] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:34.506506 20722 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:34.524622 20784 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:34.441514 20622 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:20:34.519135 20865 log.cc:927] Time spent T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:20:34.522768 20771 raft_consensus.cc:3055] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:34.444299 20643 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.090s
I20260709 13:20:34.530947 20771 raft_consensus.cc:740] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
W20260709 13:20:34.532522 20863 log.cc:927] Time spent T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 13:20:34.533387 20857 log.cc:927] Time spent T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 13:20:34.536855 15602 raft_consensus.cc:1275] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:20:34.538951 15672 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:20:34.539984 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 140119us
I20260709 13:20:34.540783 15493 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:34.438218 20578 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:34.531975 20771 consensus_queue.cc:260] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:34.537544 20660 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:34.580811 15603 raft_consensus.cc:1275] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:20:34.573938 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 70306.6us
I20260709 13:20:34.582552 15595 raft_consensus.cc:1275] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:20:34.584604 15493 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:34.592952 16205 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:34.595762 20880 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.011s
W20260709 13:20:34.601953 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 127781us
I20260709 13:20:34.605067 15526 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:20:34.605412 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:20:34.589638 20852 log.cc:927] Time spent T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.740s	user 0.000s	sys 0.006s
I20260709 13:20:34.575567 20879 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:34.575279 20684 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.028s
W20260709 13:20:34.614527 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 274702us
I20260709 13:20:34.615265 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:34.602591 20771 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:34.588406 20604 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:34.626953 16235 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:34.613909 20880 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:34.627991 15641 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:34.594151 20452 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:34.640765 15641 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:34.647312 20874 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:20:34.647681 20552 consensus_meta.cc:287] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.667s	user 0.009s	sys 0.001s
I20260709 13:20:34.648133 20552 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:34.638123 20350 consensus_meta.cc:287] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.995s	user 0.012s	sys 0.000s
I20260709 13:20:34.651844 15597 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 11: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:34.654278 15668 raft_consensus.cc:1240] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:34.653694 15641 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:20:34.656101 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 85381.7us
I20260709 13:20:34.657344 15493 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:34.625653 20884 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:20:34.660157 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 51355.9us
I20260709 13:20:34.638754 20633 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 12. Current term is 13.
I20260709 13:20:34.657157 15641 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:34.645128 20853 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:34.671018 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:34.676280 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:34.659303 20859 consensus_meta.cc:287] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.505s	user 0.006s	sys 0.007s
I20260709 13:20:34.655690 20873 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:34.705062 15642 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:34.684823 20425 raft_consensus.cc:3055] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:34.686447 20660 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:34.713894 15642 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:34.697021 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:34.714850 15672 raft_consensus.cc:2393] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 10: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:34.703302 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:34.703557 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:34.652212 20880 raft_consensus.cc:697] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:34.716010 16234 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:20:34.695969 20452 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:34.710882 16240 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:20:34.716641 16235 raft_consensus.cc:2410] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 16 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
W20260709 13:20:34.716871 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 59798.1us
I20260709 13:20:34.717473 15641 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:34.718580 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:34.718266 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:34.675858 20608 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:20:34.725142 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 50040.7us
I20260709 13:20:34.728053 15594 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:20:34.723582 20883 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:34.704572 20873 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:20:34.731835 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:34.715346 20874 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:34.709688 20425 raft_consensus.cc:740] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:34.718798 20867 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:34.732856 16205 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:34.736852 16240 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:34.732244 15493 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:34.739168 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:20:34.726513 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 116706us
I20260709 13:20:34.721360 20569 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:34.724749 20784 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:34.761933 15520 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:34.751075 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:34.704082 20879 consensus_queue.cc:1048] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:20:34.736976 20873 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:34.770051 20867 raft_consensus.cc:697] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
W20260709 13:20:34.776664 20876 log.cc:927] Time spent T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:20:34.792035 15568 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:34.749650 20884 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:34.736238 20868 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:34.793571 15568 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:34.754395 20756 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:34.806370 20450 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:20:34.817044 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 88000.4us
I20260709 13:20:34.736377 20608 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:34.809423 20774 consensus_meta.cc:287] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.654s	user 0.010s	sys 0.008s
W20260709 13:20:34.797341 20870 log.cc:927] Time spent T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:20:34.809002 20321 consensus_meta.cc:287] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.579s	user 0.008s	sys 0.009s
W20260709 13:20:34.827702 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 77738.7us
I20260709 13:20:34.827911 20660 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:34.828578 20321 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:34.828578 15493 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:34.829483 16238 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:34.791613 20879 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:34.829867 20774 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:34.831322 15493 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:34.778151 20784 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:34.813907 20489 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:34.778842 20673 consensus_queue.cc:1048] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:20:34.852890 20722 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:34.802695 20844 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:34.815827 20552 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:34.859584 20883 consensus_queue.cc:1059] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:34.859993 20777 consensus_queue.cc:1059] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:34.860782 20883 raft_consensus.cc:3055] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:34.861213 20883 raft_consensus.cc:740] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:34.862569 16235 raft_consensus.cc:1240] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:20:34.834441 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:34.841179 20887 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:20:34.842314 20864 consensus_queue.cc:1059] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.102s
I20260709 13:20:34.843182 20885 consensus_queue.cc:1059] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.114s
I20260709 13:20:34.844094 20867 consensus_queue.cc:237] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:34.845727 20585 consensus_queue.cc:260] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:34.851305 20425 consensus_queue.cc:260] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:34.853168 20673 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:34.744005 20750 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:34.853726 20833 consensus_meta.cc:287] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.600s	user 0.009s	sys 0.000s
I20260709 13:20:34.825549 20884 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:34.820616 20718 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:34.871698 15493 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:34.863492 20774 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:20:34.943502 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 71885.6us
I20260709 13:20:34.945067 15677 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 11: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:34.874971 20339 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:34.867563 20883 consensus_queue.cc:260] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:34.941154 20774 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:34.815563 20880 consensus_queue.cc:237] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:34.931233 20594 consensus_queue.cc:1059] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.090s
I20260709 13:20:34.951900 20834 raft_consensus.cc:3055] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:34.974382 20879 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:34.978524 15493 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:34.959697 20864 raft_consensus.cc:3055] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:34.973629 20339 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:34.943609 20884 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.003175 20585 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:35.007413 20774 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:35.007506 20899 raft_consensus.cc:3055] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:34.941951 20833 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:35.023463 20892 log.cc:927] Time spent T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:20:35.024036 20886 log.cc:927] Time spent T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7: Append to log took a long time: real 0.197s	user 0.000s	sys 0.002s
I20260709 13:20:35.011690 20900 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:20:35.015583 20864 raft_consensus.cc:740] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:35.021190 20883 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:35.032617 16233 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
I20260709 13:20:35.011940 20834 raft_consensus.cc:740] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:35.017252 20339 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:35.017781 20885 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:20:35.060822 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:35.036515 20899 raft_consensus.cc:740] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:35.084293 20899 consensus_queue.cc:260] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.086069 20633 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.086689 20633 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.088191 20877 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.088728 20877 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.090328 20722 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:35.086124 20791 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.091346 20791 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.092499 20833 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:35.093348 15676 raft_consensus.cc:1240] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:35.094295 20864 consensus_queue.cc:260] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.095780 20864 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:35.097558 20905 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:35.098197 20905 raft_consensus.cc:697] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:35.016698 20866 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.099182 20905 consensus_queue.cc:237] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.100737 20866 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:35.101598 20550 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:35.102164 20550 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:35.104856 20550 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:35.107980 20750 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:35.048823 20821 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.111465 20900 consensus_queue.cc:1059] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:35.033785 20801 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:35.114454 20885 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.125331 16238 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
I20260709 13:20:35.085666 20899 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:35.098095 20608 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:35.134953 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 334263us
I20260709 13:20:35.140784 20339 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:35.141652 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 126846us
I20260709 13:20:35.151131 15642 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:35.153048 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:35.083045 20578 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:35.162242 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:35.155901 15523 raft_consensus.cc:1240] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:35.157528 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:35.154692 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:35.159246 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:35.169392 15528 raft_consensus.cc:2393] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 13: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:35.160162 20722 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.162319 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:35.162798 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:35.159197 20774 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.168713 15524 raft_consensus.cc:1240] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:35.168150 15523 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:20:35.170830 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 350940us
I20260709 13:20:35.171698 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:35.172081 15641 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:35.198992 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:35.150903 20874 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:35.206197 15493 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:35.204413 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:35.215969 20339 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:35.174103 20750 raft_consensus.cc:2804] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:35.217907 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:35.133255 20821 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.214058 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:35.174872 15523 raft_consensus.cc:2393] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 12: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:35.244606 15522 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 11: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:35.135824 20873 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:35.134290 20801 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.203941 20608 raft_consensus.cc:3055] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:20:35.262173 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 48346.9us
I20260709 13:20:35.263185 15493 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:35.210249 20904 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:35.266026 20339 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:35.267654 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:35.265679 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:35.268291 16232 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:35.268421 16234 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:35.198410 20722 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.260389 20784 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:35.217211 20879 raft_consensus.cc:697] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:35.157616 20834 consensus_queue.cc:260] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.202694 20578 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.257230 20673 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.266773 20908 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:35.278488 20879 consensus_queue.cc:237] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.283275 20578 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:35.283879 20673 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:35.283324 20722 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:35.260686 20608 raft_consensus.cc:740] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:35.288735 15493 leader_election.cc:400] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12.
I20260709 13:20:35.289237 15493 leader_election.cc:403] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:20:35.268123 20904 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:35.297799 15493 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:35.323958 16232 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:20:35.329131 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:35.315137 20578 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12.
I20260709 13:20:35.309180 20904 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:35.332309 20884 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:35.345754 20771 consensus_meta.cc:287] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.717s	user 0.011s	sys 0.000s
I20260709 13:20:35.335731 20750 raft_consensus.cc:697] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:35.316622 20900 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:20:35.338781 20791 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:35.371248 20885 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:35.372047 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:35.371878 20885 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.374603 20873 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:35.405208 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 89608.1us
I20260709 13:20:35.356985 20916 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:35.409413 15493 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:35.415940 15493 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:35.425251 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:35.427032 20911 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:35.427249 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:35.428102 20750 consensus_queue.cc:237] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.428663 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:35.430142 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:35.430691 15521 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:35.436003 20887 consensus_queue.cc:1059] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:35.436571 20774 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:35.427251 20918 log.cc:927] Time spent T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:20:35.437826 20845 raft_consensus.cc:3055] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:35.438324 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:35.439114 15524 raft_consensus.cc:2410] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 9 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:20:35.438297 20845 raft_consensus.cc:740] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:35.439859 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:35.441068 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:35.441718 20425 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:35.442333 16234 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
I20260709 13:20:35.442910 15523 raft_consensus.cc:2393] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 17: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:35.444819 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:35.445843 16234 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 17: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:35.446904 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:35.447608 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:35.449815 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:35.439787 20604 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:35.451038 15641 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:35.433332 20608 consensus_queue.cc:260] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.452585 20604 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:35.454718 15493 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:35.455716 20864 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.456228 20864 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:35.460915 15523 raft_consensus.cc:1275] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:20:35.461472 15523 pending_rounds.cc:85] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 6
I20260709 13:20:35.451038 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:35.462818 15520 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:35.442173 20771 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:35.461761 15523 pending_rounds.cc:107] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:20:35.472946 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:35.426570 20880 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:35.404076 20885 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.474768 15523 raft_consensus.cc:2889] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:35.473976 15602 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 11: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:35.475705 15520 raft_consensus.cc:1275] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:20:35.479279 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:35.483548 20913 log.cc:927] Time spent T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 13:20:35.486686 20339 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:35.487195 20339 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:35.488731 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:35.477020 20900 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.490303 20900 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:35.447657 16234 raft_consensus.cc:2393] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 12: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:35.492936 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:35.493621 16234 raft_consensus.cc:2393] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 12: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:35.490294 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:35.497094 15669 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:20:35.489104 20339 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:35.498418 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 43841.4us
I20260709 13:20:35.470620 20821 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:35.473043 20673 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:35.511334 15522 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
I20260709 13:20:35.506537 16203 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:35.508533 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:35.514784 16203 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:35.515286 15524 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 11: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:35.514750 15641 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:35.516885 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:35.493520 20900 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:35.520219 15520 raft_consensus.cc:1275] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:20:35.517666 15671 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:20:35.516794 20569 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:35.525134 20718 consensus_queue.cc:1048] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:20:35.535046 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:35.507866 20693 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:35.511387 20859 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:35.534713 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:35.511679 20864 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:35.513433 20821 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.538146 15599 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:20:35.535728 15601 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:20:35.540743 15493 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:35.520119 20784 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:35.541533 20784 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:35.541538 20908 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:35.542131 20784 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:35.542160 20908 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:35.542834 20908 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:35.543838 15493 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:35.544588 20339 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:35.544955 20339 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:35.545272 20339 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:35.547037 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:35.539287 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:35.548898 15520 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 15.
I20260709 13:20:35.551324 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:35.552635 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:35.554205 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:35.554970 16239 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 11: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:35.555971 16239 raft_consensus.cc:1240] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:35.557118 16239 raft_consensus.cc:1240] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:35.561640 15642 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:35.563431 15491 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:35.525285 20673 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:35.569355 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:35.565786 15641 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:35.538666 16203 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:35.577950 15521 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
W20260709 13:20:35.583122 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 44533.1us
I20260709 13:20:35.590453 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:35.591209 15671 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:35.593338 15642 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:35.566519 20784 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.525477 20853 consensus_queue.cc:1048] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:20:35.611099 20859 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:35.571326 20718 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:20:35.581444 20673 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:35.616796 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:35.538422 20873 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:20:35.618988 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:35.620745 15491 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:35.571300 20771 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:35.581864 20864 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:35.623114 15524 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:20:35.608301 20569 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:35.571256 20693 consensus_queue.cc:1059] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:20:35.629083 20339 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:35.624271 20673 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:35.607807 20916 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.567891 20821 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:35.626711 20905 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:35.538025 20845 consensus_queue.cc:260] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.625000 20750 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:35.652166 20339 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:35.627280 20864 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:35.630568 20569 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:35.658632 20864 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:35.651091 20905 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:35.630663 20891 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.662787 20885 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:35.662200 20916 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:35.681382 20916 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:35.666528 20845 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:35.666239 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:35.675848 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:35.688489 16234 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 16.
I20260709 13:20:35.667070 20750 raft_consensus.cc:697] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:35.690466 15642 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:35.691653 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:35.693013 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:35.689145 20911 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.694725 20911 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.696897 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:35.692698 20923 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:35.697525 15523 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 15.
I20260709 13:20:35.697656 20923 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:35.685323 20569 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:35.705680 20923 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:35.666545 20905 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:35.704005 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:35.675108 20784 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:35.712779 15567 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:35.714632 15567 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:35.712661 20931 consensus_queue.cc:1059] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:35.717273 20608 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:35.714370 20750 consensus_queue.cc:237] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.719740 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:35.722782 15523 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:20:35.725649 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:35.708069 20911 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:35.726730 20550 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:20:35.727106 20928 log.cc:927] Time spent T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 13:20:35.717849 20643 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:20:35.736712 20904 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.737347 20801 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:35.739429 20897 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:20:35.739615 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:35.775266 16205 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:35.718320 20673 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:35.776496 16205 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:35.748387 20834 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:35.753804 20821 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:35.755954 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:35.757957 15523 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 15.
I20260709 13:20:35.778021 20425 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:35.780004 15523 raft_consensus.cc:1240] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:20:35.780455 15673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2746) took 1265 ms. Trace:
I20260709 13:20:35.675446 20891 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.782552 16204 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:35.773468 20908 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:35.783502 20891 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:35.784083 20891 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:35.784533 20891 raft_consensus.cc:697] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:35.785115 20908 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:35.740832 20901 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.878s	user 0.006s	sys 0.000s
I20260709 13:20:35.721901 20756 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:35.787963 16231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1551) took 1060 ms. Trace:
I20260709 13:20:35.789350 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:35.788331 16231 rpcz_store.cc:276] 0709 13:20:34.727525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:34.727648 (+   123us) service_pool.cc:224] Handling call
0709 13:20:34.727931 (+   283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:34.728255 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:34.728283 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:34.728700 (+   417us) log.cc:844] Serialized 26 byte log entry
0709 13:20:34.811141 (+ 82441us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:20:34.813089 (+  1948us) log.cc:844] Serialized 14 byte log entry
0709 13:20:34.829891 (+ 16802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:34.830021 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:35.740476 (+910455us) spinlock_profiling.cc:243] Waited 5.52 ms on lock 0x7b24002fdb48. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce17a49c4 00007f5ce17a2acb 00007f5ce195ddc9 00007f5ce195da55 00007f5ce195d09d 00007f5ce458f10e 00007f5ce45a688c 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221
0709 13:20:35.787657 (+ 47181us) raft_consensus.cc:1685] finished
0709 13:20:35.787761 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:35.787887 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18360,"prepare.queue_time_us":141025,"prepare.run_cpu_time_us":6012,"prepare.run_wall_time_us":20735,"raft.queue_time_us":19021,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":3997,"spinlock_wait_cycles":12702080,"thread_start_us":22433,"threads_started":2,"wal-append.queue_time_us":90022}
I20260709 13:20:35.780865 15673 rpcz_store.cc:276] 0709 13:20:34.515599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:34.515852 (+   253us) service_pool.cc:224] Handling call
0709 13:20:34.516196 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:34.538125 (+ 21929us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:34.538154 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:34.538644 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:20:35.379121 (+840477us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b180010c480. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce18ec2a6 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9
0709 13:20:35.508528 (+129407us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:35.511034 (+  2506us) log.cc:844] Serialized 14 byte log entry
0709 13:20:35.511243 (+   209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:35.511765 (+   522us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:35.780171 (+268406us) raft_consensus.cc:1685] finished
0709 13:20:35.780270 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:35.780400 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21554,"prepare.queue_time_us":4266,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":22299,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":6936,"spinlock_wait_cycles":15172096,"thread_start_us":128525,"threads_started":1,"wal-append.queue_time_us":1001631}
I20260709 13:20:35.790249 20784 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:35.807590 16205 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:35.808416 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:35.812800 15673 raft_consensus.cc:1240] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:35.785339 20891 consensus_queue.cc:237] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.818717 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:35.775991 20897 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.792479 20425 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:20:35.825457 20784 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:35.847388 20904 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:35.847847 20867 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:35.848376 20867 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.849445 20897 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:35.850428 20867 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:35.851022 20569 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.853165 20569 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:35.804352 20923 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.856010 20923 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:35.856834 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:35.857034 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:35.821583 20774 consensus_queue.cc:1059] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:20:35.858237 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:35.835383 16238 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 1129) took 1503 ms. Trace:
I20260709 13:20:35.859117 15673 raft_consensus.cc:1240] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:35.837335 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:35.838331 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:35.860116 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:35.860702 15671 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:35.861596 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:35.862140 16234 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:35.813452 20864 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:35.861335 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:35.840360 20673 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:35.844282 20633 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:35.845253 16233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 1117) took 2213 ms (client timeout 1746 ms). Trace:
W20260709 13:20:35.905189 16233 rpcz_store.cc:269] 0709 13:20:33.632195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:33.632331 (+   136us) service_pool.cc:224] Handling call
0709 13:20:35.845036 (+2212705us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540068f454. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:35.845149 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2381814656}
I20260709 13:20:35.839958 15677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49100 (request call id 2753) took 1105 ms. Trace:
I20260709 13:20:35.858880 16238 rpcz_store.cc:276] 0709 13:20:34.331554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:34.331688 (+   134us) service_pool.cc:224] Handling call
0709 13:20:35.835150 (+1503462us) spinlock_profiling.cc:243] Waited 497 ms on lock 0x7b54007b02d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:35.835288 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1142962560}
I20260709 13:20:35.871411 15493 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:35.871646 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:35.908056 15675 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:35.876279 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:35.880798 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:35.881335 20904 consensus_queue.cc:1059] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:35.918576 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:35.883188 20750 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:35.886341 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:35.887439 15526 raft_consensus.cc:1240] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:35.889878 20684 raft_consensus.cc:3055] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:20:35.899374 20917 log.cc:927] Time spent T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.475s	user 0.002s	sys 0.000s
W20260709 13:20:35.899956 20921 log.cc:927] Time spent T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.463s	user 0.005s	sys 0.001s
I20260709 13:20:35.900126 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:35.903834 20864 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:35.904500 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:35.910401 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:35.938259 15602 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:35.911478 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:35.940294 15600 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:20:35.914220 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:35.941232 15599 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:35.919612 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:35.924110 15671 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 15.
I20260709 13:20:35.906931 15677 rpcz_store.cc:276] 0709 13:20:34.734778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:34.734933 (+   155us) service_pool.cc:224] Handling call
0709 13:20:35.839717 (+1104784us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b54001a8e54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:35.839861 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":431085824}
I20260709 13:20:35.930552 16234 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
I20260709 13:20:35.930778 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:35.931099 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:35.933152 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 61820.8us
I20260709 13:20:35.943892 15673 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:35.933881 20821 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.934228 20801 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.934365 20845 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:35.934792 20904 raft_consensus.cc:3055] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:35.936721 16238 raft_consensus.cc:1275] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:20:35.944207 15525 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:35.944201 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:35.946043 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:35.972770 15523 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:20:35.950613 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:35.973467 16233 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:20:35.950973 16232 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:20:35.959348 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:35.966754 15599 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:20:35.969247 15673 raft_consensus.cc:1240] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:20:35.976111 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:35.933385 20774 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:35.976330 15602 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:35.937870 20684 raft_consensus.cc:740] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:35.978466 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:35.984987 15668 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:20:35.989849 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:35.937361 20633 consensus_queue.cc:1059] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:20:35.994932 16231 raft_consensus.cc:1275] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:20:35.992620 15493 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:35.971895 20904 raft_consensus.cc:740] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:35.988313 20900 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:35.969020 20821 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.012089 15641 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:36.016359 15642 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:35.954475 20750 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.018196 15526 raft_consensus.cc:1275] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:20:35.981380 20845 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:35.973292 20801 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.023593 15642 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:36.027181 20864 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:36.027303 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:36.027849 20864 raft_consensus.cc:697] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:36.027935 15521 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:36.030943 15567 leader_election.cc:336] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.521s (SENT)
I20260709 13:20:36.031209 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:36.032847 15567 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:36.027103 20874 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:36.010032 20774 raft_consensus.cc:2804] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:36.037492 15641 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:35.999972 20908 consensus_queue.cc:1059] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 13:20:36.039273 15528 raft_consensus.cc:1240] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:36.039026 20673 consensus_queue.cc:1059] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:20:36.039732 20899 consensus_queue.cc:1048] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:36.040398 20924 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:36.004369 20926 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:20:36.037499 15596 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:20:36.047466 15597 raft_consensus.cc:1275] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:20:36.054654 20869 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 1.613s	user 0.000s	sys 0.005s
I20260709 13:20:36.059429 20801 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:36.038077 15523 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
I20260709 13:20:36.050441 20853 consensus_queue.cc:1048] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:36.048501 20923 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:36.045699 20867 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:36.062911 15675 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 15.
I20260709 13:20:36.066630 15521 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 16.
W20260709 13:20:36.064082 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 88376.7us
I20260709 13:20:36.018612 20608 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:20:36.067294 15526 raft_consensus.cc:1240] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:20:36.067391 15566 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:36.063952 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:36.028882 20864 consensus_queue.cc:237] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.071764 20924 raft_consensus.cc:697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:36.072716 20924 consensus_queue.cc:237] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.074954 15641 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:36.076159 20873 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:36.076843 20873 raft_consensus.cc:697] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:36.077745 20867 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.078264 20784 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:36.078848 20784 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.079644 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:36.080461 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:36.081063 16236 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
I20260709 13:20:36.081578 20784 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:36.076206 20750 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:36.077713 20873 consensus_queue.cc:237] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:36.083397 20845 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:36.083320 15493 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:36.084730 16205 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:36.085156 15493 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:36.045845 20774 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:36.087278 15600 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:36.066183 20887 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:36.088104 15493 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:36.085641 15642 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:36.088516 16205 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:36.020826 20684 consensus_queue.cc:260] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:36.094033 15599 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:20:36.098531 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:36.099612 16205 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:36.100242 16236 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 15.
I20260709 13:20:36.089843 20908 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:36.088719 20774 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:36.103113 20908 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:36.103518 20908 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:36.103467 20693 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:36.104769 16205 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:36.105973 20693 raft_consensus.cc:697] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:36.106314 20784 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:36.106854 20693 consensus_queue.cc:237] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:36.108832 20905 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.111089 20905 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:36.112288 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:36.119007 20821 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:36.119897 20880 raft_consensus.cc:3055] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:36.120409 20880 raft_consensus.cc:740] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:36.121326 20880 consensus_queue.cc:260] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.122670 20880 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:36.105046 20949 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:36.127099 20877 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:36.099069 20777 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:36.128621 20777 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:36.129030 20777 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:36.091375 20920 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:36.134519 20920 raft_consensus.cc:697] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:36.135457 20920 consensus_queue.cc:237] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:36.137023 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:36.137696 15603 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:36.138276 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:36.104372 20643 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:36.142445 20643 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:36.142817 20643 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:36.115329 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:36.079954 20867 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:36.143817 20853 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:36.144325 20853 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:36.144726 20853 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:36.033650 20904 consensus_queue.cc:260] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:36.145702 20904 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:36.148017 20791 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:36.151449 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:36.153064 16203 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:36.154362 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:36.155145 16238 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:20:36.155158 16233 raft_consensus.cc:2393] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 12: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:36.163326 15600 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:20:36.154402 15641 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:20:36.174717 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 74623.2us
I20260709 13:20:36.112671 20908 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:36.188508 20908 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:36.190537 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:36.176507 16205 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:36.192229 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:36.193046 15600 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 11: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:36.144583 20722 raft_consensus.cc:3055] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:36.205825 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:36.092942 20594 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:36.204339 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:36.207827 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:36.208423 15600 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:36.210227 16236 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 11: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:36.211009 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:36.213121 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 63941.9us
I20260709 13:20:36.215163 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:36.215463 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:36.202535 20946 log.cc:927] Time spent T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.138s	user 0.000s	sys 0.002s
W20260709 13:20:36.202766 20950 log.cc:927] Time spent T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.098s	user 0.002s	sys 0.000s
I20260709 13:20:36.174193 20633 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:36.219774 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:36.227069 15599 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
I20260709 13:20:36.202525 20321 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:36.191783 20949 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:36.226971 20339 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 13:20:36.206214 20722 raft_consensus.cc:740] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:36.215071 20784 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:36.206648 20885 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:36.202780 20867 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:36.231894 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:36.203013 20884 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:36.245113 20884 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.247375 20884 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:36.248265 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:36.248447 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:36.248939 16231 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:20:36.249070 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:36.229434 15600 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:20:36.220613 20684 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:36.242677 20321 raft_consensus.cc:697] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:36.244867 20622 raft_consensus.cc:3055] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:36.244488 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:36.252096 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:36.189450 20791 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.242864 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:36.256090 16236 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:36.253099 16231 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:20:36.249642 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:36.229971 20864 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:36.262912 20321 consensus_queue.cc:237] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.262962 15601 raft_consensus.cc:1240] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:36.266229 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:36.268451 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:36.264181 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:36.270110 15522 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:36.221952 20911 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:20:36.271756 15603 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
I20260709 13:20:36.270779 20923 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:36.274854 15597 raft_consensus.cc:1275] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:20:36.278028 16204 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:36.258121 20622 raft_consensus.cc:740] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:36.256322 20885 raft_consensus.cc:697] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:36.274180 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:36.290019 15600 raft_consensus.cc:2376] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11. Re-sending same reply.
I20260709 13:20:36.285230 16205 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:36.242539 20949 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.297597 15668 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:36.254788 20784 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.300414 15491 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:36.258544 20867 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:36.273190 20899 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:20:36.282349 20884 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:36.271492 20864 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.301548 20883 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:36.320740 16237 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 17.
I20260709 13:20:36.301759 20718 raft_consensus.cc:3055] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:36.274739 20923 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } attrs { replace: true } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.302318 20905 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 13:20:36.320182 20884 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:36.440965 20960 log.cc:927] Time spent T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 13:20:36.415167 20718 raft_consensus.cc:740] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
W20260709 13:20:36.457499 20957 log.cc:927] Time spent T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.217s	user 0.002s	sys 0.008s
I20260709 13:20:36.315855 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:36.308017 20722 consensus_queue.cc:260] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.431285 20884 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:36.427428 20569 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:36.413573 20622 consensus_queue.cc:260] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.458546 20693 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:36.486577 20339 consensus_queue.cc:1175] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" responder_term: 5 status { last_received { term: 5 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 5 index: 7 } }
I20260709 13:20:36.459838 20874 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:36.477387 20585 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:36.441035 20853 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.488610 20693 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.485883 20569 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.529086 20569 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:36.529664 20569 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:36.530154 20569 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.536522 20569 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:36.537458 20784 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:36.495262 20585 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.508445 20791 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:36.511736 20643 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:36.516079 20774 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.461118 20885 consensus_queue.cc:237] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:36.500569 20874 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:36.507189 20959 log.cc:927] Time spent T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:20:36.562139 20585 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:36.562567 20874 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:36.548776 20883 raft_consensus.cc:697] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:36.566052 20577 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:36.567970 15525 raft_consensus.cc:1240] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:36.505096 20777 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.581511 20777 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:36.587934 20956 log.cc:927] Time spent T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
W20260709 13:20:36.589124 20958 log.cc:927] Time spent T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 13:20:36.590505 20821 consensus_queue.cc:1059] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:20:36.592885 20967 log.cc:927] Time spent T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:20:36.593232 15493 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:20:36.594430 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:36.596939 16205 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:36.651436 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:36.652120 15598 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:20:36.576290 20864 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:36.588011 20953 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:36.673305 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 80196.2us
W20260709 13:20:36.672255 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 75403.4us
I20260709 13:20:36.712479 20900 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:36.713792 16205 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:36.674911 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 499464us
W20260709 13:20:36.655287 20904 consensus_meta.cc:287] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.509s	user 0.010s	sys 0.000s
W20260709 13:20:36.710167 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 609415us
I20260709 13:20:36.618212 20585 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:36.566743 20883 consensus_queue.cc:237] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.620944 20899 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:36.718169 15600 raft_consensus.cc:1275] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:20:36.675757 20321 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:36.711350 20953 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:36.901556 20643 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:36.714404 20900 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:20:36.710834 20608 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:36.944267 20585 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.903499 20984 raft_consensus.cc:3055] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:20:36.904508 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:36.948982 20608 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.910837 20791 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:36.955216 20791 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:36.957430 20791 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:36.912313 20853 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:36.920300 20693 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:36.719204 20973 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:36.962056 20973 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:36.671066 20877 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:36.962617 20973 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:36.966668 20900 consensus_queue.cc:1048] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:20:36.975620 20718 consensus_queue.cc:260] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:36.976796 20718 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:36.977248 20923 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:36.937274 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 353113us
I20260709 13:20:36.938336 15567 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:36.978873 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:36.978960 15567 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:36.947043 20953 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:36.980070 20953 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:36.980700 20953 raft_consensus.cc:2764] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:20:36.948035 20899 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:36.982761 20899 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:36.950888 20585 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:36.985026 20722 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:36.986091 20955 consensus_queue.cc:1059] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:36.997839 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:36.999262 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:36.999887 15522 raft_consensus.cc:1240] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:37.000053 15524 raft_consensus.cc:2393] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 15: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:37.002831 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:37.003688 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:37.004624 15522 raft_consensus.cc:2393] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 13: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:36.951375 20984 raft_consensus.cc:740] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:37.006006 20984 consensus_queue.cc:260] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:37.009166 20984 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:36.718200 15566 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:36.976176 20867 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:36.961090 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:37.011982 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 293868us
I20260709 13:20:37.013255 15566 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:37.014580 15566 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:37.012516 20791 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.015964 20949 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:37.020447 15672 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:37.044521 15677 raft_consensus.cc:1240] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:37.020812 15525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2345) took 1005 ms. Trace:
I20260709 13:20:37.046708 15525 rpcz_store.cc:276] 0709 13:20:36.015692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.015851 (+   159us) service_pool.cc:224] Handling call
0709 13:20:36.016146 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:36.568386 (+552240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:37.020592 (+452206us) spinlock_profiling.cc:243] Waited 552 ms on lock 0x7b54001748d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:37.020758 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1268686720}
I20260709 13:20:37.021051 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:37.021883 15669 raft_consensus.cc:1240] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:37.023821 20622 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:37.024899 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:37.055617 15675 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 19.
I20260709 13:20:37.025831 15673 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:20:37.026433 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:37.030905 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:37.031780 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:37.012189 20953 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:37.033507 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:36.932771 20774 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:36.907006 20884 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.040704 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:37.041723 15674 raft_consensus.cc:1240] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:37.042232 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:37.042884 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:37.048025 15671 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
I20260709 13:20:37.014492 20982 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:37.056835 15676 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
I20260709 13:20:37.068866 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:37.071419 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:37.073480 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:37.074440 15670 raft_consensus.cc:2393] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 15: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:37.068181 16203 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:37.077035 15597 raft_consensus.cc:2393] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 16: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:37.079923 20452 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:37.088409 20452 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.088944 20784 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:37.090229 20975 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.055330 20955 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:37.050843 20608 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:37.090788 20975 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.095929 20955 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:37.121073 20955 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:37.125378 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 57271.2us
W20260709 13:20:37.054723 15412 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 582343936346420b87ecf2eebd00dc25 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica dd1a14c3173d49dc9b55bb116e3783c9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:20:37.140954 15412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 86391.5us
I20260709 13:20:37.166950 16232 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:20:37.064508 20791 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.142393 20569 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:37.397068 20693 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.371771 16236 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:20:37.169611 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:37.342549 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:37.523533 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:37.220523 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:37.487404 16234 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
W20260709 13:20:37.525143 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:37.205358 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:37.140246 20986 log.cc:927] Time spent T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:20:37.122335 20920 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.234730 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:37.547032 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:37.548271 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 411055us
I20260709 13:20:37.414057 20911 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.552204 16239 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
W20260709 13:20:37.570289 15601 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.018s	sys 0.023s
W20260709 13:20:37.570838 15601 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.018s	sys 0.023s
I20260709 13:20:37.570282 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:37.141764 20897 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:37.220813 20452 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:37.527352 20884 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:37.583320 15596 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.002s	sys 0.021s
I20260709 13:20:37.531098 20895 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:37.123685 20953 consensus_queue.cc:1059] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.426s
I20260709 13:20:37.529783 20608 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:20:37.585131 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:37.370515 20975 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:37.416153 20774 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:37.531072 20569 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:37.414973 20887 consensus_queue.cc:1059] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.178s
I20260709 13:20:37.589509 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:37.586040 15596 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.003s	sys 0.021s
I20260709 13:20:37.588855 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:37.607317 15674 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:37.584384 20577 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.415192 20821 consensus_queue.cc:1059] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.174s
I20260709 13:20:37.613587 15595 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 6.
I20260709 13:20:37.613044 16238 raft_consensus.cc:1813] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Handling vote request from an unknown peer 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:37.265157 20973 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.617970 16238 raft_consensus.cc:2368] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7.
W20260709 13:20:37.619065 15600 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.027s	sys 0.020s
W20260709 13:20:37.611763 20900 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.018s	sys 0.027s
I20260709 13:20:37.605422 20350 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.582509 20722 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:20:37.560297 20693 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.585098 20911 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:37.620401 15600 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.027s	sys 0.020s
I20260709 13:20:37.602950 20883 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.613329 20569 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:37.582453 20920 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.622680 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:37.585706 20897 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.637607 16231 raft_consensus.cc:1240] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:37.610538 20949 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:37.604311 20895 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:37.619065 20982 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.638495 16238 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:20:37.607698 20608 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:37.620697 20900 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.018s	sys 0.027s
I20260709 13:20:37.618309 20577 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.638259 20984 raft_consensus.cc:3055] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:37.652873 16232 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:20:37.638842 20873 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:37.616811 20774 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.635804 20350 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.639918 20883 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:37.655716 20984 raft_consensus.cc:740] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:37.670274 15601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 2215) took 1012 ms. Trace:
I20260709 13:20:37.646171 20949 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.651293 20982 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.674695 15601 rpcz_store.cc:276] 0709 13:20:36.658252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.658368 (+   116us) service_pool.cc:224] Handling call
0709 13:20:36.658633 (+   265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:36.677184 (+ 18551us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:36.677216 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:36.677697 (+   481us) log.cc:844] Serialized 26 byte log entry
0709 13:20:37.571262 (+893565us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:37.571308 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:37.571397 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:37.670056 (+ 98659us) raft_consensus.cc:1685] finished
0709 13:20:37.670076 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:37.670209 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":63634,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":150,"spinlock_wait_cycles":50944,"thread_start_us":639678,"threads_started":1,"wal-append.queue_time_us":945197}
I20260709 13:20:37.662772 20975 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:37.671360 20693 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:37.660751 20873 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:37.688231 20975 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.710793 20791 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:37.746549 16233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57258 (request call id 1148) took 1589 ms. Trace:
I20260709 13:20:37.750191 16233 rpcz_store.cc:276] 0709 13:20:36.156730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.156850 (+   120us) service_pool.cc:224] Handling call
0709 13:20:36.157087 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:36.238606 (+ 81519us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:20:36.323255 (+ 84649us) log.cc:844] Serialized 14 byte log entry
0709 13:20:36.323558 (+   303us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:36.323608 (+    50us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:36.323656 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:37.044135 (+720479us) spinlock_profiling.cc:243] Waited 8.6 ms on lock 0x7b18003dbd60. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce18ec2a6 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419
0709 13:20:37.746319 (+702184us) spinlock_profiling.cc:243] Waited 81.3 ms on lock 0x7b54004efa54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:37.746374 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:37.746493 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1169,"prepare.queue_time_us":191817,"prepare.run_cpu_time_us":2036,"prepare.run_wall_time_us":12774,"raft.queue_time_us":81,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":87106,"spinlock_wait_cycles":206724864,"thread_start_us":67692,"threads_started":1}
I20260709 13:20:37.731079 20693 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.731330 20981 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:37.746251 20821 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.746511 20920 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:37.705690 20984 consensus_queue.cc:260] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.759359 20973 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:37.772513 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 160397us
I20260709 13:20:37.759537 20321 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:37.758881 20791 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:37.793749 20321 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.795060 20873 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:37.796446 20321 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:37.798154 20998 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:37.798753 20998 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.800004 20569 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.800912 20998 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:37.801210 16240 raft_consensus.cc:2393] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 20: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:37.801509 20973 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:37.771915 20895 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:37.802242 20895 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.771793 20999 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:37.802987 20999 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.806270 20569 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:37.805163 20999 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:37.807353 20999 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.761830 20693 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.808234 20999 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.809540 20693 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:37.810587 20949 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:37.811038 20350 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:37.811668 20990 log.cc:927] Time spent T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:20:37.774183 20608 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:37.812075 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:37.789309 20897 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:37.811833 20982 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:37.812148 20984 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:37.814163 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:37.769412 20911 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:37.769702 20920 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.830933 20973 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.833196 20973 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:37.802696 20895 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.836913 20895 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:37.818019 20350 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.819600 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:37.819765 20722 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:37.838934 20350 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.820968 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:37.821532 20321 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:37.810948 20999 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:37.823724 20622 consensus_meta.cc:287] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.754s	user 0.007s	sys 0.003s
I20260709 13:20:37.824555 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:37.879556 15594 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:37.880394 20339 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.880889 20339 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.884172 20339 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:37.893088 20874 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.893658 20874 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.894755 20883 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:37.895012 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:37.895900 20874 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:37.896492 20874 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:37.897099 20874 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.828477 20975 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:37.897609 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:37.767561 20821 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.899479 15525 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:37.899593 20821 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:37.814417 20997 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:37.900518 15601 raft_consensus.cc:2393] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 13: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:37.900259 20997 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.792785 20995 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:37.902464 20995 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.904635 20995 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:37.906265 20997 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:37.909173 20995 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.910099 20995 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:37.847324 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:37.840070 20873 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.911928 20873 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.815272 20791 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.914325 20874 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:37.915427 20791 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:37.857734 20350 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:37.916486 20873 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:37.860703 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:37.861235 16203 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:37.839600 20722 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.839246 20577 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:37.827497 20955 consensus_meta.cc:287] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.706s	user 0.007s	sys 0.003s
I20260709 13:20:37.855847 20321 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.922235 20350 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:37.922672 20350 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:37.923185 20722 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:37.924077 20955 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.925297 15602 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 18: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:37.923099 20350 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:37.926302 20955 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 18 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:37.866964 20998 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.896553 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:37.827075 20756 raft_consensus.cc:3055] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:37.927728 20756 raft_consensus.cc:740] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:37.928614 20756 consensus_queue.cc:260] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:37.763623 20981 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:37.931648 20981 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:37.772150 20926 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:37.912292 20995 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:37.921670 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 60528.8us
I20260709 13:20:37.932359 20926 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.933161 16203 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:37.934176 20722 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:37.934571 20722 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:37.935745 20756 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:37.935309 20722 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:37.923672 20321 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:37.938549 20321 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:37.938804 20926 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:37.923769 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:37.939574 15673 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:37.941264 20321 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.934558 20998 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:37.943576 20321 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:37.906351 15598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 2210) took 1251 ms. Trace:
I20260709 13:20:37.825569 20774 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:37.944267 20998 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:37.844429 20920 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:37.944267 15598 rpcz_store.cc:276] 0709 13:20:36.654617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.655464 (+   847us) service_pool.cc:224] Handling call
0709 13:20:36.655753 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:36.656044 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:36.656071 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:36.656417 (+   346us) log.cc:844] Serialized 26 byte log entry
0709 13:20:37.810098 (+1153681us) spinlock_profiling.cc:243] Waited 29.8 ms on lock 0x7b18001ebec0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce18ec2a6 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9
0709 13:20:37.858627 (+ 48529us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b18001ebec0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab0d1 00007f5ce49c6a0f
0709 13:20:37.906020 (+ 47393us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:37.906060 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:37.906127 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:37.906171 (+    44us) raft_consensus.cc:1685] finished
0709 13:20:37.906181 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:37.906295 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":94102,"raft.queue_time_us":69,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":6624,"spinlock_wait_cycles":103611520,"thread_start_us":47186,"threads_started":1,"wal-append.queue_time_us":1217214}
I20260709 13:20:37.947707 20997 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:37.947670 20920 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:37.948974 20722 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.950392 20997 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:37.953169 15642 leader_election.cc:400] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7.
I20260709 13:20:37.953492 15642 leader_election.cc:403] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:37.954063 20997 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:20:37.954419 20722 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:37.955747 20991 log.cc:927] Time spent T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
I20260709 13:20:37.957881 20801 raft_consensus.cc:3055] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:37.958452 20801 raft_consensus.cc:740] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:37.959265 20801 consensus_queue.cc:260] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:37.960413 20801 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:37.961496 20997 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7.
I20260709 13:20:37.962622 15642 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:37.953195 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:37.964093 21015 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:37.965523 21015 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:37.965881 21015 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:37.967355 15670 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:37.977387 20722 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.977988 20722 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:37.978894 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:37.980988 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:37.981635 15526 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:37.981841 20981 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.982406 20981 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:37.983466 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:37.990214 20981 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:37.994545 20722 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:37.995888 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 1.01782e+06us
I20260709 13:20:37.996795 15568 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:37.997651 20981 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:37.998098 15568 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:37.999548 20756 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:38.001192 20756 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:37.999850 15568 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:38.006999 20955 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:38.007412 20955 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:38.007803 20955 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:38.010619 15568 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:38.011480 20821 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:38.014278 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:38.014911 15668 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
I20260709 13:20:38.016199 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:38.017304 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:38.010950 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:38.018527 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:38.019681 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:38.020417 15526 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 17.
I20260709 13:20:38.024726 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:38.025534 15675 raft_consensus.cc:1275] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:20:38.024731 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:38.024950 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:38.026923 15677 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:20:38.027631 15671 raft_consensus.cc:1275] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:20:38.028000 15671 pending_rounds.cc:85] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting all ops after (but not including) 6
I20260709 13:20:38.028308 15671 pending_rounds.cc:107] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:20:38.028002 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:38.028558 15671 raft_consensus.cc:2889] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:38.028832 15677 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:38.029960 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:38.030963 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:38.031672 15677 raft_consensus.cc:2393] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 11: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:38.031926 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:38.034114 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:38.034227 16238 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:38.036563 16203 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:38.037815 15566 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:38.023747 15673 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:20:38.047729 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:38.048230 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:38.048908 15595 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
I20260709 13:20:38.050868 20955 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:38.051630 20821 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:38.053393 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:38.001801 20756 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:38.056486 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:38.057331 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:38.057740 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:38.058554 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:38.059005 16232 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:38.059387 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:38.060281 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:38.060480 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:38.060894 16240 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:38.061326 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:38.062042 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:38.062086 15677 raft_consensus.cc:2393] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 16: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:38.061597 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:38.063246 15526 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:20:38.064206 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:38.065582 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:38.065780 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:38.065733 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:38.066372 15600 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:38.067534 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:38.064787 15599 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:20:38.068238 15603 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 18: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:38.035703 15426 catalog_manager.cc:5697] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 10 to 11, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 11 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:38.068571 15602 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:20:38.069033 15602 pending_rounds.cc:85] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting all ops after (but not including) 9
I20260709 13:20:38.069275 15602 pending_rounds.cc:107] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:20:38.069533 15602 raft_consensus.cc:2889] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:38.068166 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:38.072686 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:38.074126 15597 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:20:38.074146 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:38.074947 15595 raft_consensus.cc:1813] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Handling vote request from an unknown peer 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:38.075212 16238 raft_consensus.cc:2393] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 7: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:38.075420 15595 raft_consensus.cc:2393] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 6: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:38.076002 15601 raft_consensus.cc:2410] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:20:38.074159 15671 raft_consensus.cc:1240] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:38.080102 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:38.080601 15671 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:20:38.083863 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:38.084082 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:38.084668 16233 raft_consensus.cc:2393] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 17: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:38.085240 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:38.087759 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:38.088465 16240 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 5.
I20260709 13:20:38.090950 16205 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:38.093005 15566 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:38.094002 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:38.091965 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:38.095829 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:38.099483 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:38.099640 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:38.062050 20984 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:38.067691 20577 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:38.104457 15568 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:38.105515 16236 raft_consensus.cc:2393] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 18: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:38.106026 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:38.107678 15673 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:38.068552 20821 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:38.115497 15568 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:38.113344 15671 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 18.
I20260709 13:20:38.077486 20981 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:38.122750 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:38.123448 15526 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:38.118091 15426 catalog_manager.cc:5697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:38.127568 16240 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:20:38.126658 15426 catalog_manager.cc:5697] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 9 to 11, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 11 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:38.100622 20947 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:20:38.132663 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 50946us
I20260709 13:20:38.128270 16240 raft_consensus.cc:606] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:38.129686 15426 catalog_manager.cc:5697] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 9 to 12, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 12 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:38.143707 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:38.116408 20577 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:38.145246 15426 catalog_manager.cc:5697] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:38.148605 15642 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:38.122023 20953 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:38.151580 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:38.152335 15526 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 17.
I20260709 13:20:38.152529 16203 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:38.152609 15426 catalog_manager.cc:5697] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:38.154088 16203 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:38.155460 15426 catalog_manager.cc:5697] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:38.157667 16203 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:38.127637 20801 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:38.160722 16203 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:38.114604 20982 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.035s
I20260709 13:20:38.165187 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:38.144881 20947 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:38.129096 21002 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:20:38.121627 21001 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:38.148624 20577 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:38.150835 20873 raft_consensus.cc:2764] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:20:38.128960 20981 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:38.121976 20821 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:38.139801 20984 consensus_queue.cc:1059] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:20:38.167930 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:38.184873 16237 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:20:38.185240 20981 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:38.170080 20947 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:38.165146 16203 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:38.159811 20953 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:38.161257 20884 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:38.165364 20900 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:38.157302 15426 catalog_manager.cc:5697] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:38.186411 16237 raft_consensus.cc:606] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:38.165496 20973 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:38.190937 20953 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:38.195298 15596 raft_consensus.cc:1275] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:20:38.195709 20569 raft_consensus.cc:2804] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:38.196192 20569 raft_consensus.cc:697] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:20:38.176056 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 60624.5us
I20260709 13:20:38.193594 15426 catalog_manager.cc:5697] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:38.197080 20569 consensus_queue.cc:237] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:38.198129 15426 catalog_manager.cc:5697] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 10 to 13, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 13 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:38.199638 15426 catalog_manager.cc:5697] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 5 to 7, term changed from 5 to 6, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65), VOTER 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) evicted. New cstate: current_term: 6 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:38.161932 20801 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:38.177982 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:38.195163 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 87556.6us
I20260709 13:20:38.202450 15603 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:38.203159 15566 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:38.179890 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:38.183269 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:38.163726 20980 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:38.187278 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:38.188083 16203 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:38.203949 15601 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:20:38.161520 20722 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:20:38.201293 15426 catalog_manager.cc:5697] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:38.163125 20972 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:38.216138 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:38.216295 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:38.216946 15525 raft_consensus.cc:2393] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 10: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:38.187193 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:38.217559 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:38.207086 16203 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:38.204622 20801 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:38.198163 21014 consensus_queue.cc:1059] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:20:38.209182 20821 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:38.209678 15594 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:38.211589 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:38.211989 15426 catalog_manager.cc:5697] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:38.230458 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:38.230378 15426 catalog_manager.cc:5697] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 6 to 8, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 8 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:38.232498 15426 catalog_manager.cc:5697] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:38.186550 21001 raft_consensus.cc:697] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:20:38.247188 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 50276.1us
I20260709 13:20:38.247432 21001 consensus_queue.cc:237] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:38.250478 15425 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49144 (request call id 353) took 16283 ms (client timeout 15000 ms). Trace:
I20260709 13:20:38.251137 20350 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:38.252049 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 48949us
I20260709 13:20:38.252731 15566 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:38.253420 20350 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:38.215165 20791 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:38.223225 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:38.231196 15526 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:38.224299 20887 raft_consensus.cc:3055] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:38.226438 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:38.255256 15523 raft_consensus.cc:1813] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Handling vote request from an unknown peer 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:38.255702 15523 raft_consensus.cc:2368] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7.
I20260709 13:20:38.213488 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:38.221625 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:38.193295 20884 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:38.260033 15491 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:38.260651 15522 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:20:38.198037 20900 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:38.234261 20972 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:38.242899 15520 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:20:38.251494 15589 tablet_service.cc:1558] Processing DeleteTablet for tablet b99eaeae24b44aa78cfa8c90caca288e with delete_type TABLET_DATA_TOMBSTONED (TS 5498ef3e89a941e39c25f68cf64e6b0f not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:53300
I20260709 13:20:38.256100 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:38.256538 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:38.256908 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:38.234236 15426 catalog_manager.cc:5697] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:38.258908 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:38.260901 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:38.284960 15671 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:20:38.285068 15673 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:38.256422 15597 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:20:38.263796 16204 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:38.286546 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:38.210058 21010 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:38.206694 20980 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:38.267468 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:38.289096 15602 raft_consensus.cc:2393] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 16: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:38.271091 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:38.284041 15426 catalog_manager.cc:5697] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 3 to 9, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 9 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:38.165129 21015 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:38.280560 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:38.290611 16236 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:20:38.272935 15522 raft_consensus.cc:1275] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:20:38.280436 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:38.292420 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:38.293020 15520 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 6.
W20260709 13:20:38.250881 15425 rpcz_store.cc:269] 0709 13:20:21.967414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:21.967755 (+   341us) service_pool.cc:224] Handling call
0709 13:20:38.250367 (+16282612us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:34.250126 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:34.402122 (+151996us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:34.402170 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:20:34.402188 (+    18us) tablet.cc:662] Decoding operations
0709 13:20:34.522465 (+120277us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:34.522564 (+    99us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:34.522595 (+    31us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:20:34.523958 (+  1363us) tablet.cc:701] Row locks acquired
0709 13:20:34.524009 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:20:34.524177 (+   168us) write_op.cc:270] Start()
0709 13:20:34.524319 (+   142us) write_op.cc:276] Timestamp: P: 1783603234524111 usec, L: 0
0709 13:20:34.524341 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:20:34.525097 (+   756us) log.cc:844] Serialized 7101 byte log entry
0709 13:20:35.409266 (+884169us) op_driver.cc:464] REPLICATION: finished
0709 13:20:35.675528 (+266262us) write_op.cc:301] APPLY: starting
0709 13:20:35.675643 (+   115us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:35.778048 (+102405us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:35.779152 (+  1104us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:35.878499 (+ 99347us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:35.878879 (+   380us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=0,mrs_lookups=0
0709 13:20:35.878926 (+    47us) write_op.cc:312] APPLY: finished
0709 13:20:35.879388 (+   462us) log.cc:844] Serialized 164 byte log entry
0709 13:20:35.905251 (+ 25863us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:35.905420 (+   169us) write_op.cc:454] Released schema lock
0709 13:20:35.905499 (+    79us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:35.905960 (+   461us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:35.905996 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7265,"reported_tablets":46,"spinlock_wait_cycles":31488,"child_traces":[["op",{"apply.queue_time_us":250819,"apply.run_cpu_time_us":24516,"apply.run_wall_time_us":230424,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79511,"num_ops":18,"prepare.queue_time_us":172011,"prepare.run_cpu_time_us":25224,"prepare.run_wall_time_us":425601,"raft.queue_time_us":255270,"raft.run_cpu_time_us":68194,"raft.run_wall_time_us":274825,"replication_time_us":838076,"thread_start_us":502951,"threads_started":4,"wal-append.queue_time_us":236042,"wal-append.run_cpu_time_us":65063,"wal-append.run_wall_time_us":2135125}]]}
I20260709 13:20:38.299578 15674 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
I20260709 13:20:38.260363 21014 consensus_queue.cc:1243] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer d3c4f5bd76e0445795bad5f6dd93ff69 log is divergent from this leader: its last log entry 8.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:20:38.265509 21022 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:38.309150 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:38.310180 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:38.311065 15525 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:20:38.312901 15525 raft_consensus.cc:1240] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:38.261806 20664 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:38.262660 20887 raft_consensus.cc:740] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:38.303735 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:38.266414 20884 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:38.314234 15670 raft_consensus.cc:2368] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 13. Current term is 14.
I20260709 13:20:38.315513 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:38.319401 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:38.303006 15526 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:20:38.324381 15528 raft_consensus.cc:2393] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 10: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:38.315399 15596 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:20:38.242899 21008 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:20:38.320459 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 56741.5us
I20260709 13:20:38.319322 15568 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:38.321035 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:38.321990 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:38.327446 16232 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 15.
I20260709 13:20:38.321993 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:38.328279 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:38.304446 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 155911us
I20260709 13:20:38.322304 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:38.275591 20984 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:38.331315 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:38.275856 21009 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:38.245751 20577 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:38.324231 15522 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
I20260709 13:20:38.325314 15493 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:38.291692 21026 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:38.295147 20980 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:38.334875 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:38.336433 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:38.337425 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:38.338933 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:38.265563 20926 consensus_queue.cc:1048] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:20:38.311286 21015 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:38.342928 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:38.374684 15566 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:20:38.373591 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 46902.8us
I20260709 13:20:38.289605 20905 consensus_queue.cc:1048] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:20:38.326391 21022 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:38.381492 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:38.343933 20643 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:38.330215 20949 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:38.384089 16239 raft_consensus.cc:2393] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 12: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:38.382309 16233 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 5.
I20260709 13:20:38.334534 21009 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:38.386942 15493 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:38.387003 16203 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:38.388648 20984 raft_consensus.cc:697] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:38.388981 21037 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:38.389526 21037 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:38.389788 20984 consensus_queue.cc:237] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:38.390795 15493 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:20:38.377053 21017 log.cc:927] Time spent T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:20:38.344220 21010 raft_consensus.cc:697] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:38.376094 20995 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:20:38.380164 21034 log.cc:927] Time spent T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.071s	user 0.000s	sys 0.011s
I20260709 13:20:38.334668 20953 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:38.392406 21009 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:38.456032 21033 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:38.379196 20664 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.076s
I20260709 13:20:38.382753 20643 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:38.478612 15598 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 16.
I20260709 13:20:38.383994 21022 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:38.333715 21008 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:38.457262 20995 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:38.456629 20821 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:38.463984 20949 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:38.344327 21026 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:38.480490 20883 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:38.478226 21033 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:38.527478 20339 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:38.483763 20643 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:38.463249 20774 consensus_queue.cc:1048] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:20:38.529981 20339 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:38.549676 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 151757us
I20260709 13:20:38.553426 16204 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:38.506645 20995 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:38.560072 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:38.503576 21037 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:38.521620 20920 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:38.521288 20999 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:38.394337 20887 consensus_queue.cc:260] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:38.527892 20883 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:38.574664 16237 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:38.568256 20339 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:38.567888 20999 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:38.536880 21033 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:38.566512 20920 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:38.522035 20756 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:38.525534 20821 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:38.577966 20883 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:38.581326 20900 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:38.581712 20999 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:38.617892 15600 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 15.
I20260709 13:20:38.594445 20920 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:38.597015 21037 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:38.638314 20981 raft_consensus.cc:3055] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:38.638887 20981 raft_consensus.cc:740] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:38.639986 20981 consensus_queue.cc:260] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:38.597522 21035 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:38.628734 20955 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:38.634590 21043 tablet_replica.cc:333] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
W20260709 13:20:38.659570 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 106243us
I20260709 13:20:38.673365 20905 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:38.674090 20905 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:38.683548 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:38.687704 16204 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:38.571597 21010 consensus_queue.cc:237] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:38.704480 16204 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:38.638360 21037 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:38.703651 21035 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:38.707335 21037 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:38.712646 20997 raft_consensus.cc:3055] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:38.713093 20997 raft_consensus.cc:740] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:38.713907 20997 consensus_queue.cc:260] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:38.714902 20997 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:38.720027 20897 raft_consensus.cc:993] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9: : Instructing follower d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
W20260709 13:20:38.727847 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 337077us
I20260709 13:20:38.737448 16204 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:38.689076 20577 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:38.695597 20949 raft_consensus.cc:697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:38.740877 15493 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:38.746822 16204 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:38.726493 20953 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:38.751006 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:20:38.751684 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 377054us
I20260709 13:20:38.751653 16239 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 14
W20260709 13:20:38.746052 20801 consensus_meta.cc:287] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.517s	user 0.011s	sys 0.005s
I20260709 13:20:38.752643 15493 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:38.744674 20350 raft_consensus.cc:2804] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:38.746105 20973 raft_consensus.cc:1081] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Signalling peer d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:20:38.757680 16204 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:38.747130 20722 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:38.762953 16204 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:38.764396 16204 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:38.772902 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:38.775728 21035 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:38.776281 21035 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:38.776772 20643 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:38.780299 21053 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:38.786410 20980 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:38.789019 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:38.789738 15522 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:38.790120 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:38.780375 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:38.790920 15526 raft_consensus.cc:2393] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 15: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:38.757016 20350 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:38.755331 21044 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:38.792394 20350 raft_consensus.cc:3060] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:38.793334 21043 raft_consensus.cc:2243] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:38.794303 21043 raft_consensus.cc:2272] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:38.797439 21043 ts_tablet_manager.cc:1916] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:38.798077 21044 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:38.798516 21044 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:38.756634 20801 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:38.800904 20801 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:38.801312 20643 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:38.758755 20926 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:38.804066 20756 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:38.804131 20926 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:38.804924 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:38.804759 20926 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:38.810032 20821 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:20:38.811844 20905 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:38.813642 20947 consensus_meta.cc:287] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.608s	user 0.007s	sys 0.003s
I20260709 13:20:38.814121 20947 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:38.814713 15524 raft_consensus.cc:1240] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:38.948522 20947 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:38.776701 21035 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:39.230294 16239 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:20:39.276396 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:38.764971 21026 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:39.547317 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:38.762322 21051 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:38.765633 21008 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:39.540514 20995 consensus_meta.cc:287] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.971s	user 0.011s	sys 0.001s
W20260709 13:20:39.532852 21044 consensus_meta.cc:287] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.734s	user 0.005s	sys 0.004s
W20260709 13:20:39.436120 21009 consensus_meta.cc:287] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.950s	user 0.009s	sys 0.011s
W20260709 13:20:39.430617 20920 consensus_meta.cc:287] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.787s	user 0.008s	sys 0.001s
W20260709 13:20:39.534514 21022 consensus_meta.cc:287] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 1.029s	user 0.009s	sys 0.004s
I20260709 13:20:39.321638 21002 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:39.499812 20980 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:38.758931 21054 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:20:39.454155 21041 log.cc:927] Time spent T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 13:20:39.509713 20884 consensus_meta.cc:287] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 1.168s	user 0.008s	sys 0.012s
W20260709 13:20:39.519923 21053 consensus_meta.cc:287] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.739s	user 0.007s	sys 0.000s
I20260709 13:20:39.295397 20897 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:39.554157 21033 consensus_meta.cc:287] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.956s	user 0.010s	sys 0.000s
I20260709 13:20:39.295116 20947 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:39.321930 20577 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:39.554484 20973 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:39.554848 20569 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
W20260709 13:20:39.563767 21001 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.168s	user 0.009s	sys 0.009s
I20260709 13:20:39.275126 20949 consensus_queue.cc:237] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:39.581193 21009 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:39.585063 20926 consensus_meta.cc:287] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.778s	user 0.010s	sys 0.001s
W20260709 13:20:39.586628 21001 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.201s	user 0.009s	sys 0.009s
I20260709 13:20:39.583233 21054 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:39.587006 21053 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:39.572747 20995 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:39.591339 21038 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.010s	sys 0.015s
I20260709 13:20:39.573176 21044 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:39.581418 20920 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:39.582310 21022 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:39.583503 20884 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:39.586234 20973 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:39.618045 21038 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.011s	sys 0.015s
I20260709 13:20:39.593236 20577 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:39.627104 21035 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:39.627596 20995 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:39.586449 20569 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:39.632252 16237 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:20:39.585712 21033 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:39.634845 16238 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53976 (request call id 1173) took 1547 ms. Trace:
I20260709 13:20:39.585834 20947 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:39.589821 20897 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:39.634407 20973 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:39.588106 21002 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:39.612775 21054 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:39.660341 16236 raft_consensus.cc:2393] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 18: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:39.605465 20926 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:39.659796 16238 rpcz_store.cc:276] 0709 13:20:38.087307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.089179 (+  1872us) service_pool.cc:224] Handling call
0709 13:20:38.089511 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:38.091557 (+  2046us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:38.091590 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:38.098681 (+  7091us) log.cc:844] Serialized 26 byte log entry
0709 13:20:38.576388 (+477707us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:38.576429 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:38.576493 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:39.634582 (+1058089us) raft_consensus.cc:1685] finished
0709 13:20:39.634601 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:39.634781 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2057,"raft.queue_time_us":2169,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":6824,"thread_start_us":436195,"threads_started":1,"wal-append.queue_time_us":629862}
I20260709 13:20:39.667701 20577 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:39.660635 20997 consensus_meta.cc:287] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.943s	user 0.010s	sys 0.001s
I20260709 13:20:39.670110 16235 raft_consensus.cc:2393] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 13: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:39.667711 20569 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:39.679492 20883 consensus_meta.cc:287] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 1.056s	user 0.008s	sys 0.002s
I20260709 13:20:39.679170 20577 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:39.678580 21051 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:39.680058 20973 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:39.694270 20947 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:39.702411 20920 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:39.708712 21044 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:39.695474 21051 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:39.710063 20926 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:39.696382 20973 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:39.710569 21035 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:39.711318 21033 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:39.692049 20883 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:39.712421 20973 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:39.714685 21051 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:39.698745 21009 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:39.695047 20577 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:39.717437 20883 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:39.706046 20897 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:39.722132 20350 consensus_meta.cc:287] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.929s	user 0.007s	sys 0.004s
I20260709 13:20:39.722609 20350 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:39.726527 21002 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:39.728047 15520 raft_consensus.cc:2393] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 12: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:39.728482 20577 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:39.735364 21043 ts_tablet_manager.cc:1929] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 7.7
I20260709 13:20:39.735965 21043 log.cc:1199] T b99eaeae24b44aa78cfa8c90caca288e P 5498ef3e89a941e39c25f68cf64e6b0f: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-1-root/wals/b99eaeae24b44aa78cfa8c90caca288e
I20260709 13:20:39.737015 21043 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53300 (request call id 108) took 1487 ms. Trace:
I20260709 13:20:39.737257 21043 rpcz_store.cc:276] 0709 13:20:38.249786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.251192 (+  1406us) service_pool.cc:224] Handling call
0709 13:20:38.625521 (+374329us) ts_tablet_manager.cc:1151] Deleting tablet b99eaeae24b44aa78cfa8c90caca288e
0709 13:20:38.634219 (+  8698us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:20:39.726197 (+1091978us) tablet_metadata.cc:665] Metadata flushed
0709 13:20:39.735204 (+  9007us) tablet_metadata.cc:665] Metadata flushed
0709 13:20:39.736932 (+  1728us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":288933,"thread_start_us":234528,"threads_started":1}
I20260709 13:20:39.750110 20350 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:39.748174 21054 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:39.756690 21054 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:40.185266 20350 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:40.185971 20350 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:40.188271 20350 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:40.307474 20350 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:40.308136 20350 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:40.310699 20350 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:40.354686 20350 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:40.355363 20350 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:40.357769 20350 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:40.472496 20350 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:40.473166 20350 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:40.475512 20350 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:40.472398 21071 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:40.515452 21071 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:40.560428 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 1.73305e+06us
W20260709 13:20:40.589783 16235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57244 (request call id 1586) took 1785 ms (client timeout 2056 ms). Trace:
W20260709 13:20:40.590512 16235 rpcz_store.cc:269] 0709 13:20:38.804498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.804644 (+   146us) service_pool.cc:224] Handling call
0709 13:20:40.589417 (+1784773us) spinlock_profiling.cc:243] Waited 863 ms on lock 0x7b540051c554. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:40.589681 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1984329984}
I20260709 13:20:40.517848 21071 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:40.656185 21023 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:40.657068 21023 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:40.659446 21023 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:40.668198 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:40.670112 21023 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:40.670811 21023 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:40.672986 21023 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:40.689009 16236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57258 (request call id 1166) took 2369 ms (client timeout 2236 ms). Trace:
W20260709 13:20:40.689674 16236 rpcz_store.cc:269] 0709 13:20:38.319891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.320791 (+   900us) service_pool.cc:224] Handling call
0709 13:20:40.688580 (+2367789us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54001cfcd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:40.688811 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3065533184}
W20260709 13:20:40.783067 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:40.783810 15567 connection.cc:256] RPC call timeout handler was delayed by -1.540829561999999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.163s for a response
W20260709 13:20:40.784276 15567 connection.cc:256] RPC call timeout handler was delayed by -1.4698376459998599s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1938s for a response
I20260709 13:20:40.786053 21023 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:40.800374 21023 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:40.802601 21023 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:40.803213 21023 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:40.803767 21023 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:40.808375 21073 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:40.809240 21073 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:40.811841 21073 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:40.812071 20955 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:40.812541 21073 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:40.812683 20955 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:40.813091 21073 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:40.815562 21073 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:40.817426 21023 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:40.818198 20955 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:40.834702 15567 connection.cc:256] RPC call timeout handler was delayed by -1.1646657979999873s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18710000000000002s for a response
I20260709 13:20:40.837133 20955 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:40.837709 20955 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:40.838529 20972 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:40.840071 20955 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:40.840710 20955 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:40.841178 20955 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:40.842716 21074 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:40.843349 21074 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:40.843690 21022 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:40.845268 20955 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:40.848678 15568 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:40.850930 21022 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:40.851648 21022 raft_consensus.cc:697] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:40.852607 21022 consensus_queue.cc:237] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:40.857438 15568 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:40.850030 21074 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:40.860316 20577 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:40.863377 20577 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:40.863897 21076 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:40.864476 21076 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:40.865687 20577 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:40.860960 21054 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:40.866891 21054 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:40.869330 21054 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:40.861531 21048 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:40.870143 21048 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:40.872402 21048 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:40.862115 21072 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:40.873189 21072 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:40.875504 21072 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:40.875626 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:40.862661 21075 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:40.872534 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:40.877401 21075 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:40.880046 21075 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:40.884045 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:40.884940 15525 raft_consensus.cc:2393] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 11: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:40.886430 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:40.889716 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:40.890516 16236 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:40.891431 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:40.891774 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:40.892432 15524 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:20:40.893240 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:40.893697 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:40.894232 15520 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2366) took 2593 ms (client timeout 1567 ms). Trace:
I20260709 13:20:40.894409 15528 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:20:40.893986 15524 raft_consensus.cc:2393] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 15: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:40.895031 15525 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:40.895877 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:40.893535 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:40.896078 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:40.897223 15524 raft_consensus.cc:2393] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 11: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:40.897989 15527 raft_consensus.cc:2393] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 15: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:20:40.894694 15520 rpcz_store.cc:269] 0709 13:20:38.300727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.303494 (+  2767us) service_pool.cc:224] Handling call
0709 13:20:40.893997 (+2590503us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400070dd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:40.894131 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3236786432}
I20260709 13:20:40.900506 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:40.901039 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:40.900225 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:40.901249 16235 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 5.
I20260709 13:20:40.903198 21023 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:40.903929 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:40.903841 21076 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:40.904686 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:40.905157 20884 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:40.908914 21023 raft_consensus.cc:697] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:40.910460 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:40.911381 15528 raft_consensus.cc:2410] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:20:40.911988 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:40.912139 15493 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:40.912679 16236 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:20:40.913131 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:40.915318 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:40.916668 15527 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:40.918123 21038 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:40.919169 15568 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:40.921340 21076 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:40.921774 21076 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:40.922214 21076 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:40.910454 21023 consensus_queue.cc:237] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:40.926128 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:40.928184 15493 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:40.929632 21038 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:40.938109 21038 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:40.938505 21038 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:40.941532 15567 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.581s (SENT) [suppressed 6 similar messages]
I20260709 13:20:40.909756 15525 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
W20260709 13:20:41.030557 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 89055.7us
I20260709 13:20:41.034096 15568 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:41.035843 21073 raft_consensus.cc:2804] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:41.036316 21073 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:41.036724 21073 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:41.037256 15568 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:41.042339 21080 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:41.042834 21080 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:41.043181 21080 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:41.045797 21073 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.049005 21073 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:41.058791 21080 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:41.059865 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 129498us
W20260709 13:20:41.061630 15491 connection.cc:256] RPC call timeout handler was delayed by -2.0094438230000833s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16620000000000001s for a response
W20260709 13:20:41.068949 15491 connection.cc:256] RPC call timeout handler was delayed by -1.4523442350000551s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1512s for a response
W20260709 13:20:41.069386 15491 connection.cc:256] RPC call timeout handler was delayed by -1.4388442350000332s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
W20260709 13:20:41.069689 15491 connection.cc:256] RPC call timeout handler was delayed by -1.4145442349999939s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1554s for a response
W20260709 13:20:41.070003 15491 connection.cc:256] RPC call timeout handler was delayed by -1.4073442349999823s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1562s for a response
W20260709 13:20:41.070283 15491 connection.cc:256] RPC call timeout handler was delayed by -1.3677442349999183s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16060000000000002s for a response
I20260709 13:20:41.075861 21080 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:41.083217 20982 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:41.086324 21073 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:41.086547 20982 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.086933 21073 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.091042 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.091308 20982 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:41.091926 20982 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:41.092198 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:41.092429 20982 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.092804 15521 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:56840
I20260709 13:20:41.093647 15521 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:41.094008 15521 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:41.095173 21077 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:41.095638 20982 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:41.095687 21077 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:41.096589 20972 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:41.097110 20972 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:41.092803 15526 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:41.105129 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.105232 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.107743 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:41.108323 16237 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:41.108894 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.109504 15520 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:20:41.110327 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.110999 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.111912 15528 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 17.
I20260709 13:20:41.112462 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:41.112821 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:41.113183 15524 raft_consensus.cc:2393] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 13: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:41.114055 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:41.114568 15524 raft_consensus.cc:3060] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:41.115011 15528 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:41.112041 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.117695 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.118831 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:41.119407 15525 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:41.092547 21073 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:41.083312 21080 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:41.122519 21080 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.122995 21076 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.125310 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.127838 15528 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18.
I20260709 13:20:41.127935 15526 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:41.133576 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:41.134250 16239 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:41.141067 16239 raft_consensus.cc:2468] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:20:41.142071 16237 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 6.
I20260709 13:20:41.061939 21084 consensus_queue.cc:1059] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 13:20:41.144522 20982 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:41.167395 20982 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:41.169628 20982 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:41.147468 15524 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:20:41.149652 21076 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:41.151211 20972 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:41.152752 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:41.157618 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.158623 15525 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18.
I20260709 13:20:41.162174 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:41.164742 15568 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:41.185649 15520 raft_consensus.cc:3055] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:41.186043 15520 raft_consensus.cc:740] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:41.187043 15568 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:41.186812 15520 consensus_queue.cc:260] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.187716 15520 raft_consensus.cc:3060] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:41.166672 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:41.145218 21077 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:41.189020 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:41.189392 21077 raft_consensus.cc:2804] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:41.189996 16233 raft_consensus.cc:2393] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 15: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:41.188989 15598 raft_consensus.cc:2393] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 16: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:41.161211 15521 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.191085 20982 raft_consensus.cc:2804] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:41.189895 21077 raft_consensus.cc:697] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:41.192078 15521 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:41.165748 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:41.192587 21077 consensus_queue.cc:237] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.166275 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.174966 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.193638 15526 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
W20260709 13:20:41.177417 15603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53324 (request call id 2237) took 2960 ms (client timeout 2424 ms). Trace:
W20260709 13:20:41.195188 15603 rpcz_store.cc:269] 0709 13:20:38.217236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.217342 (+   106us) service_pool.cc:224] Handling call
0709 13:20:41.177141 (+2959799us) spinlock_profiling.cc:243] Waited 2.96 s on lock 0x7b5400168d54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:41.177298 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6802377088}
I20260709 13:20:41.183395 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:41.181651 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.166326 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:41.196264 15594 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 15.
I20260709 13:20:41.196415 15524 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 15.
I20260709 13:20:41.170680 21080 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:41.193249 16239 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 12: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:41.174463 21084 raft_consensus.cc:3055] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:41.200462 21084 raft_consensus.cc:740] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:41.201365 21084 consensus_queue.cc:260] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.202623 21084 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:41.203007 20887 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:20:41.184430 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 123420us
I20260709 13:20:41.206650 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:41.196437 15600 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:41.197496 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:41.208231 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:41.208576 15598 raft_consensus.cc:2393] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 11: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:41.207350 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:41.209604 15594 raft_consensus.cc:2393] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 13: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:41.210299 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:41.212824 15493 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:41.218326 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:41.220055 15602 raft_consensus.cc:1240] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:41.220633 21082 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:41.221077 21082 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:41.219977 15600 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:41.221481 21082 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:41.221195 21038 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.224365 21038 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:41.225906 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:41.226670 15598 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:41.221315 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:41.225075 15527 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:41.234479 15493 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:41.236579 15493 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:41.237717 15493 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:41.235392 21038 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:41.238461 20972 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:41.242580 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:41.246951 21082 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.251693 21082 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:41.255064 15523 raft_consensus.cc:2368] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 7. Current term is 8.
W20260709 13:20:41.256512 15523 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 2311) took 2498 ms (client timeout 1938 ms). Trace:
W20260709 13:20:41.256887 15523 rpcz_store.cc:269] 0709 13:20:38.758099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.758979 (+   880us) service_pool.cc:224] Handling call
0709 13:20:41.256283 (+2497304us) spinlock_profiling.cc:243] Waited 2.46 s on lock 0x7b5400188654. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:41.256438 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5666951424}
W20260709 13:20:41.259140 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:41.263720 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.233600 20982 raft_consensus.cc:697] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:41.266649 15520 raft_consensus.cc:1275] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:20:41.267045 15520 pending_rounds.cc:85] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 10
I20260709 13:20:41.267333 15520 pending_rounds.cc:107] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:20:41.267558 15520 raft_consensus.cc:2889] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:20:41.268550 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 44019.8us
I20260709 13:20:41.269956 20955 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:41.279456 20955 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.275220 21086 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:20:41.273535 16203 connection.cc:256] RPC call timeout handler was delayed by -1.3630579020000368s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21930000000000002s for a response
I20260709 13:20:41.282914 20664 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:41.275458 21038 raft_consensus.cc:2804] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:41.283617 21038 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:41.283532 20664 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.284097 21038 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:41.285714 20664 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:41.288128 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:41.288558 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 426282us
I20260709 13:20:41.290066 20884 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:41.290609 20884 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.292124 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:41.292538 20884 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:41.293076 21038 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.293431 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:41.293785 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:41.294247 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:41.294795 15598 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 20.
I20260709 13:20:41.293354 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:41.296164 15595 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
I20260709 13:20:41.296978 16238 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:20:41.297649 16238 raft_consensus.cc:606] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:41.298027 16238 raft_consensus.cc:614] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:41.292752 15599 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:20:41.295617 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:41.299981 16205 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:41.303105 21038 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:41.266952 20982 consensus_queue.cc:237] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.315418 16204 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:41.316711 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 79063.7us
I20260709 13:20:41.317458 16205 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:41.318608 16204 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:41.326152 20664 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:41.326669 20664 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:41.326751 20884 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:41.327137 20664 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:41.327646 21072 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:41.329205 21075 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:41.329646 21075 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:41.330084 21075 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:41.327194 20884 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:41.347333 20664 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.347556 20884 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:41.323230 16204 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:41.418349 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 100983us
W20260709 13:20:41.419373 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 96220.9us
I20260709 13:20:41.422580 16239 raft_consensus.cc:1240] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 12. Current term is 13. Ops: []
W20260709 13:20:41.407363 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:41.417229 21075 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:41.429298 15566 connection.cc:256] RPC call timeout handler was delayed by -1.0197082740000951s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16870000000000002s for a response
I20260709 13:20:41.430012 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:41.424918 21077 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:41.431255 21077 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:41.432323 15594 raft_consensus.cc:2393] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 11: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:41.433377 21077 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:41.436921 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:41.437681 15595 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:20:41.440742 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:41.441231 15595 raft_consensus.cc:1240] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:41.442476 15600 raft_consensus.cc:1240] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:20:41.443961 15598 raft_consensus.cc:1240] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:41.444973 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:41.445749 15603 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:41.415561 21072 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:41.446830 21072 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:41.447389 21072 raft_consensus.cc:697] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:20:41.450995 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:41.363657 21054 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:41.448351 21072 consensus_queue.cc:237] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.458123 20981 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:41.463665 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:41.471114 16233 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:20:41.467945 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:41.428371 21087 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:41.479303 16232 raft_consensus.cc:2393] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 12: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:41.438278 20664 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:41.481982 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:41.482625 16238 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:41.489404 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:41.490170 16232 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 15.
I20260709 13:20:41.492637 20784 consensus_queue.cc:1059] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:41.485705 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:41.505789 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:41.508338 16232 raft_consensus.cc:3055] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:41.508642 16232 raft_consensus.cc:740] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:41.475292 21075 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:41.503660 16238 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:20:41.511075 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:41.514681 16234 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:41.515445 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:41.516124 16238 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:20:41.512176 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:41.509593 16232 consensus_queue.cc:260] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.519085 16232 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:41.520924 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:41.513062 15493 connection.cc:256] RPC call timeout handler was delayed by -1.2487315240002772s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2117s for a response
W20260709 13:20:41.526556 15493 connection.cc:256] RPC call timeout handler was delayed by -1.141631524000104s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22360000000000002s for a response
I20260709 13:20:41.527096 16237 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:20:41.529140 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:41.529794 15600 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:41.531072 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.532007 15493 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:41.532238 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:41.535620 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 106904us
I20260709 13:20:41.548035 16238 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 7.
I20260709 13:20:41.539372 16205 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:41.550945 15493 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:41.470333 21054 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.573488 15600 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 15.
I20260709 13:20:41.574570 21054 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:41.574910 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:41.575517 15600 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:41.558426 16234 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 16.
I20260709 13:20:41.555814 21095 raft_consensus.cc:3055] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:41.560344 16237 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:41.576855 21095 raft_consensus.cc:740] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:41.563412 16232 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:20:41.552533 21091 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:41.577525 16234 raft_consensus.cc:1240] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:41.577845 21091 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:41.558718 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 54681.4us
I20260709 13:20:41.578297 21091 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:41.577883 21095 consensus_queue.cc:260] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.575222 21082 consensus_queue.cc:1059] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:41.579301 21095 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:41.504482 21087 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:41.582674 21087 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:41.586225 21091 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:41.589681 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 50421.9us
I20260709 13:20:41.591282 16205 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:41.596429 16205 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:41.588619 21091 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:41.603737 21075 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:41.612895 21075 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.608134 21084 consensus_queue.cc:1059] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:20:41.617318 21073 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:41.617897 21073 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.619591 20884 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.620733 16239 raft_consensus.cc:2393] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 18: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:41.621565 21073 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:41.623476 20972 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:41.625547 20972 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:20:41.654326 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 57967us
I20260709 13:20:41.660307 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:41.661051 15674 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:20:41.663941 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:41.556984 21086 consensus_queue.cc:1059] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:20:41.670974 15600 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:20:41.671695 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:41.672767 16238 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:20:41.674149 15568 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:41.655894 16205 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:41.675130 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:41.676481 21086 raft_consensus.cc:3055] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:41.677047 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:20:41.682411 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 187917us
I20260709 13:20:41.686060 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:41.681533 21075 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:41.626673 21095 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:41.687669 21107 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:41.688146 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:41.710299 15600 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:41.657099 20972 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:41.689011 21103 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:41.731570 21103 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:41.731988 21103 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:41.711452 15670 raft_consensus.cc:2393] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 14: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:41.725478 21107 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:41.735211 21107 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:20:41.687266 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 261598us
I20260709 13:20:41.692999 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:41.750293 15596 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:41.750479 16204 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:41.752839 15600 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:41.702883 15673 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 11
W20260709 13:20:41.702999 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 130858us
I20260709 13:20:41.703786 15674 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:41.758713 15493 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:41.760095 15493 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:41.766909 15673 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
W20260709 13:20:41.768271 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 112445us
I20260709 13:20:41.769002 16204 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:41.769161 16205 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:41.761021 15493 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:41.771660 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.771937 16205 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:41.773464 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.774129 15520 raft_consensus.cc:2376] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13. Re-sending same reply.
I20260709 13:20:41.775457 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.776067 15520 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:20:41.777824 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:41.778517 15522 raft_consensus.cc:2368] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 15. Current term is 16.
I20260709 13:20:41.779332 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.780011 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.780282 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.781342 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:41.782196 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.779371 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:41.782570 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:41.783017 15525 raft_consensus.cc:2368] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 7. Current term is 8.
I20260709 13:20:41.784880 15493 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:41.786245 15674 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 17.
I20260709 13:20:41.688321 21072 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:41.788385 21072 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:41.788791 21072 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:41.789137 21072 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:41.777092 21105 raft_consensus.cc:2804] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:41.790853 21084 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:20:41.788535 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 75960.6us
I20260709 13:20:41.790771 15520 raft_consensus.cc:2393] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 11: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:41.791811 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:41.792241 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.793013 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:41.793242 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.793567 15526 raft_consensus.cc:2376] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13. Re-sending same reply.
I20260709 13:20:41.788399 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:41.794349 15599 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:20:41.788476 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:41.794807 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.793977 15519 raft_consensus.cc:2368] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 15. Current term is 16.
I20260709 13:20:41.766027 20884 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:41.703395 21086 raft_consensus.cc:740] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:41.915997 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:41.916502 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:41.917245 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:41.917589 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:41.918182 15669 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:41.918363 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:41.919059 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:41.919535 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:41.920135 15677 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:41.920554 15669 raft_consensus.cc:1240] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:41.920939 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:41.918921 15668 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:41.795382 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:41.794602 15567 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:41.870016 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.922837 15528 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 12: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:41.793517 15596 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 15.
I20260709 13:20:41.923643 21109 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:20:41.891073 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 106280us
I20260709 13:20:41.924470 21109 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:41.918228 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:41.844846 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:41.924949 21109 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:41.926082 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:41.926115 15676 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:20:41.925140 15493 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:41.930346 21108 log.cc:927] Time spent T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:20:41.870407 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.932879 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:41.924016 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:41.933611 15672 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:41.770839 21110 raft_consensus.cc:2804] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:41.934302 21110 raft_consensus.cc:697] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:41.935488 21110 consensus_queue.cc:237] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.936401 20791 raft_consensus.cc:993] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9: : Instructing follower d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:20:41.771543 21082 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:41.938215 15671 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:41.938488 21082 raft_consensus.cc:697] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:41.939406 21104 raft_consensus.cc:2804] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:41.919133 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:41.939553 21082 consensus_queue.cc:237] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:41.940361 21022 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.084s	user 0.021s	sys 0.049s
I20260709 13:20:41.940488 15670 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
W20260709 13:20:41.940799 21022 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.084s	user 0.022s	sys 0.050s
I20260709 13:20:41.941301 21104 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:41.941725 21104 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:41.942893 15698 debug-util.cc:398] Leaking SignalData structure 0x7b0800350a40 after lost signal to thread 14429
I20260709 13:20:41.916198 15525 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
W20260709 13:20:41.944156 15698 debug-util.cc:398] Leaking SignalData structure 0x7b08003845c0 after lost signal to thread 14432
W20260709 13:20:41.932446 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 137912us
I20260709 13:20:41.897274 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:41.947106 15600 raft_consensus.cc:2393] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 11: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:41.877365 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:41.948108 15521 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 15: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:41.960086 15567 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:41.960438 15672 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:41.878055 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:41.968648 15673 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:20:41.902633 15599 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
W20260709 13:20:41.994309 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 69255.7us
W20260709 13:20:41.997684 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 225804us
I20260709 13:20:41.999090 16205 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:41.795578 16204 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:42.007439 16234 raft_consensus.cc:1275] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:20:42.009397 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:41.995102 15493 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:42.058764 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 59733.2us
I20260709 13:20:42.060261 16234 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:20:42.061646 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:42.062031 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:42.062314 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:41.944844 20791 raft_consensus.cc:1081] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Signalling peer d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:20:41.879561 21075 raft_consensus.cc:2764] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 13:20:42.063881 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 103864us
I20260709 13:20:41.791415 21084 raft_consensus.cc:697] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
W20260709 13:20:42.067360 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 72347us
I20260709 13:20:42.072696 15493 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:41.770089 21095 raft_consensus.cc:697] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:42.068794 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:41.877036 21115 raft_consensus.cc:2764] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:20:42.038429 20884 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:42.067657 21087 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:41.939718 21072 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:41.915069 20972 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:41.924714 21103 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.125384 21087 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:42.130695 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 335172us
I20260709 13:20:41.943972 21107 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.091648 20884 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:42.140486 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.145534 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:42.074748 20981 consensus_queue.cc:1048] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:20:42.121392 21116 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:20:42.191747 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.152051 15599 raft_consensus.cc:2368] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 15. Current term is 16.
I20260709 13:20:42.128207 21105 raft_consensus.cc:697] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:42.134060 21087 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:42.133759 21115 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:42.156924 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:42.159237 20984 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.279s	user 0.006s	sys 0.010s
I20260709 13:20:42.133416 21119 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:42.149448 20884 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:42.180663 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 114391us
I20260709 13:20:42.134397 16204 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:42.154937 21118 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:20:42.194458 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 121864us
I20260709 13:20:42.198086 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:42.201004 15672 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 8.
I20260709 13:20:42.201016 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:42.120935 21086 consensus_queue.cc:260] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:42.251071 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 116752us
W20260709 13:20:42.259349 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 54199.4us
W20260709 13:20:42.250886 20984 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.371s	user 0.006s	sys 0.010s
I20260709 13:20:42.205211 21119 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:42.256472 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:42.172405 21084 consensus_queue.cc:237] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:42.263104 15672 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:42.263932 21107 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:42.267808 21072 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:42.269341 21120 log.cc:927] Time spent T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:20:42.270766 15671 raft_consensus.cc:3055] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:42.270860 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:42.278204 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:42.279021 15671 raft_consensus.cc:740] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:42.279341 15670 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:20:42.186216 21095 consensus_queue.cc:237] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:42.289830 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.279906 15671 consensus_queue.cc:260] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.290968 15671 raft_consensus.cc:3060] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:42.257820 21086 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:42.258584 21110 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:42.292164 15566 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:42.258141 21115 raft_consensus.cc:697] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:42.275135 21119 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:42.255400 21114 consensus_queue.cc:1048] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:20:42.294870 21117 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:42.294716 21110 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:42.304319 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:42.304792 15671 raft_consensus.cc:1275] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:20:42.295059 21125 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:42.308269 16233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:20:42.308873 16233 raft_consensus.cc:606] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:42.309156 16233 raft_consensus.cc:614] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:42.314779 21115 consensus_queue.cc:237] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.307289 21100 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:42.306969 21110 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:42.312151 21117 raft_consensus.cc:697] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:42.312924 21118 raft_consensus.cc:697] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:42.347211 21118 consensus_queue.cc:237] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.335271 21087 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:42.347934 21109 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.328027 21100 raft_consensus.cc:697] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:42.350051 21087 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:42.323186 21125 raft_consensus.cc:697] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:42.350064 21100 consensus_queue.cc:237] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:42.351406 21125 consensus_queue.cc:237] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.360417 21119 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.339535 21104 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.340703 21117 consensus_queue.cc:237] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.364096 21119 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:42.308244 21121 consensus_queue.cc:1059] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 13:20:42.294093 21105 consensus_queue.cc:237] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.365829 21119 raft_consensus.cc:3055] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:42.366374 21119 raft_consensus.cc:740] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:42.367259 21119 consensus_queue.cc:260] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.324841 21103 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:42.336721 20884 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.370347 20884 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:42.370530 21104 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:42.360921 21110 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.362121 21109 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:42.410624 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 150017us
W20260709 13:20:42.413275 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 121037us
I20260709 13:20:42.414738 15566 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:42.419952 15566 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:42.423516 15493 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:42.424661 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 355004us
I20260709 13:20:42.425670 15594 raft_consensus.cc:1240] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:42.428597 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:42.429973 15596 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:42.438138 15567 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:42.444397 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:20:42.451186 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.453166 16233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:20:42.455201 20981 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:42.458784 21084 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:42.459764 21084 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.461591 21110 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:42.463974 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:42.464927 16231 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:42.467555 16233 raft_consensus.cc:606] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:42.468164 16233 raft_consensus.cc:614] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:42.469084 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:42.476099 21109 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:42.467692 21118 consensus_queue.cc:1059] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:20:42.478418 20981 raft_consensus.cc:697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:42.479724 21118 raft_consensus.cc:3055] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:42.480437 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:20:42.480147 20981 consensus_queue.cc:237] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.482540 15598 raft_consensus.cc:1240] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:42.488102 21104 raft_consensus.cc:2804] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:42.480209 21118 raft_consensus.cc:740] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
W20260709 13:20:42.473644 15601 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34102 (request call id 1538) took 1265 ms (client timeout 1519 ms). Trace:
W20260709 13:20:42.517069 15601 rpcz_store.cc:269] 0709 13:20:41.207778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:41.207966 (+   188us) service_pool.cc:224] Handling call
0709 13:20:42.473174 (+1265208us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b540043c954. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:42.473440 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":766764288}
I20260709 13:20:42.527001 15601 raft_consensus.cc:1240] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:42.527891 16232 raft_consensus.cc:1240] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:42.530108 21104 raft_consensus.cc:697] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:42.531925 21104 consensus_queue.cc:237] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.533105 21118 consensus_queue.cc:260] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.535172 21118 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:42.535807 21084 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:42.537398 15601 raft_consensus.cc:1240] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 13:20:42.543478 15698 debug-util.cc:398] Leaking SignalData structure 0x7b08001b6e20 after lost signal to thread 15460
I20260709 13:20:42.549175 15600 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:20:42.530108 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.554087 15596 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 21.
I20260709 13:20:42.567920 20955 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:42.543833 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56840 (request call id 1815) took 1436 ms. Trace:
I20260709 13:20:42.601207 15527 rpcz_store.cc:276] 0709 13:20:41.107642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:41.107772 (+   130us) service_pool.cc:224] Handling call
0709 13:20:42.543358 (+1435586us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b540029c2d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:42.543566 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":247372032}
I20260709 13:20:42.604508 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:42.473596 21022 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.103s
W20260709 13:20:42.578421 15698 debug-util.cc:398] Leaking SignalData structure 0x7b0800345ba0 after lost signal to thread 15552
I20260709 13:20:42.579833 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:42.580542 15493 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:42.582492 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:42.583312 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:42.587816 15566 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:42.590365 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:42.590894 15599 raft_consensus.cc:1240] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:42.591378 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:42.607308 20722 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:42.594933 15567 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:42.578042 21087 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:42.598346 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:42.612110 20955 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:42.612425 15567 leader_election.cc:400] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 15. Current term is 16.
I20260709 13:20:42.612852 21118 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:20:42.614634 15698 debug-util.cc:398] Leaking SignalData structure 0x7b0800338100 after lost signal to thread 15627
W20260709 13:20:42.618266 15698 debug-util.cc:398] Leaking SignalData structure 0x7b08002b95c0 after lost signal to thread 15628
I20260709 13:20:42.621639 15601 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:42.605443 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:42.621464 20955 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:42.623167 20955 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:42.613523 21118 raft_consensus.cc:697] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
W20260709 13:20:42.624087 16235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57244 (request call id 1595) took 1521 ms (client timeout 1581 ms). Trace:
W20260709 13:20:42.624577 16235 rpcz_store.cc:269] 0709 13:20:41.102657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:42.582022 (+1479365us) service_pool.cc:224] Handling call
0709 13:20:42.624016 (+ 41994us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:42.628173 15566 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:42.629455 15566 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:42.642123 20955 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:42.608207 20722 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.644119 15567 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:42.630142 16236 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:20:42.630848 16232 raft_consensus.cc:2393] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 16: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:42.624733 21118 consensus_queue.cc:237] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:42.633201 16234 raft_consensus.cc:2393] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 21: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:42.638108 15520 raft_consensus.cc:1240] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:42.638476 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:42.639343 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:42.642681 21074 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:42.630185 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:42.657943 21074 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:42.658208 15523 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:20:42.658504 21074 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:42.658900 15524 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:42.616108 21119 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:42.646384 20722 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:42.654238 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.661931 15566 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:42.662127 16236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57244 (request call id 1594) took 1559 ms (client timeout 1677 ms). Trace:
I20260709 13:20:42.662257 21103 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:42.656948 16231 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:42.645813 21059 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:42.630405 21038 raft_consensus.cc:3055] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:42.663626 15567 leader_election.cc:400] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 7. Current term is 8.
I20260709 13:20:42.673977 15567 leader_election.cc:403] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:42.657011 15525 raft_consensus.cc:1275] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:20:42.613384 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 4.26491e+06us
W20260709 13:20:42.665470 16236 rpcz_store.cc:269] 0709 13:20:41.102361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:42.579276 (+1476915us) service_pool.cc:224] Handling call
0709 13:20:42.662048 (+ 82772us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:42.697227 16231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57244 (request call id 1603) took 1522 ms. Trace:
I20260709 13:20:42.697728 16231 rpcz_store.cc:276] 0709 13:20:41.152616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:42.463285 (+1310669us) service_pool.cc:224] Handling call
0709 13:20:42.674953 (+211668us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:42.661821 20955 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:20:42.673527 21059 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:42.705430 16237 raft_consensus.cc:1240] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:42.657433 21087 raft_consensus.cc:2764] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:20:42.673707 21038 raft_consensus.cc:740] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:42.710358 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:42.712136 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 50414.2us
I20260709 13:20:42.712774 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:42.713065 15566 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:42.713583 16233 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:20:42.705806 21059 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:20:42.714598 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 51145.5us
I20260709 13:20:42.717216 15567 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:42.720414 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:42.722124 20975 consensus_queue.cc:1059] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:42.723706 21091 raft_consensus.cc:3055] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:42.724139 21091 raft_consensus.cc:740] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:42.725340 21091 consensus_queue.cc:260] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.726416 21091 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:42.731487 15641 connection.cc:256] RPC call timeout handler was delayed by -4.10898791600016s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2378s for a response
W20260709 13:20:42.732162 15641 connection.cc:256] RPC call timeout handler was delayed by -3.1606409840001106s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1567s for a response
W20260709 13:20:42.732499 15641 connection.cc:256] RPC call timeout handler was delayed by -2.9275409840001885s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1826s for a response
W20260709 13:20:42.733405 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:42.732779 15641 connection.cc:256] RPC call timeout handler was delayed by -2.8987409840001419s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18580000000000002s for a response
W20260709 13:20:42.734084 15641 connection.cc:256] RPC call timeout handler was delayed by -2.8411409840000488s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1922s for a response
W20260709 13:20:42.734386 15641 connection.cc:256] RPC call timeout handler was delayed by -2.7297696190000806s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16870000000000002s for a response
I20260709 13:20:42.735680 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:42.735968 16232 raft_consensus.cc:1240] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 12. Current term is 13. Ops: []
W20260709 13:20:42.734661 15641 connection.cc:256] RPC call timeout handler was delayed by -2.6105531660000452s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1771s for a response
W20260709 13:20:42.737471 15641 connection.cc:256] RPC call timeout handler was delayed by -2.2318409840002005s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2599s for a response
W20260709 13:20:42.704268 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 107296us
I20260709 13:20:42.742529 15600 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 16.
I20260709 13:20:42.744196 20972 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:42.746851 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:42.747700 15526 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:20:42.752244 21074 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.754029 15566 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:42.761962 15523 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:42.770382 15524 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 17.
I20260709 13:20:42.770768 15601 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:42.771716 20887 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:42.772058 21103 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:42.772495 21074 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:42.772696 21077 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:42.773116 21077 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:42.773277 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.773517 21077 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:42.769857 20955 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 7. Current term is 8.
I20260709 13:20:40.857760 15412 catalog_manager.cc:5028] TS 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): tablet b99eaeae24b44aa78cfa8c90caca288e (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
W20260709 13:20:42.800215 15412 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 1.94226e+06us
I20260709 13:20:42.801450 20972 consensus_queue.cc:1059] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:42.803078 20972 raft_consensus.cc:3055] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:42.803583 20972 raft_consensus.cc:740] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:42.804803 20972 consensus_queue.cc:260] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.806720 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:42.808534 15566 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:42.808446 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 91382.6us
I20260709 13:20:42.761022 21059 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:42.814903 15493 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.810230 21022 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:42.817982 15568 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:42.819478 21059 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:42.826604 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.821646 21059 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:42.807591 20887 raft_consensus.cc:697] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:42.835675 21109 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:42.834643 21103 raft_consensus.cc:697] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:42.837827 21109 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:42.837935 16205 leader_election.cc:400] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 15. Current term is 16.
I20260709 13:20:42.838331 21109 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:42.834221 15567 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:42.840099 16205 leader_election.cc:403] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:42.838662 21103 consensus_queue.cc:237] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.846104 15566 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:42.848073 15602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 2248) took 1344 ms. Trace:
I20260709 13:20:42.834558 21077 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.848845 15602 rpcz_store.cc:276] 0709 13:20:41.462096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:42.474904 (+1012808us) service_pool.cc:224] Handling call
0709 13:20:42.475513 (+   609us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:42.475993 (+   480us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:20:42.571022 (+ 95029us) log.cc:844] Serialized 14 byte log entry
0709 13:20:42.806329 (+235307us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:42.806378 (+    49us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:42.806424 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:42.806489 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:42.806625 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31742,"prepare.queue_time_us":99906,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":66,"raft.queue_time_us":36224,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":655,"thread_start_us":271280,"threads_started":2,"wal-append.queue_time_us":185397}
I20260709 13:20:42.840567 20972 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:42.834779 21119 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:42.847366 21059 raft_consensus.cc:697] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:42.868059 21077 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:42.890092 21023 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:20:42.894053 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:42.894416 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 48426.2us
I20260709 13:20:42.888273 21074 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:42.895246 21074 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:42.895686 21074 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:42.897145 21022 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:42.896893 20887 consensus_queue.cc:237] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:42.897646 21022 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:42.898288 20955 consensus_queue.cc:1048] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:42.907683 21022 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.910558 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:42.911614 21022 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:42.923168 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:42.923957 16238 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
W20260709 13:20:42.925061 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 94765.2us
I20260709 13:20:42.923231 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:20:42.927120 15698 debug-util.cc:398] Leaking SignalData structure 0x7b08001e5680 after lost signal to thread 15701
W20260709 13:20:42.927443 15698 debug-util.cc:398] Leaking SignalData structure 0x7b080039f700 after lost signal to thread 15702
W20260709 13:20:42.928748 15698 debug-util.cc:398] Leaking SignalData structure 0x7b08002c66e0 after lost signal to thread 16266
W20260709 13:20:42.929019 15698 debug-util.cc:398] Leaking SignalData structure 0x7b08003d9820 after lost signal to thread 16267
W20260709 13:20:42.929419 15698 debug-util.cc:398] Leaking SignalData structure 0x7b08002bcba0 after lost signal to thread 20873
W20260709 13:20:42.929644 15698 debug-util.cc:398] Leaking SignalData structure 0x7b08003391a0 after lost signal to thread 20874
I20260709 13:20:42.936352 15567 leader_election.cc:400] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 15. Current term is 16.
I20260709 13:20:42.936805 15567 leader_election.cc:403] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:20:42.938153 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 51137us
W20260709 13:20:42.944782 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.942669 16205 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:42.945469 16233 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:20:42.946003 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:42.955029 16240 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:20:42.951202 15698 debug-util.cc:398] Leaking SignalData structure 0x7b08003f1fa0 after lost signal to thread 20997
I20260709 13:20:42.953168 15493 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:42.957957 15698 debug-util.cc:398] Leaking SignalData structure 0x7b0800367420 after lost signal to thread 20998
W20260709 13:20:42.962646 15698 debug-util.cc:398] Leaking SignalData structure 0x7b0800355460 after lost signal to thread 21010
W20260709 13:20:42.962965 15698 debug-util.cc:398] Leaking SignalData structure 0x7b08002f4ee0 after lost signal to thread 21014
W20260709 13:20:42.963147 15698 debug-util.cc:398] Leaking SignalData structure 0x7b0800318ce0 after lost signal to thread 21015
I20260709 13:20:42.963662 21079 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:42.964287 21079 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.967525 21014 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:42.968226 21014 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.970320 21079 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:42.944489 21119 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:42.970088 21116 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 15. Current term is 16.
I20260709 13:20:42.972221 21119 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:42.960984 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:42.964699 21100 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 15. Current term is 16.
W20260709 13:20:42.983232 15698 debug-util.cc:398] Leaking SignalData structure 0x7b08003cc260 after lost signal to thread 21026
I20260709 13:20:42.985291 16240 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 17.
W20260709 13:20:42.992278 15698 debug-util.cc:398] Leaking SignalData structure 0x7b0800386400 after lost signal to thread 21030
I20260709 13:20:42.992890 15568 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:42.981961 21014 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:43.018370 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:42.968717 21109 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:42.996492 21059 consensus_queue.cc:237] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.022548 21109 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:43.016624 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:43.024730 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:20:43.015450 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 67566.1us
I20260709 13:20:43.019322 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:20:43.001420 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 75604.2us
W20260709 13:20:43.006335 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 120963us
I20260709 13:20:43.018947 15523 raft_consensus.cc:1240] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:43.019722 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:43.020076 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:43.018278 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:43.032868 15526 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:42.996766 21118 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:43.036948 15493 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:20:43.003422 21147 log.cc:927] Time spent T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:20:43.033651 20955 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:43.050531 15527 raft_consensus.cc:2393] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 12: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:43.052736 15675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2790) took 1385 ms. Trace:
I20260709 13:20:43.053329 21098 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:43.053707 15675 rpcz_store.cc:276] 0709 13:20:41.666926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:41.667077 (+   151us) service_pool.cc:224] Handling call
0709 13:20:41.667418 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:43.052083 (+1384665us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:43.052140 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:43.052205 (+    65us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:43.052243 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:43.052350 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:43.052545 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1299003,"thread_start_us":63520,"threads_started":1}
I20260709 13:20:43.053802 20997 consensus_queue.cc:1048] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:20:43.056770 20350 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:43.058100 21144 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:43.058874 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 304388us
W20260709 13:20:43.058347 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:43.060318 15641 leader_election.cc:400] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 6 pre-election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 6. Current term is 7.
I20260709 13:20:43.061160 15493 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:20:43.058759 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 380191us
I20260709 13:20:43.063118 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:43.063946 15674 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:43.064109 21030 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:43.065153 21121 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:43.065734 21121 raft_consensus.cc:697] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:43.066128 15601 raft_consensus.cc:3055] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:43.066305 21030 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:43.066877 21121 consensus_queue.cc:237] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:43.069478 21125 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:43.070137 21125 raft_consensus.cc:697] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:43.071107 21125 consensus_queue.cc:237] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:43.071833 15641 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:43.072686 21014 consensus_queue.cc:1048] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:43.054157 15520 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:43.074332 21146 raft_consensus.cc:2804] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:43.074805 21146 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:43.075169 21146 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:43.075884 16232 raft_consensus.cc:1240] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:43.076912 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:43.077587 15523 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:43.077080 16235 raft_consensus.cc:1275] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:20:43.064114 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:43.058936 21144 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.080614 21014 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:43.052723 20873 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:43.081887 21010 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:43.001677 21038 consensus_queue.cc:260] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.083974 21038 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:43.084635 15601 raft_consensus.cc:740] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:43.082453 15524 raft_consensus.cc:2393] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 9: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:20:43.089892 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:43.076045 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:43.096186 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:43.097289 21098 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.097720 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:43.099908 21098 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:43.100653 16205 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:43.101581 15567 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:43.085531 15601 consensus_queue.cc:260] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:43.102777 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 60708.4us
I20260709 13:20:43.103387 15601 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:43.104537 15567 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:43.096271 21161 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
W20260709 13:20:43.106190 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 68341.2us
I20260709 13:20:43.106714 20350 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:43.107903 21100 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:43.095630 15642 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:43.111380 21030 consensus_queue.cc:1059] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:43.112226 15641 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:43.112803 21098 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:43.113672 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:43.109346 21010 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:43.082923 20873 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.116734 21146 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.118736 16233 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:20:43.119999 21091 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:43.085492 21104 consensus_queue.cc:1175] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" responder_term: 11 status { last_received { term: 10 index: 9 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:20:43.082612 21144 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:43.122592 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.122991 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:43.124468 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:43.124846 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:43.127203 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:20:43.130172 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:43.135555 15524 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:56840
I20260709 13:20:43.199877 15524 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:43.200313 15524 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:43.138288 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:20:43.146386 21151 log.cc:927] Time spent T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.309s	user 0.001s	sys 0.000s
I20260709 13:20:43.204659 15526 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 15.
W20260709 13:20:43.095520 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 111093us
I20260709 13:20:43.207033 21046 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:43.208089 21030 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:43.208559 21030 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:43.208663 20873 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:43.209005 21030 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:43.209259 21091 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.211577 15671 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 20.
I20260709 13:20:43.212004 21091 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:43.162703 15674 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:43.191068 15675 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:43.213687 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:43.214358 15527 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:43.214522 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:20:43.214793 16203 leader_election.cc:336] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.646s (SENT)
I20260709 13:20:43.215111 15674 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:43.218547 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 114115us
I20260709 13:20:43.219426 15567 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:43.220458 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 124936us
I20260709 13:20:43.221397 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:43.222625 15674 raft_consensus.cc:2468] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:20:43.222997 15642 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:43.222424 21164 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:43.224038 21164 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.224426 15642 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:43.226449 21164 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:43.226789 21074 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:43.227424 20998 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:43.227944 21098 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:43.228052 20998 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.228727 21098 raft_consensus.cc:2764] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:20:43.227017 21164 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:43.229161 21074 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:43.229548 21164 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.213161 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:43.238376 15670 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:43.235467 20998 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:43.221705 15675 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:20:43.238313 15668 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:43.238462 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:20:43.198643 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 86476.7us
I20260709 13:20:43.239336 15598 raft_consensus.cc:1240] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:43.239895 16203 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:43.209012 15676 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:43.241426 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:43.184233 15525 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
W20260709 13:20:43.249121 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 43642.6us
I20260709 13:20:43.244069 15520 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:20:43.250756 15601 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:20:43.251873 21015 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:43.235899 15566 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:43.195300 20350 raft_consensus.cc:697] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:43.258860 15521 raft_consensus.cc:1275] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:20:43.259259 15519 raft_consensus.cc:1240] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:43.260367 15528 raft_consensus.cc:2393] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 16: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
W20260709 13:20:43.263644 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 44313.6us
I20260709 13:20:43.268823 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:20:43.270709 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 63912.9us
I20260709 13:20:43.302479 15568 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:43.303020 15597 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:20:43.272801 15641 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:43.213246 21145 raft_consensus.cc:3055] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:43.273303 21164 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:43.255878 15672 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:43.256582 21116 consensus_queue.cc:1059] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 13:20:43.266322 20955 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:43.277874 15670 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:43.279933 15523 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18.
I20260709 13:20:43.280555 21022 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:43.281384 15668 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:20:43.281656 15676 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:20:43.310702 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:43.314455 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 78663us
I20260709 13:20:43.315229 15566 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:43.316790 15568 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:43.318440 15675 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:43.319005 15520 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:20:43.319439 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:43.282603 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:43.208810 21146 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:43.279549 15524 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.289099 15493 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:43.292682 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 66997.7us
I20260709 13:20:43.237706 21030 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.293426 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:43.296007 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:43.296850 15601 pending_rounds.cc:85] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting all ops after (but not including) 11
I20260709 13:20:43.320703 15521 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:20:43.321030 15566 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:43.322199 15567 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:43.293133 21038 consensus_queue.cc:1048] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:20:43.304709 21145 raft_consensus.cc:740] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
W20260709 13:20:43.384730 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 112003us
I20260709 13:20:43.382187 15670 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:43.327867 15668 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:43.327327 15676 raft_consensus.cc:1240] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:43.306596 21059 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:20:43.325091 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 110358us
I20260709 13:20:43.388482 15528 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:20:43.309089 20955 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:43.386583 15594 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:43.399816 15524 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:43.402369 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 98528.8us
I20260709 13:20:43.386195 15601 pending_rounds.cc:107] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:20:43.404330 15601 raft_consensus.cc:2889] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:43.321559 21164 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:43.325390 21103 raft_consensus.cc:2804] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:43.324754 21109 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:20:43.414515 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 93551.9us
I20260709 13:20:43.327435 21119 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:43.417217 15566 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:43.418162 20887 raft_consensus.cc:2764] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:20:43.422562 15673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2810) took 1355 ms. Trace:
I20260709 13:20:43.423017 15673 rpcz_store.cc:276] 0709 13:20:42.068447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:42.068592 (+   145us) service_pool.cc:224] Handling call
0709 13:20:42.074608 (+  6016us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:43.407912 (+1333304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:43.414373 (+  6461us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:43.418173 (+  3800us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:43.418215 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:43.422200 (+  3985us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:43.422398 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":868700,"raft.queue_time_us":979969,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":130,"thread_start_us":399604,"threads_started":1}
I20260709 13:20:43.424045 21100 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:43.424494 20980 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.424957 21059 raft_consensus.cc:697] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:43.425465 21023 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:43.426054 21059 consensus_queue.cc:237] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.427640 15675 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:43.429745 15566 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:43.431150 21077 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:43.431665 21077 raft_consensus.cc:697] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:43.432550 21077 consensus_queue.cc:237] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:43.434223 21076 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:43.434734 21076 raft_consensus.cc:697] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:43.435590 21076 consensus_queue.cc:237] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.406155 20955 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:43.436540 20884 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:43.417092 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:43.448549 15642 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:43.449682 15676 raft_consensus.cc:1240] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:43.452126 20664 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:43.442116 16204 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:43.454433 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 47790.2us
I20260709 13:20:43.425102 20980 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.426065 21023 raft_consensus.cc:697] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:43.457999 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:43.480808 15674 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:43.487215 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:43.490963 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:43.493978 16204 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:43.495081 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:43.476285 15527 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
W20260709 13:20:43.496712 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 176124us
I20260709 13:20:43.499450 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.505895 15519 raft_consensus.cc:1240] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:20:43.506951 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:43.508451 15672 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 17.
I20260709 13:20:43.512650 15525 raft_consensus.cc:1240] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:20:43.513216 15521 raft_consensus.cc:1240] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:20:43.476475 20884 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:43.510211 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:43.517467 15526 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:43.510972 15528 raft_consensus.cc:1240] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:20:43.494369 21164 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:43.477098 21109 raft_consensus.cc:697] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:20:43.530241 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 108910us
I20260709 13:20:43.534904 15523 raft_consensus.cc:1240] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13. Ops: []
W20260709 13:20:43.534065 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 78196us
I20260709 13:20:43.437139 21125 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 13:20:43.507332 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:43.552060 15602 raft_consensus.cc:1240] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:43.556564 15676 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:20:43.553642 15670 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:43.547370 21146 consensus_queue.cc:1059] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 13:20:43.564908 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.566594 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.567299 15672 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:43.570948 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:43.571044 21146 raft_consensus.cc:3055] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:43.574280 16204 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:43.575954 21146 raft_consensus.cc:740] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:43.577952 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:43.578568 15672 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:43.506719 20350 consensus_queue.cc:237] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.592433 15672 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18.
I20260709 13:20:43.590559 21164 raft_consensus.cc:2764] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:20:43.594298 21174 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:43.598465 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.598963 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:43.599963 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.601227 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:43.602718 21167 log.cc:927] Time spent T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:20:43.603353 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.603998 21030 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:43.604537 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.604717 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.605300 15675 raft_consensus.cc:2368] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 6. Current term is 7.
I20260709 13:20:43.605533 15669 raft_consensus.cc:2368] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 15. Current term is 16.
I20260709 13:20:43.606314 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:43.515259 21145 consensus_queue.cc:260] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:43.607940 21174 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:43.607969 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.609234 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.598364 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:43.611840 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.611452 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.615806 15672 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:43.616551 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.616784 15675 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 20.
I20260709 13:20:43.592527 21116 raft_consensus.cc:2804] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:43.618501 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:43.610186 15674 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:43.620347 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:43.621088 15602 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
W20260709 13:20:43.620325 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 230550us
W20260709 13:20:43.622782 15491 leader_election.cc:336] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.512s (SENT)
W20260709 13:20:43.623859 15491 leader_election.cc:336] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.527s (SENT)
W20260709 13:20:43.625088 15491 leader_election.cc:336] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.554s (SENT)
W20260709 13:20:43.626354 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 187932us
I20260709 13:20:43.627173 15520 raft_consensus.cc:2368] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 17. Current term is 18.
W20260709 13:20:43.627028 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 197357us
I20260709 13:20:43.626713 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:20:43.629684 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 178839us
I20260709 13:20:43.630683 15676 raft_consensus.cc:1240] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:43.615119 20980 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:43.572957 21023 consensus_queue.cc:237] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.619288 21116 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:43.630092 15566 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:43.646624 15520 raft_consensus.cc:1240] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13. Ops: []
W20260709 13:20:43.649092 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 80928.3us
I20260709 13:20:43.651665 15520 raft_consensus.cc:1240] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:43.654371 15641 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:20:43.655936 16203 leader_election.cc:336] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.662s (SENT)
I20260709 13:20:43.657080 21076 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:43.657682 21076 raft_consensus.cc:697] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:43.658718 21076 consensus_queue.cc:237] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:43.666596 16203 leader_election.cc:336] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.685s (SENT)
I20260709 13:20:43.667109 16203 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:20:43.668437 16203 leader_election.cc:336] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.692s (SENT)
I20260709 13:20:43.668807 16203 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:43.671068 20980 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:43.671556 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:43.672343 15675 raft_consensus.cc:2368] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 16. Current term is 17.
I20260709 13:20:43.671716 20980 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:20:43.674036 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 44030.5us
I20260709 13:20:43.678218 15642 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:43.679210 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.677772 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.650739 21116 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:43.684911 15491 leader_election.cc:336] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.562s (SENT)
I20260709 13:20:43.686259 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:20:43.687853 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 365717us
I20260709 13:20:43.688867 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.691227 15491 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:43.691304 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.692338 15672 raft_consensus.cc:2368] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 6. Current term is 7.
I20260709 13:20:43.686903 15675 raft_consensus.cc:2376] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12. Re-sending same reply.
I20260709 13:20:43.690827 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:43.701247 21163 raft_consensus.cc:2764] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:20:43.701898 16235 raft_consensus.cc:1275] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:20:43.702461 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.701252 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:43.703873 15566 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:43.704082 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:43.704406 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:43.704844 15676 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:20:43.706103 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:43.706229 16205 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:43.703593 15673 raft_consensus.cc:2376] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12. Re-sending same reply.
I20260709 13:20:43.685031 21164 raft_consensus.cc:2804] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:43.713227 15641 leader_election.cc:400] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 17. Current term is 18.
I20260709 13:20:43.716739 15641 leader_election.cc:403] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:20:43.712548 15598 raft_consensus.cc:1275] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:20:43.714154 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.639604 21146 consensus_queue.cc:260] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.674428 21030 consensus_queue.cc:1059] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:20:43.718375 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:43.714833 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:43.618700 21109 consensus_queue.cc:237] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:43.721524 15673 raft_consensus.cc:1275] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:20:43.725570 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:43.726616 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 53123.9us
I20260709 13:20:43.720016 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.720705 21125 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:20:43.710136 21163 consensus_queue.cc:1059] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:20:43.712858 20873 consensus_queue.cc:1059] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 13:20:43.716003 21022 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:43.718055 21119 consensus_queue.cc:1059] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:20:43.727015 15676 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:43.687549 21161 consensus_queue.cc:1059] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:20:43.716684 21164 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:43.722198 21079 raft_consensus.cc:3055] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:43.749022 21079 raft_consensus.cc:740] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:43.749244 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.749460 21164 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:43.749900 15671 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 17.
I20260709 13:20:43.736377 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:43.742632 16232 raft_consensus.cc:3055] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:43.744378 16237 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:20:43.743374 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.729190 21104 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:43.747493 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:43.722057 21175 consensus_queue.cc:1059] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:20:43.732290 20972 consensus_queue.cc:1048] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:20:43.729079 21046 consensus_queue.cc:1059] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:20:43.760442 21079 consensus_queue.cc:260] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.761605 21079 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:20:43.762169 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 88936.6us
W20260709 13:20:43.762899 15491 leader_election.cc:336] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.606s (SENT)
I20260709 13:20:43.763247 15491 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:43.764487 15491 leader_election.cc:336] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.662s (SENT)
I20260709 13:20:43.765878 21022 raft_consensus.cc:697] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:43.767015 21022 consensus_queue.cc:237] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:43.769737 21179 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:43.770471 21179 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.772691 21179 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:43.772882 21091 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:43.773329 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:43.748636 15675 raft_consensus.cc:1275] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:43.751366 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.775825 15673 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 20.
I20260709 13:20:43.727486 21144 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 17. Current term is 18.
I20260709 13:20:43.746946 21146 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:43.746308 21172 log.cc:927] Time spent T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:20:43.751461 16232 raft_consensus.cc:740] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:43.752916 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:43.782637 15669 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:43.755187 15676 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:20:43.776993 16238 raft_consensus.cc:1240] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:43.736635 21030 consensus_queue.cc:1243] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer d3c4f5bd76e0445795bad5f6dd93ff69 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:20:43.729321 21010 consensus_queue.cc:1059] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:20:43.779464 15596 raft_consensus.cc:1240] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:43.785002 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:43.686363 21103 raft_consensus.cc:697] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:43.784940 15674 raft_consensus.cc:2376] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18. Re-sending same reply.
I20260709 13:20:43.786819 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:43.787480 15599 raft_consensus.cc:1240] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 14. Current term is 16. Ops: []
W20260709 13:20:43.787636 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 188022us
I20260709 13:20:43.796190 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:43.798074 15603 raft_consensus.cc:1240] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:20:43.800231 15594 raft_consensus.cc:2368] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12.
W20260709 13:20:43.801028 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:43.801653 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:43.802206 21161 consensus_queue.cc:1243] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer d3c4f5bd76e0445795bad5f6dd93ff69 log is divergent from this leader: its last log entry 15.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:20:43.810225 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:43.818317 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:43.820725 15602 raft_consensus.cc:1240] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:43.826187 21195 raft_consensus.cc:3055] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:43.826776 21195 raft_consensus.cc:740] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:43.827804 21195 consensus_queue.cc:260] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:43.829219 21195 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:43.826066 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:43.821352 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:43.838815 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:43.840523 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 123987us
I20260709 13:20:43.821991 21046 consensus_queue.cc:1243] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer d3c4f5bd76e0445795bad5f6dd93ff69 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:20:43.832911 16232 consensus_queue.cc:260] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.843415 21010 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.855842 15601 raft_consensus.cc:1240] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:43.857198 16237 raft_consensus.cc:2393] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 16: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:43.885403 16232 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:43.901639 15594 raft_consensus.cc:1240] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:20:43.904070 15670 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:20:43.927959 15672 raft_consensus.cc:3055] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:43.928391 15672 raft_consensus.cc:740] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:43.928481 21116 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.933682 21116 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:43.934470 15672 consensus_queue.cc:260] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.935425 15672 raft_consensus.cc:3060] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:43.878931 21010 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:43.940186 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:43.945365 16232 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:20:43.949641 15672 raft_consensus.cc:1275] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:20:43.950420 21161 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:20:43.952907 21180 consensus_queue.cc:1048] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:20:43.959983 20955 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:43.960657 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:43.964138 15567 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:43.957898 15669 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 21.
I20260709 13:20:43.916724 21103 consensus_queue.cc:237] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.967710 15567 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:43.969724 21188 log.cc:927] Time spent T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:20:43.975528 21181 log.cc:927] Time spent T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.229s	user 0.000s	sys 0.004s
I20260709 13:20:43.977610 21177 consensus_queue.cc:1059] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
W20260709 13:20:43.979440 21183 log.cc:927] Time spent T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.207s	user 0.002s	sys 0.000s
I20260709 13:20:43.980513 21144 raft_consensus.cc:3055] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:43.980960 21144 raft_consensus.cc:740] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:43.981859 21144 consensus_queue.cc:260] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.983040 21144 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:43.984261 21164 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:43.986554 21164 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:43.987716 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:43.988657 20955 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:43.997478 15668 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18.
I20260709 13:20:43.999379 21010 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:43.977953 20975 raft_consensus.cc:3055] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:44.002609 20975 raft_consensus.cc:740] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:44.003516 20975 consensus_queue.cc:260] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:44.004565 20975 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:44.005286 21030 raft_consensus.cc:3055] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:44.005677 21030 raft_consensus.cc:740] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:44.026472 21074 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:20:44.028203 15642 connection.cc:256] RPC call timeout handler was delayed by -4.3249637240000993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1567s for a response
W20260709 13:20:44.028563 15642 connection.cc:256] RPC call timeout handler was delayed by -4.1143637240002136s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1801s for a response
W20260709 13:20:44.028774 15642 connection.cc:256] RPC call timeout handler was delayed by -4.0630637240001306s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18580000000000002s for a response
W20260709 13:20:44.029004 15642 connection.cc:256] RPC call timeout handler was delayed by -4.0054637240000375s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1922s for a response
W20260709 13:20:44.029207 15642 connection.cc:256] RPC call timeout handler was delayed by -3.3961637240001892s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2599s for a response
I20260709 13:20:44.034837 15669 raft_consensus.cc:1240] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:44.034943 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.035684 15677 raft_consensus.cc:2393] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 18: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:44.035578 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.035836 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.038533 15670 raft_consensus.cc:2393] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 11: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:44.037796 15672 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:20:44.041555 21030 consensus_queue.cc:260] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:44.043058 21030 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:44.046072 21179 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:44.046631 21179 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:44.047053 21179 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:44.053136 21180 raft_consensus.cc:3055] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:44.053618 21180 raft_consensus.cc:740] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:44.054673 21180 consensus_queue.cc:260] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:44.055986 21180 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:44.053187 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.053572 21163 raft_consensus.cc:3055] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:44.063369 21163 raft_consensus.cc:740] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:44.064307 21163 consensus_queue.cc:260] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:44.065544 21163 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:44.067440 21076 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.068024 21076 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:44.070996 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:44.081048 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:44.087821 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:44.103374 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:44.106455 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.093456 21179 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:44.124872 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:44.131808 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:44.131094 15491 leader_election.cc:336] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 2.065s (SENT)
W20260709 13:20:44.141218 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:44.142652 15491 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:44.143553 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.121315 21074 raft_consensus.cc:697] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:44.152518 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.165067 20975 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:20:44.169191 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 112507us
I20260709 13:20:44.177105 15669 raft_consensus.cc:2410] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:20:44.181083 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.178565 21210 consensus_queue.cc:1048] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:20:44.188016 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.188795 15670 raft_consensus.cc:2393] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 7: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:44.192081 20980 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:20:44.192358 16203 leader_election.cc:336] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.724s (SENT)
I20260709 13:20:44.192610 20980 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:44.192982 20980 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:44.193413 21186 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.194033 21186 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:44.194590 21076 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:44.195281 21174 raft_consensus.cc:3055] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:44.195727 21174 raft_consensus.cc:740] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:20:44.196189 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 389981us
I20260709 13:20:44.196935 16204 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:44.196671 21174 consensus_queue.cc:260] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:44.197870 16204 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:20:44.200528 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 79433.9us
W20260709 13:20:44.195897 16203 leader_election.cc:336] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.838s (SENT)
I20260709 13:20:44.202682 21190 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:44.203266 21190 raft_consensus.cc:697] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:44.204145 21190 consensus_queue.cc:237] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:44.205843 21187 log.cc:927] Time spent T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.445s	user 0.001s	sys 0.005s
I20260709 13:20:44.198283 21074 consensus_queue.cc:237] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:44.211776 21191 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:44.212374 21191 raft_consensus.cc:697] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:44.213235 21191 consensus_queue.cc:237] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:44.215714 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:44.216351 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.217849 15670 raft_consensus.cc:2410] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:20:44.199688 21174 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:43.900831 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:20:44.234975 15491 leader_election.cc:336] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 2.157s (SENT)
I20260709 13:20:44.235531 15491 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:44.236933 15491 leader_election.cc:336] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 2.199s (SENT)
I20260709 13:20:44.237486 15491 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:44.238040 21118 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:44.237594 15671 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:20:44.215227 21179 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:44.248541 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:44.250026 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 54169.6us
I20260709 13:20:44.251160 20980 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:44.252058 21014 raft_consensus.cc:3055] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:44.257316 15668 raft_consensus.cc:1240] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:20:44.256595 15421 catalog_manager.cc:5697] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:44.257949 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.259212 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:44.262430 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:44.276840 15676 raft_consensus.cc:2410] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:20:44.267058 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.264684 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.276190 15671 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:44.282086 15675 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:20:44.282809 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.285815 15673 raft_consensus.cc:1275] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:20:44.275471 21014 raft_consensus.cc:740] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:44.294842 21014 consensus_queue.cc:260] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:44.295670 20980 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:44.295871 21014 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:44.279052 21118 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:20:44.300640 16203 leader_election.cc:336] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.890s (SENT)
W20260709 13:20:44.302021 16203 leader_election.cc:336] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.896s (SENT)
I20260709 13:20:44.302388 16203 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:20:44.303650 16203 leader_election.cc:336] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.908s (SENT)
I20260709 13:20:44.304015 16203 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:44.308530 15673 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:20:44.311337 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:44.311913 15595 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:44.308588 15671 raft_consensus.cc:1275] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:20:44.313031 15675 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:20:44.316815 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 119067us
I20260709 13:20:44.317612 16204 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:44.320670 16240 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:20:44.321095 16240 pending_rounds.cc:85] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9: Aborting all ops after (but not including) 12
I20260709 13:20:44.320741 16237 raft_consensus.cc:3055] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:44.321689 16237 raft_consensus.cc:740] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:44.321316 16240 pending_rounds.cc:107] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:20:44.322795 16240 raft_consensus.cc:2889] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:44.327477 15673 raft_consensus.cc:1240] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:20:44.313257 20350 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:44.333276 16231 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:20:44.313442 21175 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:44.341123 20975 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:44.340790 16237 consensus_queue.cc:260] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:44.341727 20975 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:44.342424 16237 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:44.347692 21212 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:44.349268 21100 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:44.350191 21104 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:44.350795 21104 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:44.350063 21059 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:44.351229 15589 tablet_service.cc:1558] Processing DeleteTablet for tablet b99eaeae24b44aa78cfa8c90caca288e with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 5 (current committed config index is 8)) from {username='slave'} at 127.0.0.1:53300
I20260709 13:20:44.334293 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:44.354089 16233 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:44.355692 15672 raft_consensus.cc:1240] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11. Ops: []
W20260709 13:20:44.356668 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:44.376621 16204 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:44.382836 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:44.385807 15602 raft_consensus.cc:1275] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:20:44.388334 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:44.390039 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:44.391613 15527 raft_consensus.cc:1275] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:20:44.382278 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:44.394186 15596 raft_consensus.cc:1275] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:20:44.376757 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:44.341535 15421 catalog_manager.cc:5697] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 5 to 8, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 8 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:44.399288 15595 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:44.399801 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:44.400561 15525 raft_consensus.cc:2393] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 16: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:44.401166 15599 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:20:44.402804 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.403429 15676 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
W20260709 13:20:44.403945 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:44.404856 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:44.405462 15675 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:20:44.392048 15527 pending_rounds.cc:85] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 6
I20260709 13:20:44.406426 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:44.406900 15597 raft_consensus.cc:1275] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:20:44.349182 21212 consensus_queue.cc:1048] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:20:44.373147 21116 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:44.410383 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:44.411263 15672 raft_consensus.cc:2410] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:20:44.411588 15601 raft_consensus.cc:1275] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:20:44.412241 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:44.375131 21145 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:20:44.413434 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 176545us
I20260709 13:20:44.410733 16238 raft_consensus.cc:2368] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 15. Current term is 16.
W20260709 13:20:44.408658 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 105053us
I20260709 13:20:44.406744 15527 pending_rounds.cc:107] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:20:44.415331 15527 raft_consensus.cc:2889] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:44.416589 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:44.372416 20664 consensus_queue.cc:1048] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:20:44.416080 15524 raft_consensus.cc:1275] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:20:44.422161 15526 raft_consensus.cc:3055] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:44.422617 15526 raft_consensus.cc:740] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:44.424110 16204 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:44.404986 21038 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.428762 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:44.430675 15525 raft_consensus.cc:2368] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 15. Current term is 16.
I20260709 13:20:44.375255 20350 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:20:44.391039 21059 consensus_queue.cc:1048] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:20:44.419137 21104 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:44.443513 16237 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:44.440826 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:44.448928 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.447116 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:44.451078 15524 raft_consensus.cc:3060] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:44.450999 15525 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:44.450201 15491 leader_election.cc:336] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 2.236s (SENT)
I20260709 13:20:44.455988 15491 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:44.456496 15527 raft_consensus.cc:1240] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:44.462396 15567 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:44.465197 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:44.468427 15567 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:44.485004 15568 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:44.441747 21186 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:44.440943 21116 raft_consensus.cc:697] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:44.492377 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:44.430485 21190 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 13:20:44.494877 15676 raft_consensus.cc:2393] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 11: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:44.495565 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.453758 21191 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:44.510555 15671 raft_consensus.cc:2393] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 12: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:44.482570 15526 consensus_queue.cc:260] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:44.440362 21038 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:44.522759 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:44.517254 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:44.516141 15675 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 15.
I20260709 13:20:44.482978 21175 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:44.443895 21121 consensus_queue.cc:1048] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.053s
W20260709 13:20:44.538973 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 94316.6us
I20260709 13:20:44.541637 15527 raft_consensus.cc:1240] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13. Ops: []
W20260709 13:20:44.545735 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 61038.6us
I20260709 13:20:44.546717 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.552044 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.552870 15672 raft_consensus.cc:2410] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:20:44.553282 15526 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:44.459141 20955 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 13:20:44.502166 21104 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:44.556466 16204 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:44.557236 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.560226 16237 raft_consensus.cc:1275] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:44.516112 21180 raft_consensus.cc:2804] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:44.561240 15676 raft_consensus.cc:2393] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 7: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:44.551154 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:44.496778 21074 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:44.508985 21212 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:20:44.486455 20884 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:20:44.537626 21191 raft_consensus.cc:697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:20:44.574288 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:44.510023 21194 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:44.577713 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.579627 21038 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:44.582048 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:44.582261 21221 log.cc:927] Time spent T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 13:20:44.563689 21180 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:44.583042 21180 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:44.508116 21010 consensus_queue.cc:1059] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 13:20:44.583961 21010 raft_consensus.cc:3055] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:44.584314 21010 raft_consensus.cc:740] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:44.585120 21010 consensus_queue.cc:260] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:44.574887 21116 consensus_queue.cc:237] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:44.588872 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:44.591312 21218 log.cc:927] Time spent T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 13:20:44.583990 15525 raft_consensus.cc:1275] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:20:44.592607 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.593395 15672 raft_consensus.cc:2393] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 11: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:44.581298 21191 consensus_queue.cc:237] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:44.594029 20975 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:44.595080 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.595233 21186 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:44.595984 15675 raft_consensus.cc:2410] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:20:44.595786 21186 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:44.596840 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.598201 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.598784 15671 raft_consensus.cc:2468] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:20:44.599627 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:44.600777 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.601895 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.603052 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.604713 15675 raft_consensus.cc:2393] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 18: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:44.601641 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.613497 15524 raft_consensus.cc:2468] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
W20260709 13:20:44.620121 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 180959us
I20260709 13:20:44.622555 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.623404 15669 raft_consensus.cc:2376] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 15. Re-sending same reply.
W20260709 13:20:44.624261 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:44.582002 15426 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49140 (request call id 363) took 10452 ms. Trace:
I20260709 13:20:44.628777 15671 raft_consensus.cc:2393] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 18: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:44.615109 15641 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:44.628794 15519 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:44.637748 15526 raft_consensus.cc:1275] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:20:44.638326 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:44.648339 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:44.649179 15672 raft_consensus.cc:2393] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 7: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:44.650632 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:44.644701 21161 raft_consensus.cc:2804] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:44.653277 21161 raft_consensus.cc:697] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:44.653559 15674 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 15.
I20260709 13:20:44.654626 21010 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:44.654251 21161 consensus_queue.cc:237] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:44.654858 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:44.655894 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:44.658171 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:44.659806 21220 log.cc:927] Time spent T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:20:44.649977 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.663450 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:44.594061 21175 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:44.662596 15671 raft_consensus.cc:2468] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:44.670149 15671 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:44.671695 15669 raft_consensus.cc:1275] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:20:44.668004 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 199646us
I20260709 13:20:44.675493 15493 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:44.677349 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:44.672022 15568 leader_election.cc:400] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 15. Current term is 16.
I20260709 13:20:44.678673 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:44.681908 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:44.673635 15412 catalog_manager.cc:5028] TS 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): tablet b99eaeae24b44aa78cfa8c90caca288e (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:20:44.684027 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:44.685220 16231 raft_consensus.cc:2393] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 11: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:44.602149 21144 consensus_queue.cc:1048] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 13:20:44.663117 20975 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.686105 20955 consensus_queue.cc:1048] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:20:44.686328 16236 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:20:44.683658 16238 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:20:44.683490 21074 consensus_queue.cc:1048] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:20:44.703032 15672 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:20:44.705467 15672 raft_consensus.cc:1275] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:20:44.707801 16203 leader_election.cc:336] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 2.013s (SENT)
I20260709 13:20:44.708423 16203 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:44.715026 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:44.715276 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:44.715574 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:44.715847 16238 raft_consensus.cc:2393] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 7: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:44.716572 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:44.716646 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:44.690752 21203 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 13:20:44.721071 21186 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:44.628679 15426 rpcz_store.cc:276] 0709 13:20:34.014584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:34.043093 (+ 28509us) service_pool.cc:224] Handling call
0709 13:20:44.454825 (+10411732us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:38.470751 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:38.471144 (+   393us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:38.471166 (+    22us) write_op.cc:435] Acquired schema lock
0709 13:20:38.471177 (+    11us) tablet.cc:662] Decoding operations
0709 13:20:38.472128 (+   951us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:38.472188 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:38.472205 (+    17us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:20:38.473392 (+  1187us) tablet.cc:701] Row locks acquired
0709 13:20:38.473419 (+    27us) write_op.cc:260] PREPARE: finished
0709 13:20:38.473587 (+   168us) write_op.cc:270] Start()
0709 13:20:38.473701 (+   114us) write_op.cc:276] Timestamp: P: 1783603238473545 usec, L: 0
0709 13:20:38.473719 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:20:38.474457 (+   738us) log.cc:844] Serialized 6634 byte log entry
0709 13:20:39.447970 (+973513us) op_driver.cc:464] REPLICATION: finished
0709 13:20:39.681611 (+233641us) write_op.cc:301] APPLY: starting
0709 13:20:39.685662 (+  4051us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:39.742265 (+ 56603us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:39.742362 (+    97us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:39.764906 (+ 22544us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:39.765203 (+   297us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=0,mrs_lookups=0
0709 13:20:39.765261 (+    58us) write_op.cc:312] APPLY: finished
0709 13:20:39.765631 (+   370us) log.cc:844] Serialized 156 byte log entry
0709 13:20:39.766705 (+  1074us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:39.766874 (+   169us) write_op.cc:454] Released schema lock
0709 13:20:39.766928 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:39.767388 (+   460us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:39.767416 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":148918,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":219691,"apply.run_cpu_time_us":21376,"apply.run_wall_time_us":88758,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75323,"num_ops":17,"prepare.queue_time_us":11625,"prepare.run_cpu_time_us":27275,"prepare.run_wall_time_us":110441,"raft.queue_time_us":115463,"raft.run_cpu_time_us":14807,"raft.run_wall_time_us":840128,"replication_time_us":342661,"thread_start_us":344573,"threads_started":5,"wal-append.queue_time_us":151095,"wal-append.run_cpu_time_us":20916,"wal-append.run_wall_time_us":2113537}]]}
W20260709 13:20:44.723769 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:44.724226 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:44.719545 15676 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 16.
I20260709 13:20:44.724869 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:44.694621 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:44.696943 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:44.685638 15568 leader_election.cc:403] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:20:44.728057 15567 leader_election.cc:400] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 15. Current term is 16.
I20260709 13:20:44.728220 16231 raft_consensus.cc:1240] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:44.683643 21109 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.691920 21175 raft_consensus.cc:697] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:44.685848 21038 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
W20260709 13:20:44.758242 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 50487us
W20260709 13:20:44.832195 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 160297us
I20260709 13:20:44.727169 20975 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:44.844806 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:44.872624 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 193791us
W20260709 13:20:44.867604 21225 log.cc:927] Time spent T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:20:44.743383 21109 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:44.900913 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:44.891291 21175 consensus_queue.cc:237] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:44.750458 21121 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:44.743595 21186 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:44.908387 15669 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:44.922183 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.922919 15669 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:44.913267 15677 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:44.925681 21109 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:44.910907 15671 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 15.
I20260709 13:20:44.914719 21203 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:44.938408 16203 leader_election.cc:336] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 2.037s (SENT)
I20260709 13:20:44.940428 15671 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:20:44.946393 16203 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:44.929251 20955 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:20:44.928615 21074 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:20:44.957373 21217 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:44.974756 20975 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:44.985111 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.986394 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:44.986979 15673 raft_consensus.cc:2376] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 15. Re-sending same reply.
I20260709 13:20:44.987797 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:44.988418 15673 raft_consensus.cc:2393] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 12: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:44.989169 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:44.990896 15600 raft_consensus.cc:1240] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:44.991102 16205 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:44.996170 21038 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.996788 21038 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:44.948534 21230 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.016s
I20260709 13:20:45.007462 21194 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:20:45.010812 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 333430us
I20260709 13:20:45.011900 15567 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:45.014814 16232 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:20:45.017836 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:45.018736 15675 raft_consensus.cc:2393] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 11: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:45.019620 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:45.020133 15675 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:45.025048 21230 consensus_queue.cc:1048] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
W20260709 13:20:45.031826 21219 log.cc:927] Time spent T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.570s	user 0.000s	sys 0.002s
I20260709 13:20:45.036084 20664 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:45.036697 20664 raft_consensus.cc:697] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:20:45.037040 16203 leader_election.cc:336] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 2.193s (SENT)
I20260709 13:20:45.037462 16203 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:45.037603 20664 consensus_queue.cc:237] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.039974 21038 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:45.042315 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:45.106034 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:45.112576 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 90696.4us
I20260709 13:20:45.113297 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:20:45.117805 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:45.119629 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 77694.6us
I20260709 13:20:45.110563 21121 raft_consensus.cc:697] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:45.166143 15675 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:20:45.171103 15675 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:45.173697 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:45.112215 21194 consensus_queue.cc:1048] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:20:45.125458 21190 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:45.070784 21231 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:20:45.180226 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:45.140776 21116 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.184715 15675 raft_consensus.cc:1275] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:20:45.164279 21100 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
W20260709 13:20:45.185989 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 148995us
I20260709 13:20:45.186672 15675 raft_consensus.cc:1275] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:20:45.187050 15675 pending_rounds.cc:85] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting all ops after (but not including) 7
I20260709 13:20:45.187342 15675 pending_rounds.cc:107] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:20:45.187690 15675 raft_consensus.cc:2889] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:45.189188 15675 raft_consensus.cc:1275] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:20:45.191560 16240 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:20:45.192039 15675 raft_consensus.cc:1275] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:20:45.201210 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:45.123524 21076 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.059s
W20260709 13:20:45.184041 21180 consensus_meta.cc:287] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.597s	user 0.011s	sys 0.000s
I20260709 13:20:45.205451 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:45.217550 15520 raft_consensus.cc:1275] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:20:45.222436 15525 raft_consensus.cc:1275] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:20:45.256280 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:45.261433 15526 raft_consensus.cc:1275] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:20:45.267165 15523 raft_consensus.cc:1275] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:20:45.270222 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:45.280633 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:45.231349 21249 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.256131 21247 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.231056 21014 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:20:45.291366 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:45.200028 21116 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:45.302467 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 78812.4us
I20260709 13:20:45.308470 20350 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:45.303148 21014 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:45.308894 21227 log.cc:927] Time spent T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:20:45.309450 21235 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.310063 21235 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.249717 21180 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.293146 21249 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.309054 20350 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.293556 21194 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:45.340658 21098 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.342123 21116 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:45.344022 21014 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:45.345504 16203 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:45.269724 21175 consensus_queue.cc:1059] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:20:45.352110 21236 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:45.358105 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 54590.7us
I20260709 13:20:45.359836 21246 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.366910 21246 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.367355 21038 consensus_queue.cc:1048] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:45.368861 21038 consensus_queue.cc:1048] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:45.369227 21246 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:45.354624 21261 log.cc:927] Time spent T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:20:45.328119 20884 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:45.374281 16203 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:45.302510 21121 consensus_queue.cc:237] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.315250 21119 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:20:45.297963 21247 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.358541 21174 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:45.341369 21098 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.369653 21262 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:20:45.387800 21014 consensus_queue.cc:1048] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:20:45.394090 21104 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:45.395645 21247 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:45.388978 21246 consensus_queue.cc:1048] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
W20260709 13:20:45.401690 21232 log.cc:927] Time spent T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 13:20:45.346561 21194 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.408768 21104 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.400939 21014 consensus_queue.cc:1243] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer d3c4f5bd76e0445795bad5f6dd93ff69 log is divergent from this leader: its last log entry 8.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:20:45.401103 21098 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:45.413252 15668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2855) took 1054 ms. Trace:
I20260709 13:20:45.413722 15668 rpcz_store.cc:276] 0709 13:20:44.359407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:44.359841 (+   434us) service_pool.cc:224] Handling call
0709 13:20:45.183176 (+823335us) spinlock_profiling.cc:243] Waited 1.68 ms on lock 0x7b5800012398. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99e4fc0 00007f5ce99cf929 00007f5ce99cf3d2 00007f5ce2f46bd4
0709 13:20:45.183405 (+   229us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:45.191827 (+  8422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:45.191859 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:45.192313 (+   454us) log.cc:844] Serialized 26 byte log entry
0709 13:20:45.264531 (+ 72218us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:45.278641 (+ 14110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:45.278741 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:45.399078 (+120337us) raft_consensus.cc:1685] finished
0709 13:20:45.399102 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:45.413168 (+ 14066us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8075,"raft.queue_time_us":9066,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":7482,"spinlock_wait_cycles":3868928,"thread_start_us":71818,"threads_started":1,"wal-append.queue_time_us":177720}
I20260709 13:20:45.383208 20884 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.383610 21175 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:45.449010 21098 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.449443 21257 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.450204 21257 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.451351 21249 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:45.451731 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 53050.2us
I20260709 13:20:45.450999 21098 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.452431 21257 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:45.371237 21236 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.452951 21180 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:45.391921 21174 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:45.453912 21174 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:45.454244 21236 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:45.454397 21098 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:45.452476 16203 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:45.456779 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:45.457898 15521 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 18.
I20260709 13:20:45.455425 21175 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:20:45.463603 21203 consensus_meta.cc:287] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.519s	user 0.007s	sys 0.006s
I20260709 13:20:45.463745 21249 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:45.480507 21235 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:45.480980 20884 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:45.473340 21255 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.489817 21266 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.473326 20975 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:45.474268 21175 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:45.492022 21249 consensus_queue.cc:1048] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:45.476557 21194 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:45.513016 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:45.482088 21203 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 15. Current term is 16.
W20260709 13:20:45.527724 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 173318us
W20260709 13:20:45.527309 21259 log.cc:927] Time spent T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 13:20:45.533771 15491 leader_election.cc:336] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 2.424s (SENT)
I20260709 13:20:45.535476 15491 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:45.537039 21245 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:45.514222 21255 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:45.539397 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:45.517046 20975 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:45.516256 21266 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:45.539888 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:45.544137 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:45.544790 15674 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:20:45.544934 15671 raft_consensus.cc:1275] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:20:45.546571 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 83139us
I20260709 13:20:45.549734 15677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 2366) took 1132 ms. Trace:
I20260709 13:20:45.551283 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:45.550163 15677 rpcz_store.cc:276] 0709 13:20:44.411892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:44.412007 (+   115us) service_pool.cc:224] Handling call
0709 13:20:45.544303 (+1132296us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b540046da54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:45.544433 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":751380736}
I20260709 13:20:45.551959 15520 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:45.553716 21228 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.554301 21228 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:45.555953 20350 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:45.556392 21228 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:45.540316 21014 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.558831 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:45.559448 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:45.534363 21194 consensus_queue.cc:1048] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:20:45.565212 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:45.567039 15595 raft_consensus.cc:2393] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 12: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:45.536586 21059 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:20:45.538053 21203 consensus_queue.cc:1048] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:20:45.565687 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:45.548537 20887 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:45.575930 15523 raft_consensus.cc:2393] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 11: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:45.576159 20887 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.578750 20887 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:45.580061 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:45.580298 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:45.580753 15524 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
W20260709 13:20:45.581965 21269 log.cc:927] Time spent T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:20:45.553167 21104 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:45.582687 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:45.606163 21192 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.628031 21192 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:45.629856 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:45.605849 15520 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
W20260709 13:20:45.630959 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:45.640089 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:45.653615 21192 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:45.658072 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:45.608016 21281 log.cc:927] Time spent T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:20:45.662357 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:45.665292 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:45.665663 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:45.668756 15676 raft_consensus.cc:2376] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13. Re-sending same reply.
W20260709 13:20:45.670361 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 111007us
I20260709 13:20:45.666884 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:45.671753 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:45.672161 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:45.671140 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:45.672408 15522 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
W20260709 13:20:45.671089 15491 leader_election.cc:336] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 2.987s (SENT)
I20260709 13:20:45.677471 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:45.678203 15674 raft_consensus.cc:2393] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 7: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:45.619258 21255 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:45.682138 21287 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:45.683259 21116 raft_consensus.cc:3055] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:45.683676 21116 raft_consensus.cc:740] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:45.684165 21287 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:45.684629 21116 consensus_queue.cc:260] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.688311 15676 raft_consensus.cc:1275] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:20:45.570495 21014 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:45.695930 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:45.712683 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:45.737844 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:45.693825 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:45.699822 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:45.690397 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:45.685643 21287 consensus_queue.cc:1048] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
W20260709 13:20:45.738874 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 64936.3us
I20260709 13:20:45.766309 16240 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:45.691092 15660 tablet_service.cc:1558] Processing DeleteTablet for tablet bd36c2cfd99c45cea9e48096d54bed1a with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 5 (current committed config index is 6)) from {username='slave'} at 127.0.0.1:49088
W20260709 13:20:45.780656 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:45.774015 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:45.769586 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:45.780073 15491 leader_election.cc:336] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 3.167s (SENT)
I20260709 13:20:45.778625 15566 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:45.782375 15491 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:45.691183 15664 tablet_service.cc:1558] Processing DeleteTablet for tablet 0af5649725244ae599dfc6ed7aeac5f1 with delete_type TABLET_DATA_TOMBSTONED (TS 3ef186ec12554a1d87814af0a0ace9a4 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:49088
I20260709 13:20:45.783681 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:45.784364 15668 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:45.788236 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:45.790325 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 58326.5us
I20260709 13:20:45.792434 21121 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.770776 21175 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.793006 21121 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:45.766808 21191 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:45.794740 21175 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:45.795254 21121 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:45.795078 21191 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.797376 21191 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:45.768303 21116 raft_consensus.cc:3060] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:45.690721 21174 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.766992 21104 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.829553 15669 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:45.829892 21121 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.829325 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:45.827301 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 136986us
I20260709 13:20:45.836057 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:45.831873 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:45.834394 15677 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:20:45.834579 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:45.838780 15676 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:20:45.841758 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:45.805570 21161 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:45.808192 21098 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:45.826902 21103 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:45.851116 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:45.769765 21217 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 13:20:45.832792 21177 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:45.834108 21022 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.860375 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:45.833824 21074 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:45.863327 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:45.863993 15676 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:20:45.865381 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:45.867386 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:45.868067 16232 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:45.871094 15675 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 15.
W20260709 13:20:45.873339 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 93299.6us
I20260709 13:20:45.879153 16240 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:45.882306 16237 raft_consensus.cc:1240] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:20:45.861304 21175 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.857278 21177 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:45.881098 20887 consensus_queue.cc:1048] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:20:45.843480 21121 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.944029 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:45.862046 21161 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:45.932382 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 59489.8us
I20260709 13:20:45.839957 21104 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:45.950755 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 67137.5us
W20260709 13:20:45.953159 15491 leader_election.cc:336] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 3.257s (SENT)
I20260709 13:20:45.954896 15671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49094 (request call id 2391) took 1019 ms. Trace:
I20260709 13:20:45.887485 21266 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:45.930539 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:45.938335 21177 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:45.865264 21103 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.944788 16240 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:20:45.955402 15671 rpcz_store.cc:276] 0709 13:20:44.936634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:44.941239 (+  4605us) service_pool.cc:224] Handling call
0709 13:20:45.544578 (+603339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:45.545288 (+   710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:45.954671 (+409383us) spinlock_profiling.cc:243] Waited 603 ms on lock 0x7b540046da50. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:45.954831 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1386958592}
I20260709 13:20:45.970225 15671 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:20:45.958523 15642 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:45.966393 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:45.874073 21074 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.978540 15526 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:45.978444 15603 raft_consensus.cc:1240] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:20:45.870193 21022 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:45.976907 16203 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:45.974408 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:45.900027 21175 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:46.010403 15491 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 3.301s (SENT) [suppressed 71 similar messages]
I20260709 13:20:45.890308 15677 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18.
I20260709 13:20:45.986253 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:46.037410 15594 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 16.
W20260709 13:20:46.049954 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:45.975483 21287 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 13:20:46.051872 15599 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:20:46.004781 21245 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:46.060562 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:46.010917 21217 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:20:46.084561 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 118911us
I20260709 13:20:46.085399 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:46.087266 15603 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:46.088903 15672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2864) took 1384 ms. Trace:
I20260709 13:20:46.052304 21191 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:46.089706 15672 rpcz_store.cc:276] 0709 13:20:44.704087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:44.706635 (+  2548us) service_pool.cc:224] Handling call
0709 13:20:44.706957 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:44.707606 (+   649us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:20:44.742312 (+ 34706us) log.cc:844] Serialized 14 byte log entry
0709 13:20:45.614001 (+871689us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b18002008c0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab23a 00007f5ce4a15a2a
0709 13:20:45.907619 (+293618us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:45.943760 (+ 36141us) log.cc:844] Serialized 26 byte log entry
0709 13:20:45.943956 (+   196us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:45.943993 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:45.944064 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:46.061153 (+117089us) raft_consensus.cc:1685] finished
0709 13:20:46.088682 (+ 27529us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:46.088838 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17928,"prepare.run_cpu_time_us":12731,"prepare.run_wall_time_us":148225,"raft.queue_time_us":243710,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":3885,"spinlock_wait_cycles":5061888,"thread_start_us":293305,"threads_started":2,"wal-append.queue_time_us":1100548}
I20260709 13:20:46.099856 21298 tablet_replica.cc:333] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
W20260709 13:20:46.084470 21290 log.cc:927] Time spent T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:20:46.103569 16203 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:46.112195 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 160740us
I20260709 13:20:46.085438 21174 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:46.117769 15568 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:46.119515 20887 raft_consensus.cc:2804] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:46.120070 20887 raft_consensus.cc:697] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:46.120441 16236 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:20:46.120993 20887 consensus_queue.cc:237] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:46.134083 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:46.140625 21023 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:20:46.169469 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:46.172889 21174 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:46.173651 21174 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:46.176066 21174 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:46.176622 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:46.110319 21292 log.cc:927] Time spent T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:20:46.181885 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:46.101418 21175 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:46.184948 21121 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:46.185266 21298 raft_consensus.cc:2243] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:46.185737 21290 log.cc:927] Time spent T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:20:46.187943 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:46.190920 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:46.107666 21267 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:46.111961 21217 raft_consensus.cc:2804] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:46.110635 21014 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:46.197016 15594 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
W20260709 13:20:46.209383 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 257874us
I20260709 13:20:46.215770 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:46.216557 15672 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 21.
I20260709 13:20:46.108932 21191 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:46.192397 21174 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:46.233621 21104 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:46.196138 21301 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:46.201097 21217 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:46.198490 21302 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:46.245532 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:46.252748 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:46.252789 15677 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:46.262035 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:46.241448 21307 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:46.274376 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 315936us
W20260709 13:20:46.275287 15491 leader_election.cc:336] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.507s (SENT)
I20260709 13:20:46.253858 21217 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:46.302556 15525 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:20:46.358793 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:46.324455 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:46.359735 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:46.363592 15521 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:20:46.364648 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:46.365767 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:46.367101 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:46.367981 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:46.370591 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2394) took 1931 ms. Trace:
I20260709 13:20:46.371131 15527 rpcz_store.cc:276] 0709 13:20:44.438734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:44.568174 (+129440us) service_pool.cc:224] Handling call
0709 13:20:44.590821 (+ 22647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:44.591021 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:20:44.766387 (+175366us) log.cc:844] Serialized 14 byte log entry
0709 13:20:45.950594 (+1184207us) spinlock_profiling.cc:243] Waited 6.52 ms on lock 0x7b180038d760. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce45b34bc 00007f5ce190148b 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191b455 00007f5ce4a03060 00007f5ce4a15716 00007f5ce4a26287
0709 13:20:46.125171 (+174577us) log.cc:844] Serialized 14 byte log entry
0709 13:20:46.362148 (+236977us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:46.362252 (+   104us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:46.362334 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:46.362462 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:46.370473 (+  8011us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":102385,"prepare.queue_time_us":418645,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":68,"raft.queue_time_us":91607,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":38949,"spinlock_wait_cycles":14985600,"thread_start_us":782583,"threads_started":4}
I20260709 13:20:46.257659 21161 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:46.240175 21174 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:46.257741 21317 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:46.379583 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:46.377976 15525 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:20:46.238926 21267 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:46.380533 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 59181.2us
W20260709 13:20:46.379053 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 103814us
I20260709 13:20:46.398304 20887 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:20:46.401619 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 183437us
W20260709 13:20:46.402496 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:46.403199 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:46.403798 15491 leader_election.cc:336] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.682s (SENT)
I20260709 13:20:46.404832 16204 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:46.378968 21074 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:46.258272 21302 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:46.482415 15596 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:20:46.437356 15677 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:46.412197 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:46.482152 16204 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:46.483361 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:46.487380 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 100404us
I20260709 13:20:46.488123 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:46.490844 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 103725us
W20260709 13:20:46.492954 15491 leader_election.cc:336] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.708s (SENT)
I20260709 13:20:46.493373 15491 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:46.484298 16239 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:46.387045 21298 raft_consensus.cc:2272] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:46.506822 15603 raft_consensus.cc:1240] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:46.367480 21307 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:46.510239 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:46.514261 21279 log.cc:927] Time spent T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.852s	user 0.005s	sys 0.000s
W20260709 13:20:46.518656 15491 leader_election.cc:336] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.746s (SENT)
I20260709 13:20:46.532622 15601 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 16.
W20260709 13:20:46.520344 21306 log.cc:927] Time spent T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.278s	user 0.000s	sys 0.001s
I20260709 13:20:46.528211 21174 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:46.512287 21175 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:46.519490 15598 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:46.532805 15641 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:20:46.531375 21285 log.cc:927] Time spent T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.873s	user 0.000s	sys 0.006s
I20260709 13:20:46.546751 15603 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:20:46.542112 15493 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:46.392094 21098 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:46.541371 15601 raft_consensus.cc:1275] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:20:46.447050 21334 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:46.574065 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:46.575062 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:46.575759 15493 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:46.576363 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:46.576774 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 94716.2us
I20260709 13:20:46.542318 21175 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:46.539935 21314 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:20:46.605806 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:46.606206 15642 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:46.394176 21317 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:46.607468 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:46.548063 21301 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:46.608227 16204 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:46.584684 21121 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:46.551214 21235 consensus_queue.cc:1059] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:20:46.605485 21104 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:46.544636 21316 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:46.636823 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2395) took 1430 ms. Trace:
I20260709 13:20:46.611330 21175 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:46.604179 21098 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:46.690502 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:46.690376 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 327422us
I20260709 13:20:46.633826 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:46.636384 21322 log.cc:927] Time spent T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 13:20:46.635941 20975 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:46.704684 15566 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:46.634145 21320 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:20:46.644874 21319 log.cc:927] Time spent T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
W20260709 13:20:46.686230 21300 log.cc:927] Time spent T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.328s	user 0.000s	sys 0.001s
I20260709 13:20:46.689159 21307 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:46.644668 21191 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:46.686030 21326 raft_consensus.cc:2804] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:46.713236 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:46.635427 21267 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:46.714462 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 108357us
W20260709 13:20:46.716658 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 74297.2us
I20260709 13:20:46.644971 21301 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:46.697243 21104 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:46.737994 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:46.698086 15528 rpcz_store.cc:276] 0709 13:20:45.206557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.230813 (+ 24256us) service_pool.cc:224] Handling call
0709 13:20:45.265339 (+ 34526us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:45.310359 (+ 45020us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:45.310397 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:45.310890 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 13:20:45.495635 (+184745us) spinlock_profiling.cc:243] Waited 5.15 ms on lock 0x7b4800002958. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce9934aa1 00007f5ce992bb37 00007f5ce992b9ca 00007f5ce9a2326f 00007f5ce9a22f37 00007f5ce9a3755c 00007f5ce9a37508 00007f5ce9a37488 00007f5ce9a37443 00007f5ce9a366ec 00007f5ce8960a96
0709 13:20:45.535251 (+ 39616us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:45.535291 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:45.535462 (+   171us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:46.513552 (+978090us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:20:46.636613 (+123061us) raft_consensus.cc:1685] finished
0709 13:20:46.636634 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:46.636766 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40617,"raft.queue_time_us":80846,"raft.run_cpu_time_us":1020,"raft.run_wall_time_us":146051,"spinlock_wait_cycles":42510976,"thread_start_us":223996,"threads_started":1,"wal-append.queue_time_us":264032}
W20260709 13:20:46.785090 21281 log.cc:927] Time spent T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 13:20:46.714408 21327 log.cc:927] Time spent T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
W20260709 13:20:46.715837 21303 log.cc:927] Time spent T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.268s	user 0.000s	sys 0.001s
W20260709 13:20:46.792013 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 78842.6us
W20260709 13:20:46.801242 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:46.649394 21314 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:46.785710 21014 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:46.857954 21014 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:46.724078 21339 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:20:46.822172 21332 log.cc:927] Time spent T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:20:46.823073 21314 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:46.823362 21341 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:46.859524 21341 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:46.860190 21014 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:46.823781 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:46.788997 21326 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:46.795063 21235 raft_consensus.cc:3055] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:46.825420 16238 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:20:46.803457 21104 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:46.826785 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:46.827610 21317 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:46.827831 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:46.857237 15596 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:20:46.861696 21103 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:46.826859 15642 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:46.831213 21116 consensus_meta.cc:287] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.996s	user 0.011s	sys 0.000s
I20260709 13:20:46.801077 21301 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:46.836973 21217 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:46.698087 21121 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:46.842852 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:46.843777 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:46.844237 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1870) took 1290 ms. Trace:
I20260709 13:20:46.845070 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:46.874467 16205 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:46.880906 15670 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:20:46.854921 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:46.857395 21298 ts_tablet_manager.cc:1916] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:46.858964 21339 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:46.815467 15566 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:46.864588 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:46.870545 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 43757.9us
I20260709 13:20:46.951526 15642 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:46.878779 15598 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:20:46.791191 20975 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:46.880108 15522 rpcz_store.cc:276] 0709 13:20:45.553742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.970060 (+416318us) service_pool.cc:224] Handling call
0709 13:20:46.102553 (+132493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:46.107692 (+  5139us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:46.107735 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:46.180598 (+ 72863us) log.cc:844] Serialized 26 byte log entry
0709 13:20:46.557623 (+377025us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:46.557670 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:46.557758 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:46.844011 (+286253us) raft_consensus.cc:1685] finished
0709 13:20:46.844030 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:46.844173 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":90101,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":2824,"thread_start_us":350800,"threads_started":1,"wal-append.queue_time_us":600504}
I20260709 13:20:46.788017 21161 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:20:46.944092 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:46.943994 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:46.946069 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 370925us
I20260709 13:20:46.862737 21235 raft_consensus.cc:740] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:46.879400 21317 raft_consensus.cc:2764] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:20:46.862654 21326 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:46.885571 16233 raft_consensus.cc:1275] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:20:46.791756 21323 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:46.861877 21341 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:46.947649 16205 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:46.723192 21320 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:46.882612 21302 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:46.956665 15595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 2292) took 1092 ms. Trace:
W20260709 13:20:46.951284 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 135924us
I20260709 13:20:46.971359 21323 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:46.970749 15595 rpcz_store.cc:276] 0709 13:20:45.858647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.858796 (+   149us) service_pool.cc:224] Handling call
0709 13:20:45.860134 (+  1338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:45.872624 (+ 12490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:45.872654 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:45.886439 (+ 13785us) log.cc:844] Serialized 26 byte log entry
0709 13:20:46.517970 (+631531us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b1800320fe0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab0d1 00007f5ce49c6a0f
0709 13:20:46.638537 (+120567us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:46.943973 (+305436us) log.cc:844] Serialized 14 byte log entry
0709 13:20:46.945677 (+  1704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:46.945746 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:46.945783 (+    37us) raft_consensus.cc:1685] finished
0709 13:20:46.950513 (+  4730us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:46.950668 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12262,"raft.queue_time_us":12364,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":20945,"spinlock_wait_cycles":19543680,"thread_start_us":360150,"threads_started":2,"wal-append.queue_time_us":932095}
W20260709 13:20:46.973626 21289 log.cc:927] Time spent T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7: Append to log took a long time: real 1.319s	user 0.002s	sys 0.003s
W20260709 13:20:46.957165 21177 consensus_meta.cc:287] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.953s	user 0.011s	sys 0.000s
I20260709 13:20:46.954171 21116 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:46.969444 21022 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:46.973812 21341 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:46.966781 21307 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:47.001870 21301 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:46.980489 21316 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:20:46.956467 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:47.005558 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:47.006176 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:47.006722 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:47.007359 15524 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 17.
I20260709 13:20:47.009400 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:46.967372 15493 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:46.987141 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1865) took 1434 ms. Trace:
I20260709 13:20:47.010818 15519 rpcz_store.cc:276] 0709 13:20:45.552587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.961069 (+408482us) service_pool.cc:224] Handling call
0709 13:20:45.973661 (+ 12592us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:46.159848 (+186187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:46.159878 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:46.160405 (+   527us) log.cc:844] Serialized 26 byte log entry
0709 13:20:46.384413 (+224008us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:46.384478 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:46.384594 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:46.986882 (+602288us) raft_consensus.cc:1685] finished
0709 13:20:46.986904 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:46.987071 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":136982,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":66979,"thread_start_us":409001,"threads_started":2,"wal-append.queue_time_us":426085}
I20260709 13:20:46.955770 20975 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:46.957000 21100 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:47.013121 21100 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:47.013518 21100 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:47.015018 21304 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:47.015575 21304 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:46.954710 21339 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:46.992653 21177 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:47.019135 21177 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:47.019848 15600 raft_consensus.cc:1240] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:20:46.973626 21323 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:46.996441 21217 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:46.997088 15597 raft_consensus.cc:1275] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:20:46.970865 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 44798.4us
I20260709 13:20:47.021198 15597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 2284) took 1460 ms. Trace:
I20260709 13:20:47.021540 15597 rpcz_store.cc:276] 0709 13:20:45.560619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.560737 (+   118us) service_pool.cc:224] Handling call
0709 13:20:46.996541 (+1435804us) spinlock_profiling.cc:243] Waited 3.33 ms on lock 0x7b5800113498. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99e4fc0 00007f5ce99cf929 00007f5ce99cf3d2 00007f5ce2f46bd4
0709 13:20:46.996736 (+   195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:47.020911 (+ 24175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.021137 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7661952}
I20260709 13:20:47.022908 21249 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:47.024096 16205 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:47.024937 21318 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:47.025450 21318 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:47.033605 21339 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:47.036262 21339 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:47.037093 20975 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:47.037782 15676 raft_consensus.cc:1240] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:46.997081 21121 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:47.039111 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 71812.6us
I20260709 13:20:47.042356 15493 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:46.993854 21116 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:47.040259 21316 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:47.080797 15675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 2403) took 1418 ms. Trace:
I20260709 13:20:47.086901 15675 rpcz_store.cc:276] 0709 13:20:45.661958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.662631 (+   673us) service_pool.cc:224] Handling call
0709 13:20:47.080616 (+1417985us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b540042a254. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:47.080738 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":296812800}
I20260709 13:20:47.088335 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:47.089399 15675 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:47.090586 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 139158us
I20260709 13:20:47.062377 21287 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:47.091518 15642 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:47.057538 21349 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:47.094143 15642 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:47.102154 15675 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:20:47.103264 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:47.104355 15675 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:47.077085 21116 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:47.040773 21175 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:47.074652 21174 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:47.109948 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:47.056355 21341 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:20:47.141201 15675 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:20:47.067102 21217 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.039s
I20260709 13:20:47.072293 21249 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:20:47.079005 21348 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:47.144181 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:20:47.190228 15599 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53324 (request call id 2291) took 1347 ms (client timeout 1760 ms). Trace:
I20260709 13:20:47.175128 15668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2876) took 1387 ms. Trace:
I20260709 13:20:47.190896 15673 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:47.155431 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:47.147071 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1869) took 1593 ms. Trace:
W20260709 13:20:47.190863 15599 rpcz_store.cc:269] 0709 13:20:45.842727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.843121 (+   394us) service_pool.cc:224] Handling call
0709 13:20:47.190034 (+1346913us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b540043b554. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:47.190161 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":438761728}
I20260709 13:20:47.151842 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:47.105724 21317 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:47.105837 21307 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:47.192070 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:47.108177 21100 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:47.192865 15526 rpcz_store.cc:276] 0709 13:20:45.553546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.797346 (+243800us) service_pool.cc:224] Handling call
0709 13:20:45.797657 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:45.978976 (+181319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.146783 (+1167807us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54003d9ad4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:47.146999 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":415261440}
I20260709 13:20:47.198902 21201 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:47.199326 21335 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.158s
I20260709 13:20:47.144496 21174 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:47.204538 21326 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:47.205332 21313 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:47.186158 21235 consensus_queue.cc:260] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:47.202010 15566 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:47.209905 15673 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 17.
I20260709 13:20:47.191256 15668 rpcz_store.cc:276] 0709 13:20:45.787384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.788668 (+  1284us) service_pool.cc:224] Handling call
0709 13:20:45.827460 (+ 38792us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:45.966880 (+139420us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:20:46.004362 (+ 37482us) log.cc:844] Serialized 14 byte log entry
0709 13:20:46.262362 (+258000us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:46.262413 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:46.262456 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:46.635830 (+373374us) spinlock_profiling.cc:243] Waited 5.83 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:20:47.174864 (+539034us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b540021d754. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:47.174919 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:47.175057 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22052,"prepare.queue_time_us":43122,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":288007,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"spinlock_wait_cycles":333533312,"thread_start_us":235071,"threads_started":1,"wal-append.queue_time_us":379931}
I20260709 13:20:47.168957 21098 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:47.176530 21320 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:47.226343 15410 catalog_manager.cc:5028] TS 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): tablet bd36c2cfd99c45cea9e48096d54bed1a (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:20:47.275471 21313 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
W20260709 13:20:47.276310 15410 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 50169.5us
W20260709 13:20:47.236349 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:47.277433 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:47.106065 21318 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:47.242805 21175 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:47.246485 21301 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:47.279116 21100 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 22 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:47.091975 21316 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:47.253041 21104 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:47.280349 15600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 2293) took 1392 ms. Trace:
I20260709 13:20:47.280718 15600 rpcz_store.cc:276] 0709 13:20:45.888420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.888552 (+   132us) service_pool.cc:224] Handling call
0709 13:20:45.888859 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:47.020273 (+1131414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.280082 (+259809us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540016b054. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:47.280291 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2600522624}
I20260709 13:20:47.281529 21316 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:47.282186 16233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1670) took 1574 ms. Trace:
I20260709 13:20:47.282581 16233 rpcz_store.cc:276] 0709 13:20:45.707844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.708045 (+   201us) service_pool.cc:224] Handling call
0709 13:20:45.773655 (+ 65610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:46.968346 (+1194691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.281906 (+313560us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540068f454. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:47.282088 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2524859776}
W20260709 13:20:47.255697 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:47.283883 15642 connection.cc:256] RPC call timeout handler was delayed by -1.9710507629999938s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1511s for a response
W20260709 13:20:47.284236 15642 connection.cc:256] RPC call timeout handler was delayed by -1.7172507629998108s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17930000000000001s for a response
W20260709 13:20:47.284515 15642 connection.cc:256] RPC call timeout handler was delayed by -1.6686507629999596s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1847s for a response
W20260709 13:20:47.284790 15642 connection.cc:256] RPC call timeout handler was delayed by -1.621850762999884s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1899s for a response
W20260709 13:20:47.285072 15642 connection.cc:256] RPC call timeout handler was delayed by -1.5984507629998461s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1925s for a response
W20260709 13:20:47.285317 15642 connection.cc:256] RPC call timeout handler was delayed by -1.5606507630000124s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1967s for a response
W20260709 13:20:47.285543 15642 connection.cc:256] RPC call timeout handler was delayed by -1.5471507629999905s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19820000000000002s for a response
I20260709 13:20:47.195293 21349 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:47.286626 21194 raft_consensus.cc:2804] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:47.287302 21194 raft_consensus.cc:697] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:47.287519 21301 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:47.288096 21301 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:47.288671 16203 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:47.289469 21249 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:47.288115 21175 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.002s
I20260709 13:20:47.291374 21174 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:47.287303 21235 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:47.263870 21201 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:47.272334 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 70431.4us
I20260709 13:20:47.274559 21304 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:47.291988 15677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2885) took 1036 ms. Trace:
I20260709 13:20:47.286741 21349 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:47.299695 20975 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:47.300412 21349 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:47.300817 15525 raft_consensus.cc:1240] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:47.286621 21116 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:47.302716 21194 consensus_queue.cc:237] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:47.304028 21116 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:47.304793 15672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 1756) took 1067 ms. Trace:
I20260709 13:20:47.305600 21116 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:47.289875 16203 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:47.305955 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:47.307178 21191 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:47.307691 21191 raft_consensus.cc:697] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:47.305203 15672 rpcz_store.cc:276] 0709 13:20:46.237299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:46.237563 (+   264us) service_pool.cc:224] Handling call
0709 13:20:46.254424 (+ 16861us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:46.572960 (+318536us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:20:46.609391 (+ 36431us) log.cc:844] Serialized 14 byte log entry
0709 13:20:46.820444 (+211053us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:46.834225 (+ 13781us) log.cc:844] Serialized 26 byte log entry
0709 13:20:46.834396 (+   171us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:20:46.834429 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.304433 (+470004us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b540021dc54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:47.304479 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:47.304525 (+    46us) raft_consensus.cc:1685] finished
0709 13:20:47.304613 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:47.304738 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":36761,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":15840,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":24481,"spinlock_wait_cycles":677256576,"thread_start_us":201915,"threads_started":1,"wal-append.queue_time_us":348247}
I20260709 13:20:47.312935 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:47.275106 21326 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:47.319478 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:47.320431 15600 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:47.297111 15677 rpcz_store.cc:276] 0709 13:20:46.255228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:46.533554 (+278326us) service_pool.cc:224] Handling call
0709 13:20:46.548503 (+ 14949us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:46.694391 (+145888us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:20:46.709759 (+ 15368us) log.cc:844] Serialized 14 byte log entry
0709 13:20:46.944756 (+234997us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:46.944814 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:46.944863 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.291213 (+346350us) spinlock_profiling.cc:243] Waited 1.95 ms on lock 0x7b18003fe5c0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce18ec2a6 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419
0709 13:20:47.291773 (+   560us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b540046ee54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:47.291819 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:47.291933 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75,"prepare.queue_time_us":15197,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":186,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":7968,"spinlock_wait_cycles":339468800,"thread_start_us":226062,"threads_started":1,"wal-append.queue_time_us":255056}
W20260709 13:20:47.285804 15642 connection.cc:256] RPC call timeout handler was delayed by -1.4229507630000171s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21200000000000002s for a response
I20260709 13:20:47.321549 15642 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:47.296244 21201 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:47.328245 15566 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:47.277726 21313 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:20:47.306173 21249 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:47.338230 16203 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:47.342602 15599 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 22 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:20:47.306175 21116 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:47.308537 21191 consensus_queue.cc:237] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:47.351275 21116 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:47.352430 16203 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:47.353091 21326 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:47.353622 16203 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:47.354223 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:47.281563 21104 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:47.359831 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 289643us
I20260709 13:20:47.327152 21304 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:47.360875 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 18 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:47.361030 15493 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:47.337692 21014 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:47.342283 20955 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:47.364440 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:47.367304 15677 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:20:47.365314 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 43852us
I20260709 13:20:47.353578 21326 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:47.368429 15642 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:47.381755 15600 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:47.308558 21298 ts_tablet_manager.cc:1929] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 5.6
I20260709 13:20:47.386229 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:47.387972 15493 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:47.383805 16205 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:47.368451 21174 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:47.362913 21301 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:47.365612 21316 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:47.368355 21335 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:47.384610 21326 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:47.421113 15677 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 22.
I20260709 13:20:47.390298 21104 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:47.358366 21217 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.046s
I20260709 13:20:47.435582 15566 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:47.401131 21314 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:47.357738 21249 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:47.390820 21098 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:47.427598 15669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2887) took 1167 ms. Trace:
I20260709 13:20:47.390898 20955 raft_consensus.cc:697] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:47.434690 15642 leader_election.cc:400] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12.
I20260709 13:20:47.467942 15642 leader_election.cc:403] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:20:47.409554 21298 log.cc:1199] T 0af5649725244ae599dfc6ed7aeac5f1 P 3ef186ec12554a1d87814af0a0ace9a4: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-2-root/wals/0af5649725244ae599dfc6ed7aeac5f1
I20260709 13:20:47.438874 15669 rpcz_store.cc:276] 0709 13:20:46.260235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:46.863997 (+603762us) service_pool.cc:224] Handling call
0709 13:20:46.864320 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:47.016617 (+152297us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:20:47.016926 (+   309us) log.cc:844] Serialized 14 byte log entry
0709 13:20:47.387600 (+370674us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:47.387654 (+    54us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:20:47.387717 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.427332 (+ 39615us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b54002cd154. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:47.427393 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:47.427533 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18778,"prepare.queue_time_us":553,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":19353,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":13021,"spinlock_wait_cycles":349747456,"thread_start_us":351530,"threads_started":1}
I20260709 13:20:47.412794 21316 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:47.384701 21304 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:47.400902 21348 consensus_queue.cc:1059] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
W20260709 13:20:47.489282 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 450359us
W20260709 13:20:47.490759 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 77566us
W20260709 13:20:47.492168 15491 leader_election.cc:336] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.795s (SENT)
I20260709 13:20:47.492623 15642 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:47.493592 15491 leader_election.cc:336] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.823s (SENT)
I20260709 13:20:47.469870 21349 raft_consensus.cc:2804] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:47.468737 21313 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:20:47.501860 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:47.475281 21316 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:47.472424 21335 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:47.486732 21217 raft_consensus.cc:2804] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:47.482127 21318 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12.
I20260709 13:20:47.468127 21104 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:47.500131 21098 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:47.500049 21341 raft_consensus.cc:3055] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:20:47.517477 15602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 2285) took 1956 ms. Trace:
I20260709 13:20:47.498202 20975 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:47.533252 15600 raft_consensus.cc:1240] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:47.534307 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:47.590420 15568 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:47.534418 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:47.591481 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:47.534346 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 216681us
W20260709 13:20:47.592196 21361 log.cc:927] Time spent T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:20:47.592752 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:47.593374 15600 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:20:47.594175 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:47.535724 16240 raft_consensus.cc:1240] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:47.594588 15568 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:47.595140 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:47.596196 15670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2889) took 1335 ms. Trace:
I20260709 13:20:47.597096 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:47.537334 16239 raft_consensus.cc:1240] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:47.521534 21348 raft_consensus.cc:3055] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:47.574723 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:47.597901 15600 raft_consensus.cc:2410] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:20:47.535019 21298 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49088 (request call id 93) took 1833 ms. Trace:
W20260709 13:20:47.578423 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 76670.2us
I20260709 13:20:47.533857 21191 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:47.602739 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:47.603262 21104 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:47.604089 15642 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:47.576647 21349 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:47.591615 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:47.604967 21098 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:47.590158 15602 rpcz_store.cc:276] 0709 13:20:45.560911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.561000 (+    89us) service_pool.cc:224] Handling call
0709 13:20:46.998244 (+1437244us) spinlock_profiling.cc:243] Waited 1.03 ms on lock 0x7b5800113498. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99e4fc0 00007f5ce99cf929 00007f5ce99cf3d2 00007f5ce2f46bd4
0709 13:20:46.998441 (+   197us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:46.998638 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:20:47.254153 (+255515us) log.cc:844] Serialized 14 byte log entry
0709 13:20:47.517120 (+262967us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:47.517175 (+    55us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:47.517219 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.517284 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:47.517413 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":10207,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":2281,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":1668,"spinlock_wait_cycles":2379008,"thread_start_us":517677,"threads_started":2}
I20260709 13:20:47.598402 21189 consensus_queue.cc:1048] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.008s
I20260709 13:20:47.605507 21191 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:47.611590 21191 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:47.589881 21341 raft_consensus.cc:740] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:47.612807 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:47.599675 21348 raft_consensus.cc:740] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:47.596607 15670 rpcz_store.cc:276] 0709 13:20:46.261164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:46.987268 (+726104us) service_pool.cc:224] Handling call
0709 13:20:46.987538 (+   270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:46.991449 (+  3911us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:46.991480 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:47.038511 (+ 47031us) log.cc:844] Serialized 26 byte log entry
0709 13:20:47.319895 (+281384us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:47.319939 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.320008 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:47.595971 (+275963us) raft_consensus.cc:1685] finished
0709 13:20:47.595991 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:47.596138 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4273,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":81,"thread_start_us":245378,"threads_started":1,"wal-append.queue_time_us":452113}
I20260709 13:20:47.587100 16233 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:47.613114 21341 consensus_queue.cc:260] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:47.622625 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:47.626813 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 133247us
I20260709 13:20:47.592197 15596 raft_consensus.cc:2410] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 13:20:47.632547 15491 leader_election.cc:336] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 1.983s (SENT)
W20260709 13:20:47.637199 15491 leader_election.cc:336] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 2.033s (SENT)
I20260709 13:20:47.581678 21301 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:47.637707 15491 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:47.535154 20955 consensus_queue.cc:237] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:47.639837 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:47.587910 21313 raft_consensus.cc:697] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 22 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:47.591516 21103 raft_consensus.cc:2764] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 13:20:47.656095 15491 leader_election.cc:336] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 2.085s (SENT)
I20260709 13:20:47.630425 21349 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:47.662118 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:47.590472 21217 raft_consensus.cc:697] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:20:47.677580 15491 leader_election.cc:336] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 2.153s (SENT)
I20260709 13:20:47.643429 15642 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:47.636445 21098 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:47.585971 21335 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:47.705193 21348 consensus_queue.cc:260] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:47.771584 21304 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:47.699343 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:47.639384 21191 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:47.637295 21023 consensus_queue.cc:1059] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:20:47.668527 21341 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:20:47.834210 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 479431us
W20260709 13:20:47.834786 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 430081us
W20260709 13:20:47.858233 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 185192us
W20260709 13:20:47.832892 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 152157us
I20260709 13:20:47.652441 21307 consensus_queue.cc:1059] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:20:47.638999 21318 consensus_queue.cc:1059] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:20:47.665222 20975 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:20:47.816571 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 290452us
W20260709 13:20:47.880895 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 203474us
W20260709 13:20:47.881569 15491 leader_election.cc:336] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Timed out: RequestConsensusVote RPC to 127.14.23.67:32779 timed out after 2.221s (SENT)
I20260709 13:20:47.881680 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:47.862141 16203 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:47.670076 21249 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:47.636962 21339 consensus_queue.cc:1059] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:20:47.865617 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:47.883121 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:47.883751 15675 raft_consensus.cc:2368] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 16. Current term is 17.
I20260709 13:20:47.884371 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:47.884585 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:47.885556 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:47.866114 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:47.870999 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:47.890933 15673 raft_consensus.cc:2393] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 11: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:47.871500 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:47.870911 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:47.891608 15520 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:20:47.876652 15525 pending_rounds.cc:85] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 9
I20260709 13:20:47.892455 15525 pending_rounds.cc:107] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 8.10
I20260709 13:20:47.892728 15525 raft_consensus.cc:2889] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:47.877187 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:47.893399 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:47.866312 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:47.882014 15491 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:47.897465 15670 raft_consensus.cc:2368] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 18. Current term is 19.
W20260709 13:20:47.855028 21182 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.008s	sys 0.003s
I20260709 13:20:47.900087 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:47.905349 16239 raft_consensus.cc:2393] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 11: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
W20260709 13:20:47.906070 21368 log.cc:927] Time spent T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
W20260709 13:20:47.877966 21365 log.cc:927] Time spent T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
I20260709 13:20:47.917666 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:47.906345 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:47.906620 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:47.922266 15523 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:20:47.906934 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:47.907284 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:47.923655 15676 raft_consensus.cc:2410] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:20:47.907552 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:47.924398 15672 raft_consensus.cc:2376] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 16. Re-sending same reply.
I20260709 13:20:47.908034 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:47.908653 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:47.908886 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:47.844300 21301 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:47.911428 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:47.911713 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:47.911845 15669 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:20:47.912760 15520 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:47.905911 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:47.906572 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:47.929252 15671 raft_consensus.cc:2376] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13. Re-sending same reply.
I20260709 13:20:47.602113 21298 rpcz_store.cc:276] 0709 13:20:45.689979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.690940 (+   961us) service_pool.cc:224] Handling call
0709 13:20:46.099435 (+408495us) ts_tablet_manager.cc:1151] Deleting tablet 0af5649725244ae599dfc6ed7aeac5f1
0709 13:20:46.099490 (+    55us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:20:46.984405 (+884915us) tablet_metadata.cc:665] Metadata flushed
0709 13:20:47.308418 (+324013us) tablet_metadata.cc:665] Metadata flushed
0709 13:20:47.523220 (+214802us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":97609,"tablet-delete.queue_time_us":295532}
I20260709 13:20:47.921525 15528 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:20:47.930900 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:47.940650 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 59108.9us
W20260709 13:20:47.942209 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:47.942687 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:47.943009 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:47.944927 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 68698.9us
I20260709 13:20:47.950246 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:47.954008 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:47.955672 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:47.956434 15527 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 18.
W20260709 13:20:47.912475 21182 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.008s	sys 0.004s
I20260709 13:20:47.938565 21363 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:47.972723 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:47.973086 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:47.973246 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:47.973500 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:47.975968 15524 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:47.880998 21104 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:20:47.906096 21201 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:47.910794 21335 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:47.914441 21249 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:47.815886 21313 consensus_queue.cc:237] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:47.997275 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:47.997499 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:48.003890 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:48.005009 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:47.874855 21194 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.088s
I20260709 13:20:47.932013 21098 raft_consensus.cc:3055] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:47.932308 21301 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:47.931171 21014 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:47.939985 21320 raft_consensus.cc:3055] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:48.019135 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:47.949893 21348 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:48.047914 15520 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 19.
I20260709 13:20:47.879112 21191 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:48.050200 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:48.052289 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:20:47.987793 21329 log.cc:927] Time spent T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:20:48.049647 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:48.024391 15519 raft_consensus.cc:1275] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:20:48.061728 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:48.068384 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 166867us
W20260709 13:20:48.083976 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 63420.7us
I20260709 13:20:48.085608 15676 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 22 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:20:48.083976 15523 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
I20260709 13:20:48.087778 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:48.088016 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:48.008072 21378 raft_consensus.cc:2804] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:47.988495 21192 raft_consensus.cc:3055] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:48.089128 16234 raft_consensus.cc:2393] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 18: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
W20260709 13:20:48.110724 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:48.041093 21098 raft_consensus.cc:740] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:48.041308 21301 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:47.963640 21287 consensus_queue.cc:1059] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.113s
I20260709 13:20:48.039463 21314 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:48.101500 16233 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
I20260709 13:20:48.041805 21320 raft_consensus.cc:740] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:48.108667 16238 raft_consensus.cc:2410] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:20:48.111435 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:48.110670 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:48.113240 15597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 1593) took 1047 ms. Trace:
I20260709 13:20:48.114981 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:48.121004 15676 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:20:48.122154 15672 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
I20260709 13:20:48.120903 15597 rpcz_store.cc:276] 0709 13:20:47.065608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.065779 (+   171us) service_pool.cc:224] Handling call
0709 13:20:47.066181 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:47.066575 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:47.066608 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:47.078029 (+ 11421us) log.cc:844] Serialized 26 byte log entry
0709 13:20:47.436198 (+358169us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:20:47.436239 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.436310 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:48.112969 (+676659us) raft_consensus.cc:1685] finished
0709 13:20:48.112994 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:48.113152 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2644,"raft.queue_time_us":546,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":7734,"thread_start_us":293667,"threads_started":1,"wal-append.queue_time_us":417023}
I20260709 13:20:48.127964 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:48.130915 15523 raft_consensus.cc:1275] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:47.989614 21217 consensus_queue.cc:237] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:48.058301 21014 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:48.111804 21192 raft_consensus.cc:740] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:20:48.149492 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 306095us
I20260709 13:20:48.064446 21249 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:48.194094 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:20:48.221264 21235 consensus_meta.cc:287] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.929s	user 0.009s	sys 0.017s
W20260709 13:20:48.223161 21379 log.cc:927] Time spent T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
W20260709 13:20:48.224942 21373 log.cc:927] Time spent T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:20:48.129680 21314 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:48.235313 21098 consensus_queue.cc:260] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:48.236636 21098 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:48.155122 21116 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.084s
I20260709 13:20:48.179291 21194 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:48.239568 21194 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:48.241266 21320 consensus_queue.cc:260] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:48.179317 21318 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:48.242643 21318 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:48.243046 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 43695.9us
I20260709 13:20:48.144620 21369 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 13:20:48.246538 15522 raft_consensus.cc:2410] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
W20260709 13:20:48.301570 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 52632.8us
W20260709 13:20:48.352713 21396 log.cc:927] Time spent T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:20:48.356077 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:48.355978 21387 log.cc:927] Time spent T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.249s	user 0.000s	sys 0.001s
I20260709 13:20:48.393810 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:48.388208 21375 log.cc:927] Time spent T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.518s	user 0.000s	sys 0.001s
I20260709 13:20:48.407805 21378 raft_consensus.cc:697] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:48.245219 21192 consensus_queue.cc:260] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:48.495023 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 398809us
I20260709 13:20:48.487825 21235 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:48.498479 21235 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:48.500427 15595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 1598) took 1190 ms. Trace:
I20260709 13:20:48.500876 15595 rpcz_store.cc:276] 0709 13:20:47.310011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.311592 (+  1581us) service_pool.cc:224] Handling call
0709 13:20:47.335874 (+ 24282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:47.343717 (+  7843us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:47.343755 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:47.349868 (+  6113us) log.cc:844] Serialized 26 byte log entry
0709 13:20:47.550862 (+200994us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:47.550899 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.550960 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:48.500184 (+949224us) raft_consensus.cc:1685] finished
0709 13:20:48.500217 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:48.500358 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2686,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":11589,"thread_start_us":200610,"threads_started":1,"wal-append.queue_time_us":351440}
W20260709 13:20:48.504550 21391 log.cc:927] Time spent T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:20:48.504354 21399 log.cc:927] Time spent T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
W20260709 13:20:48.521960 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 168620us
W20260709 13:20:48.526628 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 282600us
W20260709 13:20:48.507783 21398 log.cc:927] Time spent T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 13:20:48.525815 16204 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:48.537606 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:48.538419 15520 raft_consensus.cc:2368] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 13. Current term is 14.
W20260709 13:20:48.560599 21395 log.cc:927] Time spent T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:20:48.543473 21363 consensus_queue.cc:1048] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:20:48.572983 21194 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:48.574002 21414 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:48.574522 21414 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:48.574944 21414 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:48.582978 15642 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.021s	sys 0.017s
W20260709 13:20:48.583896 15642 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.021s	sys 0.017s
W20260709 13:20:48.585278 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 44843.4us
W20260709 13:20:48.587697 15527 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.012s	sys 0.028s
W20260709 13:20:48.588426 15527 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.012s	sys 0.028s
I20260709 13:20:48.564136 21191 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:48.563651 21014 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:48.595011 15642 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:48.531958 21390 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.065s
I20260709 13:20:48.567137 21235 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:48.602327 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:48.544567 21104 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:20:48.601851 15595 raft_consensus.cc:1240] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:20:48.598920 15642 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:48.593271 15410 catalog_manager.cc:5028] TS 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): tablet 0af5649725244ae599dfc6ed7aeac5f1 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:20:48.639449 16235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1703) took 1007 ms. Trace:
W20260709 13:20:48.641044 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:48.644850 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:48.559641 21378 consensus_queue.cc:237] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:48.601406 21401 consensus_queue.cc:1059] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:20:48.637158 21314 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:48.637264 21014 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:48.694070 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:48.642730 16235 rpcz_store.cc:276] 0709 13:20:47.632160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.632425 (+   265us) service_pool.cc:224] Handling call
0709 13:20:47.651805 (+ 19380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:47.666372 (+ 14567us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:47.666406 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:47.666930 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 13:20:47.945323 (+278393us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:20:48.006331 (+ 61008us) log.cc:844] Serialized 14 byte log entry
0709 13:20:48.006549 (+   218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:48.006630 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:48.639223 (+632593us) raft_consensus.cc:1685] finished
0709 13:20:48.639257 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:48.639388 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14099,"prepare.queue_time_us":122833,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":49,"raft.queue_time_us":20938,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":34686,"thread_start_us":108459,"threads_started":1,"wal-append.queue_time_us":479873}
I20260709 13:20:48.598449 21406 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:20:48.700910 15519 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:20:48.637096 21235 consensus_queue.cc:1048] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
W20260709 13:20:48.702855 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 104034us
I20260709 13:20:48.652263 21419 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
W20260709 13:20:48.704661 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 163998us
I20260709 13:20:48.709080 15491 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:48.710830 21014 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:48.710340 15642 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:48.700963 21316 consensus_meta.cc:287] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 1.115s	user 0.014s	sys 0.013s
I20260709 13:20:48.678733 21174 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:20:48.709192 21326 consensus_meta.cc:287] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 1.242s	user 0.016s	sys 0.015s
I20260709 13:20:48.743492 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:48.637023 21389 consensus_queue.cc:1048] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 13:20:48.698839 16203 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:20:48.756474 16204 leader_election.cc:336] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.700s (SENT)
I20260709 13:20:48.756970 16204 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:48.755769 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 45504.4us
I20260709 13:20:48.748296 15491 leader_election.cc:400] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 election: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 16. Current term is 17.
W20260709 13:20:48.757265 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 58520.8us
I20260709 13:20:48.758410 15642 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:48.761423 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:48.763720 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:48.743943 21369 raft_consensus.cc:3055] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:48.770485 16203 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:48.750336 21014 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:48.778517 15491 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:48.781806 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:48.783888 16233 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:20:48.780313 15491 leader_election.cc:400] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 18. Current term is 19.
I20260709 13:20:48.781138 16203 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:48.785529 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:48.784552 15491 leader_election.cc:403] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:48.747936 21314 consensus_queue.cc:1048] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:20:48.788295 16235 raft_consensus.cc:1275] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:20:48.747807 21316 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:48.755504 21326 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:48.772948 21369 raft_consensus.cc:740] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:48.779031 21416 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:48.798225 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:48.754043 21174 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:48.799649 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:48.790385 16203 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:48.741597 21419 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:48.801170 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:48.791541 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 99914us
I20260709 13:20:48.804327 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:48.805182 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:48.811286 15598 raft_consensus.cc:2393] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 17: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:48.809442 15521 raft_consensus.cc:2393] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 13: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:48.808768 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:48.806757 15519 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:48.784287 21430 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:48.823513 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:48.824028 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 43893.1us
I20260709 13:20:48.825688 15491 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:48.789064 21235 consensus_queue.cc:1048] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:20:48.801412 21412 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:20:48.806934 21348 consensus_meta.cc:287] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.748s	user 0.016s	sys 0.009s
I20260709 13:20:48.804104 21389 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 18. Current term is 19.
I20260709 13:20:48.783902 21014 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:48.849676 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:48.805951 21426 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:48.855849 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:20:48.857041 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 46832us
I20260709 13:20:48.806190 21345 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:48.859025 21345 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:48.859479 21345 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:48.860081 21430 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:48.860747 21430 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:48.864310 21430 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:48.866503 15641 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:48.868867 21249 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:48.874531 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:48.874816 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:48.875671 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:48.876606 16232 raft_consensus.cc:2410] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 12 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 10.
I20260709 13:20:48.876910 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:48.824019 21406 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:20:48.877674 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 100190us
W20260709 13:20:48.880542 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 54919.4us
I20260709 13:20:48.875599 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:48.881803 15491 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:48.878504 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:48.864166 16235 raft_consensus.cc:1240] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:20:48.883693 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:48.881363 15641 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:48.884397 15669 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:48.888949 15669 raft_consensus.cc:1240] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
W20260709 13:20:48.878031 21417 log.cc:927] Time spent T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
W20260709 13:20:48.894429 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 136616us
I20260709 13:20:48.899521 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:48.895998 15641 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:48.897877 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:48.901095 15519 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:48.903432 16236 raft_consensus.cc:1240] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:20:48.907107 16236 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:48.907956 15567 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:48.910524 21109 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:48.911154 21339 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:48.912009 21109 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:48.914637 21380 log.cc:927] Time spent T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.184s	user 0.001s	sys 0.001s
I20260709 13:20:48.916318 15601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 1599) took 1606 ms. Trace:
I20260709 13:20:48.917546 21014 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:48.918349 21429 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:48.880864 21416 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:48.921267 15669 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:20:48.881287 21406 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:49.024724 21389 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:49.025033 21406 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:20:49.025876 21415 log.cc:927] Time spent T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
I20260709 13:20:48.916676 15601 rpcz_store.cc:276] 0709 13:20:47.310277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.311673 (+  1396us) service_pool.cc:224] Handling call
0709 13:20:47.312024 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:47.312421 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:20:47.343098 (+ 30677us) log.cc:844] Serialized 14 byte log entry
0709 13:20:47.549563 (+206465us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:47.550271 (+   708us) log.cc:844] Serialized 26 byte log entry
0709 13:20:47.550468 (+   197us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:47.550521 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.550607 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:48.916105 (+1365498us) raft_consensus.cc:1685] finished
0709 13:20:48.916129 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:48.916254 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":30487,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":2664,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":13631,"thread_start_us":159144,"threads_started":1,"wal-append.queue_time_us":1168733}
I20260709 13:20:48.930696 15491 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:48.990059 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:48.896649 15493 connection.cc:256] RPC call timeout handler was delayed by -1.8133129029999964s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17600000000000002s for a response
I20260709 13:20:48.990806 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:48.997006 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:49.002933 21014 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:20:49.016928 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 109049us
I20260709 13:20:49.017195 21450 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:20:49.024369 21433 log.cc:927] Time spent T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:20:49.024920 21077 raft_consensus.cc:3055] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:49.028363 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 18 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:49.002377 21442 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:49.030834 15567 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:48.899924 21426 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:20:49.027767 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 97171.1us
I20260709 13:20:49.045853 16203 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:49.030686 15520 raft_consensus.cc:2410] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:20:49.056535 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 2385) took 1091 ms. Trace:
I20260709 13:20:49.056942 15527 rpcz_store.cc:276] 0709 13:20:47.964949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.991622 (+ 26673us) service_pool.cc:224] Handling call
0709 13:20:48.046214 (+ 54592us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:48.046574 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:48.046601 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:48.047121 (+   520us) log.cc:844] Serialized 26 byte log entry
0709 13:20:48.590174 (+543053us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:48.590224 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:48.590293 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:49.056334 (+466041us) raft_consensus.cc:1685] finished
0709 13:20:49.056353 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:49.056477 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1056,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":12882,"thread_start_us":542551,"threads_started":1,"wal-append.queue_time_us":655232}
I20260709 13:20:49.042133 21348 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:49.033739 21104 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:49.033392 21434 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:49.048050 21077 raft_consensus.cc:740] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:49.064913 16204 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:49.050772 21442 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:48.877058 21369 consensus_queue.cc:260] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.073191 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:49.070578 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 217666us
I20260709 13:20:49.089428 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:49.058897 21426 raft_consensus.cc:697] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
W20260709 13:20:49.094798 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:49.092163 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:49.088985 21430 consensus_queue.cc:1059] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 13:20:49.097900 21439 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:49.098158 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:49.098457 21413 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:49.098870 21413 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:49.099201 21413 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:49.099392 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:49.100500 21301 consensus_meta.cc:287] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.969s	user 0.011s	sys 0.015s
I20260709 13:20:49.100997 21301 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:49.102016 21425 log.cc:927] Time spent T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
I20260709 13:20:49.099965 15526 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 6.
W20260709 13:20:49.102612 21423 log.cc:927] Time spent T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.287s	user 0.001s	sys 0.000s
I20260709 13:20:49.065487 21235 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:49.072021 21442 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:49.103852 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:49.054919 21450 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:49.068940 21014 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:49.033075 21416 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:49.097649 21424 log.cc:927] Time spent T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.286s	user 0.000s	sys 0.001s
W20260709 13:20:49.089272 21386 log.cc:927] Time spent T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.963s	user 0.003s	sys 0.000s
W20260709 13:20:49.092705 21098 consensus_meta.cc:287] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.854s	user 0.010s	sys 0.014s
W20260709 13:20:49.115996 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 216531us
I20260709 13:20:49.117683 16239 raft_consensus.cc:1240] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:49.102442 21077 consensus_queue.cc:260] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:49.119834 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 223921us
I20260709 13:20:49.140153 15641 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:49.104288 21175 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:49.138182 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:49.139462 15673 raft_consensus.cc:2410] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
W20260709 13:20:49.137588 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 108627us
W20260709 13:20:49.141708 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 76897.9us
I20260709 13:20:49.146540 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:49.115022 21369 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:49.141527 15641 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:49.117030 21450 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:49.116736 21431 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:49.116772 21014 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:49.115217 21235 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.162388 21436 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:49.162915 21453 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:49.163964 21437 log.cc:927] Time spent T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:20:49.164829 21432 log.cc:927] Time spent T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:20:49.070676 21434 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.192019 21434 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:49.192775 21434 consensus_queue.cc:1048] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:49.165683 21452 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:49.193742 21452 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:49.194188 21452 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:49.153519 21326 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:49.166857 15668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2899) took 1820 ms. Trace:
I20260709 13:20:49.167387 21077 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:49.168048 21416 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:49.151949 21217 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
W20260709 13:20:49.173229 21341 consensus_meta.cc:287] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 1.280s	user 0.008s	sys 0.019s
I20260709 13:20:49.153976 21439 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:49.136865 21419 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:49.160688 16204 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:49.188663 21446 log.cc:927] Time spent T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:20:49.188241 21453 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:49.198699 15668 rpcz_store.cc:276] 0709 13:20:47.346152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.346971 (+   819us) service_pool.cc:224] Handling call
0709 13:20:47.360548 (+ 13577us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:47.387517 (+ 26969us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:47.387549 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:47.472311 (+ 84762us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b2c0006cad8. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce3304a11 00007f5ce3314372 00007f5ce4a46a01 00007f5ce4a0b43a 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2
0709 13:20:47.489747 (+ 17436us) log.cc:844] Serialized 26 byte log entry
0709 13:20:47.835498 (+345751us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:49.026353 (+1190855us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:20:49.040258 (+ 13905us) spinlock_profiling.cc:243] Waited 1.06 ms on lock 0x7b18002b3180. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce45b34bc 00007f5ce190148b 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191b455 00007f5ce84033c7 00007f5ce4a159a7 00007f5ce4a250bf
0709 13:20:49.040459 (+   201us) log.cc:844] Serialized 14 byte log entry
0709 13:20:49.054753 (+ 14294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:49.054834 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:49.166592 (+111758us) raft_consensus.cc:1685] finished
0709 13:20:49.166619 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:49.166784 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19178,"prepare.queue_time_us":552887,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":108671,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":15519,"spinlock_wait_cycles":43071360,"thread_start_us":531544,"threads_started":3,"wal-append.queue_time_us":530675}
I20260709 13:20:49.207777 16231 raft_consensus.cc:2410] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
W20260709 13:20:49.227494 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 66886.1us
I20260709 13:20:49.233623 16204 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:49.200902 21349 consensus_meta.cc:287] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 1.363s	user 0.008s	sys 0.020s
W20260709 13:20:49.203076 21405 log.cc:927] Time spent T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.608s	user 0.005s	sys 0.004s
I20260709 13:20:49.254745 15526 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:49.183924 21014 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:49.207363 21439 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:49.257262 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 110790us
W20260709 13:20:49.258955 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:49.248454 21341 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:20:49.197904 21414 consensus_meta.cc:287] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.622s	user 0.008s	sys 0.004s
W20260709 13:20:49.267407 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 125946us
I20260709 13:20:49.201406 21104 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:49.204921 21416 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:49.270368 15493 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:49.275926 15641 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:49.256861 21217 raft_consensus.cc:2804] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:49.271693 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:49.277518 15668 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:49.275542 21104 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:49.279129 15493 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:49.182927 21406 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.260145 21429 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:49.284332 21362 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:49.303853 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:49.271617 21439 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:49.275676 21416 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:49.262024 21349 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.282125 21217 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:49.303610 21456 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:49.316545 21362 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:49.316130 21434 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:49.316305 21301 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:49.383757 15524 raft_consensus.cc:2393] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 10: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:49.325980 21416 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:49.385412 15525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2398) took 1518 ms. Trace:
I20260709 13:20:49.282550 21414 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.205076 21426 consensus_queue.cc:237] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:49.390401 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 111356us
I20260709 13:20:49.336807 21217 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:49.370215 21454 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:49.393656 15525 rpcz_store.cc:276] 0709 13:20:47.866953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.867113 (+   160us) service_pool.cc:224] Handling call
0709 13:20:47.876434 (+  9321us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:47.918757 (+ 42323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:47.918792 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:47.919412 (+   620us) log.cc:844] Serialized 52 byte log entry
0709 13:20:48.443959 (+524547us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:48.891648 (+447689us) log.cc:844] Serialized 14 byte log entry
0709 13:20:48.931489 (+ 39841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:48.931577 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:49.202372 (+270795us) spinlock_profiling.cc:243] Waited 2.23 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:20:49.372720 (+170348us) raft_consensus.cc:1685] finished
0709 13:20:49.385119 (+ 12399us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:49.385338 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50257,"prepare.queue_time_us":531226,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":48,"raft.queue_time_us":52748,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":122730,"spinlock_wait_cycles":5120384,"thread_start_us":856922,"threads_started":2,"wal-append.queue_time_us":541649}
I20260709 13:20:49.416589 15527 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:20:49.417040 15527 pending_rounds.cc:85] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 9
I20260709 13:20:49.417246 15527 pending_rounds.cc:107] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:20:49.417441 15527 raft_consensus.cc:2889] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:49.415668 15493 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:49.389051 21362 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:49.311908 21429 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.391569 21459 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:49.333525 21413 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.326776 21345 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.373075 21456 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.415647 21434 raft_consensus.cc:697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:49.450146 21349 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:49.450946 21468 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:20:49.453810 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 303022us
I20260709 13:20:49.455509 21301 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:49.456058 21301 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.459313 16240 raft_consensus.cc:1240] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:49.470815 16235 raft_consensus.cc:1240] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9. Ops: []
W20260709 13:20:49.451064 21458 log.cc:927] Time spent T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
W20260709 13:20:49.466781 21367 log.cc:927] Time spent T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 1.613s	user 0.003s	sys 0.013s
I20260709 13:20:49.458724 21434 consensus_queue.cc:237] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.476352 21429 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:49.476920 21372 log.cc:927] Time spent T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 1.855s	user 0.010s	sys 0.007s
I20260709 13:20:49.477591 21453 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:49.462206 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:49.489475 15601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:59198
I20260709 13:20:49.490936 16236 raft_consensus.cc:1240] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:49.493621 15601 raft_consensus.cc:606] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:49.501865 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:49.502615 15675 raft_consensus.cc:2376] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14. Re-sending same reply.
I20260709 13:20:49.503746 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:49.507994 15675 raft_consensus.cc:1240] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:20:49.515506 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:49.509949 21189 consensus_queue.cc:1059] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:20:49.517405 21460 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:49.518080 21460 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:49.520076 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:49.520294 21460 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:49.520870 21460 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:49.521330 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:49.515578 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:49.522162 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:49.522756 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:49.523381 15675 raft_consensus.cc:1240] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:20:49.492864 21014 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:49.495850 21301 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:49.534011 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:49.534868 16236 raft_consensus.cc:2393] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 13: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:49.479288 21450 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.532533 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:49.531484 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:49.511725 21453 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:49.544431 15673 raft_consensus.cc:1240] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:20:49.544293 15668 raft_consensus.cc:1240] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:20:49.545560 15601 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:49.552042 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:49.522110 21406 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:49.538784 21419 raft_consensus.cc:3055] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:20:49.559875 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 128772us
I20260709 13:20:49.557107 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:49.562203 21217 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:49.562296 21463 log.cc:927] Time spent T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:20:49.563952 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:49.566323 15669 raft_consensus.cc:1240] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:49.568392 15493 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:49.572305 15493 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:49.570621 21348 consensus_queue.cc:1059] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:20:49.573561 21389 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:49.581909 21318 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:49.588335 21174 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:49.591220 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 2390) took 1442 ms. Trace:
I20260709 13:20:49.591632 15522 rpcz_store.cc:276] 0709 13:20:48.148750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:48.148983 (+   233us) service_pool.cc:224] Handling call
0709 13:20:49.591014 (+1442031us) spinlock_profiling.cc:243] Waited 51.3 ms on lock 0x7b5400262854. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:49.591163 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118073088}
I20260709 13:20:49.592792 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:49.594870 15520 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 13:20:49.595237 15520 pending_rounds.cc:85] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 9
I20260709 13:20:49.595441 15520 pending_rounds.cc:107] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:20:49.595650 15520 raft_consensus.cc:2889] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:49.521350 21460 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.523488 21442 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:49.600263 21318 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:49.565279 21419 raft_consensus.cc:740] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:49.621059 21419 consensus_queue.cc:260] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.597590 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:49.601872 16232 raft_consensus.cc:1240] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:49.622865 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:49.623384 21217 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:49.623581 15520 raft_consensus.cc:2368] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 13. Current term is 14.
I20260709 13:20:49.624367 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:49.625164 15520 raft_consensus.cc:2410] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:20:49.593859 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:49.570948 21431 consensus_queue.cc:1059] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:20:49.605886 15493 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:49.606264 21345 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:49.608759 21442 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:49.610152 21318 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.613618 21450 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:49.615547 21413 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:49.618558 21456 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:49.628415 15493 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:49.626856 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:49.625890 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:49.635435 21431 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:49.613977 21451 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:49.597102 21465 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:49.688660 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:49.605198 21414 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:49.691143 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:49.687119 21461 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:49.626892 21454 raft_consensus.cc:3055] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:49.628135 21460 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:49.570148 21426 consensus_queue.cc:1048] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:20:49.598266 21470 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:49.692327 15522 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:49.691318 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:49.634662 21349 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:49.685695 15601 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
W20260709 13:20:49.723084 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 94719.3us
I20260709 13:20:49.691131 21419 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:49.726735 15491 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:49.728157 15493 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:49.752012 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:49.701404 21454 raft_consensus.cc:740] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:49.756202 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:49.778483 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 50480.5us
I20260709 13:20:49.626263 21452 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.832850 15520 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:49.833400 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:49.729758 21431 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:49.836747 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:49.693022 21362 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.687326 21451 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:49.844233 15676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57710 (request call id 1775) took 1751 ms. Trace:
I20260709 13:20:49.847992 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:49.728917 21461 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.723214 21470 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:49.832481 21348 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:49.729333 21349 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.885279 15527 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:20:49.847765 15676 rpcz_store.cc:276] 0709 13:20:48.092841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:48.192130 (+ 99289us) service_pool.cc:224] Handling call
0709 13:20:48.244893 (+ 52763us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:48.245134 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:20:48.528593 (+283459us) log.cc:844] Serialized 14 byte log entry
0709 13:20:49.479302 (+950709us) spinlock_profiling.cc:243] Waited 1.24 ms on lock 0x7b18003f3220. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab23a 00007f5ce4a15a2a
0709 13:20:49.843884 (+364582us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:49.843940 (+    56us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:49.843984 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:49.844049 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:49.844170 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":261020,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":8718,"raft.queue_time_us":10345,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"spinlock_wait_cycles":2848512,"thread_start_us":514392,"threads_started":3,"wal-append.queue_time_us":1176457}
W20260709 13:20:49.858161 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:49.856117 21390 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:49.888338 21390 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.889679 21453 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:49.890651 21390 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:49.896742 21362 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:49.905140 21416 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:49.861302 21473 consensus_queue.cc:1059] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:49.836616 21474 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:49.922902 21474 raft_consensus.cc:3055] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:49.923301 21474 raft_consensus.cc:740] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:49.924116 21474 consensus_queue.cc:260] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.838290 21481 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:49.925197 21481 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.927165 15522 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18.
I20260709 13:20:49.862280 21431 consensus_queue.cc:1059] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:49.865185 15641 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:49.928025 21431 raft_consensus.cc:3055] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:49.928241 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:49.928404 21431 raft_consensus.cc:740] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
W20260709 13:20:49.928412 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 63305.8us
I20260709 13:20:49.929848 21474 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:49.872702 15493 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:49.856904 21014 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:49.874369 21235 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:49.939993 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 67392.5us
W20260709 13:20:49.875209 21369 consensus_meta.cc:287] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.721s	user 0.009s	sys 0.000s
I20260709 13:20:49.879288 21452 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:49.876524 21104 consensus_meta.cc:287] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.571s	user 0.009s	sys 0.000s
I20260709 13:20:49.941172 16237 raft_consensus.cc:2410] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 14 because replica has last-logged OpId of term: 13 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 13:20:49.882622 21461 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:49.883832 16236 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:20:49.942622 16236 raft_consensus.cc:606] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:49.942981 16236 raft_consensus.cc:614] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:49.884326 21470 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:49.943940 21470 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:49.944424 21470 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.909696 21349 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:49.947888 21470 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:49.952747 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 66945.6us
W20260709 13:20:49.914804 21439 consensus_meta.cc:287] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.581s	user 0.009s	sys 0.000s
I20260709 13:20:49.921809 15670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2921) took 1087 ms. Trace:
I20260709 13:20:49.927364 21481 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:49.863862 21318 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:49.848273 21460 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:49.960013 21439 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.962414 21460 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.964313 21439 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:49.964689 21460 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:49.930310 15641 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:49.871292 21316 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:49.930882 21442 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:49.937121 16239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57258 (request call id 1237) took 1062 ms. Trace:
I20260709 13:20:49.938360 21431 consensus_queue.cc:260] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:49.940008 21235 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:49.880952 21451 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:49.941020 21104 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:49.958202 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:49.960016 15670 rpcz_store.cc:276] 0709 13:20:48.834571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:48.858431 (+ 23860us) service_pool.cc:224] Handling call
0709 13:20:48.858720 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:48.889454 (+ 30734us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:48.889482 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:48.889527 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:48.889557 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:49.921497 (+1031940us) spinlock_profiling.cc:243] Waited 30.5 ms on lock 0x7b540021b454. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:49.921560 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:49.921732 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":70241152}
I20260709 13:20:49.924779 21416 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:49.859972 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 133305us
I20260709 13:20:49.967743 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:50.006639 15523 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
W20260709 13:20:49.969442 21376 log.cc:927] Time spent T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 2.005s	user 0.002s	sys 0.002s
I20260709 13:20:49.970475 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56840 (request call id 1888) took 2002 ms. Trace:
I20260709 13:20:49.970664 15528 raft_consensus.cc:1240] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 12. Current term is 13. Ops: [12.12-12.12]
W20260709 13:20:49.959107 21475 log.cc:927] Time spent T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.354s	user 0.000s	sys 0.010s
I20260709 13:20:50.013526 15521 rpcz_store.cc:276] 0709 13:20:47.967993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.996742 (+ 28749us) service_pool.cc:224] Handling call
0709 13:20:49.970230 (+1973488us) spinlock_profiling.cc:243] Waited 750 ms on lock 0x7b5400023354. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:49.970403 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1725371776}
I20260709 13:20:50.017347 15641 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:49.965898 16239 rpcz_store.cc:276] 0709 13:20:48.874716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:48.876726 (+  2010us) service_pool.cc:224] Handling call
0709 13:20:48.877010 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:49.141073 (+264063us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:49.936856 (+795783us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b54006937d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:49.937053 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":547322752}
I20260709 13:20:50.032189 15491 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:50.014029 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:50.035694 15522 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:20:50.038074 15641 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:50.046831 15491 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:50.054594 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:49.974933 21318 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:20:50.076092 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 320475us
I20260709 13:20:50.078493 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:50.003634 21442 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:50.052595 21431 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:50.052863 21451 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:50.061061 21426 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:50.060277 21416 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:50.027547 21412 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:20:50.083038 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:50.062742 21481 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:50.084079 21451 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:50.087284 21481 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:50.110040 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:50.012153 16204 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:50.086933 21416 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:50.108678 21481 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:50.106175 21451 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:20:50.127270 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 80528us
I20260709 13:20:50.132296 15522 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
W20260709 13:20:50.140035 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 61602.3us
I20260709 13:20:50.153898 15669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 2923) took 1049 ms. Trace:
I20260709 13:20:50.154461 15669 rpcz_store.cc:276] 0709 13:20:49.104102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:49.104311 (+   209us) service_pool.cc:224] Handling call
0709 13:20:49.104575 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:49.570002 (+465427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:50.153590 (+583588us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b540009d3d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:50.153819 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1061512448}
W20260709 13:20:50.162128 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 150044us
I20260709 13:20:50.164470 16204 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:50.199275 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 50389.3us
W20260709 13:20:50.212105 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 47744.7us
W20260709 13:20:50.203897 16238 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 1190) took 2273 ms (client timeout 2109 ms). Trace:
I20260709 13:20:50.216490 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:50.215240 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:50.082667 21454 consensus_queue.cc:260] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:50.241091 15670 raft_consensus.cc:1240] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:20:50.231837 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:50.242202 15675 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:20:50.213828 16238 rpcz_store.cc:269] 0709 13:20:47.881881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.882483 (+   602us) service_pool.cc:224] Handling call
0709 13:20:50.155210 (+2272727us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b5400621e54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:50.155380 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":379615616}
I20260709 13:20:50.198462 21482 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:50.237522 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:50.251314 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:50.242283 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:50.224171 21116 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:50.257768 15669 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:50.243513 21363 consensus_queue.cc:1059] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:20:50.275285 21363 raft_consensus.cc:3055] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:50.275691 21363 raft_consensus.cc:740] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:50.162103 21426 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:50.212203 21316 consensus_queue.cc:1059] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:20:50.288653 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:50.267181 21362 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:50.249653 21349 consensus_queue.cc:1059] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:20:50.257114 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:50.262187 21348 consensus_queue.cc:1059] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:50.262573 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:50.300115 15677 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:50.263342 21473 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:50.264312 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:50.212105 21369 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:50.211761 21470 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:50.250425 15521 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 15.
I20260709 13:20:50.281116 15491 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:50.291913 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:20:50.295197 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:50.259865 16239 raft_consensus.cc:3055] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:20:50.297112 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:50.299021 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:50.278687 21318 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:50.293099 21452 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:50.331589 15491 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:50.333244 16238 raft_consensus.cc:2410] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:20:50.330924 16239 raft_consensus.cc:740] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:50.320250 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:50.322665 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1900) took 1256 ms. Trace:
I20260709 13:20:50.276467 21363 consensus_queue.cc:260] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:50.298383 21230 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:50.334609 16239 consensus_queue.cc:260] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:50.334481 15523 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:20:50.340930 16239 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:50.335429 15520 rpcz_store.cc:276] 0709 13:20:49.066442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:49.853977 (+787535us) service_pool.cc:224] Handling call
0709 13:20:49.854258 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:49.854438 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:20:49.863318 (+  8880us) log.cc:844] Serialized 14 byte log entry
0709 13:20:50.322156 (+458838us) log.cc:844] Serialized 14 byte log entry
0709 13:20:50.322321 (+   165us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:50.322366 (+    45us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:20:50.322410 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:50.322479 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:50.322603 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17693,"prepare.queue_time_us":8907,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":27633,"thread_start_us":440504,"threads_started":1}
I20260709 13:20:50.341334 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:50.342108 15523 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:50.332222 21369 raft_consensus.cc:697] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:50.302223 21453 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:50.295980 21362 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:50.347584 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:50.349225 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:50.349516 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:50.356180 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:50.349779 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:50.350083 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:50.350841 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:50.345105 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:50.354085 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:50.357800 15598 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:20:50.354063 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:50.336302 21461 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:50.364152 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:50.336979 21439 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:50.345286 21314 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:50.375758 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:50.368593 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:50.376415 15521 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:50.342696 21182 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:50.378834 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:50.319931 21470 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:50.379953 15523 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:50.382964 15595 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18.
I20260709 13:20:50.354332 21436 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:50.362243 21363 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:50.362829 21461 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:50.385844 15597 raft_consensus.cc:1240] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:50.380203 21439 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:50.392686 15520 raft_consensus.cc:1240] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:20:50.393785 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1901) took 1327 ms. Trace:
I20260709 13:20:50.394141 15520 rpcz_store.cc:276] 0709 13:20:49.066706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:50.392132 (+1325426us) service_pool.cc:224] Handling call
0709 13:20:50.392466 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:50.393519 (+  1053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:50.393730 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:50.338960 21452 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:50.338424 21318 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:50.396121 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:50.400496 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:50.352857 21230 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:50.405272 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:50.380445 21314 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:50.413276 16205 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:50.395088 21461 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:50.418577 21439 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:50.419265 21116 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:50.419730 21116 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:50.420127 21116 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:50.420644 16203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.346s	user 0.034s	sys 0.014s
I20260709 13:20:50.394217 21416 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:50.416906 16205 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:50.426306 16205 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:50.428320 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:50.423432 16239 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 13:20:50.424391 15596 raft_consensus.cc:1275] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:20:50.429870 16239 pending_rounds.cc:85] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9: Aborting all ops after (but not including) 14
I20260709 13:20:50.430277 16239 pending_rounds.cc:107] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 13:20:50.430593 16239 raft_consensus.cc:2889] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:50.430622 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:50.432994 16203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.359s	user 0.035s	sys 0.014s
I20260709 13:20:50.414865 21314 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:50.437855 15523 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
W20260709 13:20:50.438504 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 1.36862e+06us
I20260709 13:20:50.452071 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:50.387414 21182 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:50.454380 15597 raft_consensus.cc:2393] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 12: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:50.394642 21436 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:50.457209 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:50.426667 21334 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:50.458598 15597 raft_consensus.cc:1240] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 13. Current term is 14. Ops: [12.7-13.8]
I20260709 13:20:50.436587 21427 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:50.410949 21369 consensus_queue.cc:237] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:50.471100 21452 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:50.516600 21416 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:50.517812 21416 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:50.520092 21416 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:50.516439 21494 log.cc:927] Time spent T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.183s	user 0.001s	sys 0.001s
I20260709 13:20:50.468524 21230 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:50.461030 21413 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:20:50.523550 21451 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:50.609695 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:50.610796 15525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1898) took 1547 ms. Trace:
I20260709 13:20:50.611253 15525 rpcz_store.cc:276] 0709 13:20:49.063132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:49.420180 (+357048us) service_pool.cc:224] Handling call
0709 13:20:49.422197 (+  2017us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:49.422414 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:20:49.434367 (+ 11953us) log.cc:844] Serialized 14 byte log entry
0709 13:20:50.273537 (+839170us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b18001f14a0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab23a 00007f5ce4a15a2a
0709 13:20:50.610215 (+336678us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:50.610460 (+   245us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:20:50.610510 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:50.610586 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:50.610729 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":118,"prepare.queue_time_us":11793,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":2071,"raft.queue_time_us":4045,"raft.run_cpu_time_us":62,"raft.run_wall_time_us":62,"spinlock_wait_cycles":6374784,"thread_start_us":336113,"threads_started":2,"wal-append.queue_time_us":1096500}
I20260709 13:20:50.613242 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:50.614043 15675 raft_consensus.cc:2393] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 14: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:50.613873 15672 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 15.
W20260709 13:20:50.618461 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 144310us
I20260709 13:20:50.523504 21481 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:50.626612 15520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1902) took 1556 ms. Trace:
I20260709 13:20:50.615520 21363 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:50.626720 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 170540us
I20260709 13:20:50.615718 21362 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:50.628328 21235 consensus_meta.cc:287] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.620s	user 0.011s	sys 0.000s
W20260709 13:20:50.658306 15524 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56840 (request call id 1889) took 2690 ms (client timeout 1928 ms). Trace:
I20260709 13:20:50.638751 21389 raft_consensus.cc:3055] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:20:50.666266 15524 rpcz_store.cc:269] 0709 13:20:47.968231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.976752 (+  8521us) service_pool.cc:224] Handling call
0709 13:20:50.658071 (+2681319us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b5400261e54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:50.658203 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3066433408}
W20260709 13:20:50.647743 21487 log.cc:927] Time spent T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.344s	user 0.000s	sys 0.002s
I20260709 13:20:50.650967 21454 raft_consensus.cc:3060] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:50.654223 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 2409) took 1859 ms. Trace:
I20260709 13:20:50.655570 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1899) took 1592 ms. Trace:
I20260709 13:20:50.678542 15527 rpcz_store.cc:276] 0709 13:20:49.066145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:49.419744 (+353599us) service_pool.cc:224] Handling call
0709 13:20:49.431983 (+ 12239us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:49.909027 (+477044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:50.655328 (+746301us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b54003cf5d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:50.655510 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1041952512}
I20260709 13:20:50.656998 21461 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:50.627666 21304 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:50.659960 21192 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:50.658344 15673 raft_consensus.cc:2393] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 13: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:50.675336 21436 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:50.664038 21470 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:50.622188 21474 consensus_queue.cc:1059] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:20:50.678298 15519 rpcz_store.cc:276] 0709 13:20:48.794679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:49.138984 (+344305us) service_pool.cc:224] Handling call
0709 13:20:49.160477 (+ 21493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:49.209662 (+ 49185us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:20:49.256123 (+ 46461us) log.cc:844] Serialized 14 byte log entry
0709 13:20:49.387733 (+131610us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:49.387786 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:49.387831 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:50.654000 (+1266169us) spinlock_profiling.cc:243] Waited 48.9 ms on lock 0x7b5400260f54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:50.654050 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:50.654164 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":47061,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":142503,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":22530,"spinlock_wait_cycles":112516352,"thread_start_us":131060,"threads_started":2,"wal-append.queue_time_us":155017,"wal-append.run_cpu_time_us":1311,"wal-append.run_wall_time_us":1063927}
I20260709 13:20:50.664093 15521 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
W20260709 13:20:50.683871 21512 log.cc:927] Time spent T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.002s
I20260709 13:20:50.670564 21235 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:20:50.683629 21489 log.cc:927] Time spent T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.247s	user 0.000s	sys 0.002s
I20260709 13:20:50.710968 16235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57258 (request call id 1239) took 1821 ms. Trace:
I20260709 13:20:50.713827 16235 rpcz_store.cc:276] 0709 13:20:48.889647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:48.896392 (+  6745us) service_pool.cc:224] Handling call
0709 13:20:48.910248 (+ 13856us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:49.480472 (+570224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:50.710707 (+1230235us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b54008a02d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:50.710903 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1285357824}
I20260709 13:20:50.670065 21452 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:50.712703 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1911) took 1158 ms. Trace:
W20260709 13:20:50.688242 21462 log.cc:927] Time spent T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 1.300s	user 0.013s	sys 0.000s
W20260709 13:20:50.680398 21493 log.cc:927] Time spent T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.200s	user 0.000s	sys 0.002s
I20260709 13:20:50.715298 15521 rpcz_store.cc:276] 0709 13:20:49.554638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:50.711888 (+1157250us) service_pool.cc:224] Handling call
0709 13:20:50.712199 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:50.712395 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:50.712419 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:50.712460 (+    41us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:20:50.712490 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:50.712543 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:50.712650 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:50.717173 21335 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 13:20:50.717360 21316 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:50.671195 21389 raft_consensus.cc:740] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
W20260709 13:20:50.681694 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 183932us
I20260709 13:20:50.681427 21318 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:50.635289 21363 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:50.689623 21451 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:50.672159 15493 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:50.728624 21316 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:50.741515 16237 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57258 (request call id 1236) took 1857 ms (client timeout 2042 ms). Trace:
W20260709 13:20:50.742295 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 70224.3us
I20260709 13:20:50.697185 21304 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:50.736394 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:50.737134 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:50.743570 15672 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 15.
I20260709 13:20:50.738204 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:50.739328 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:50.744875 15674 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:50.745105 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:50.743677 15671 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:20:50.745875 16239 raft_consensus.cc:2393] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 13: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
W20260709 13:20:50.741968 16237 rpcz_store.cc:269] 0709 13:20:48.874432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:48.874563 (+   131us) service_pool.cc:224] Handling call
0709 13:20:50.731193 (+1856630us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54007b0f54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:50.731365 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2450664192}
I20260709 13:20:50.737910 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:50.748106 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:50.749246 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:50.751206 15672 raft_consensus.cc:1275] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:20:50.749869 15668 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 22.
I20260709 13:20:50.756361 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:50.757164 16239 raft_consensus.cc:2393] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 13: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:50.758574 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:50.764209 15674 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:20:50.747995 21335 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:50.746625 21235 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:50.725621 21452 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:50.766250 21431 consensus_meta.cc:287] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.681s	user 0.010s	sys 0.001s
I20260709 13:20:50.786032 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:50.774340 21182 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:50.785054 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:50.798017 15525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1903) took 1720 ms. Trace:
I20260709 13:20:50.798552 15525 rpcz_store.cc:276] 0709 13:20:49.077535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:50.613480 (+1535945us) service_pool.cc:224] Handling call
0709 13:20:50.613885 (+   405us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:50.614573 (+   688us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:50.614600 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:50.615185 (+   585us) log.cc:844] Serialized 52 byte log entry
0709 13:20:50.725278 (+110093us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:50.755439 (+ 30161us) log.cc:844] Serialized 14 byte log entry
0709 13:20:50.759442 (+  4003us) log.cc:844] Serialized 14 byte log entry
0709 13:20:50.759616 (+   174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:50.759687 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:50.797724 (+ 38037us) raft_consensus.cc:1685] finished
0709 13:20:50.797746 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:50.797945 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3583,"prepare.queue_time_us":36723,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":3515,"raft.queue_time_us":4708,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":4437,"spinlock_wait_cycles":75370624,"thread_start_us":92424,"threads_started":1,"wal-append.queue_time_us":161526}
I20260709 13:20:50.802716 21316 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:50.806847 21461 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:50.807129 21304 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:50.808467 21363 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:50.809325 21452 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:50.810410 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:50.812363 21116 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:50.816138 15602 raft_consensus.cc:2410] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 13 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:20:50.816992 16203 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:50.818171 15493 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:50.818559 16203 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:50.775460 21335 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:50.825538 21418 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:50.826184 21418 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:50.826279 21363 consensus_queue.cc:1059] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:50.827277 21335 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:50.827492 21363 raft_consensus.cc:3055] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:50.827925 21363 raft_consensus.cc:740] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:50.828785 21363 consensus_queue.cc:260] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:50.825559 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:50.831137 15525 raft_consensus.cc:1240] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:20:50.831581 15493 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:50.831872 15525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 2415) took 1248 ms. Trace:
I20260709 13:20:50.832476 15525 rpcz_store.cc:276] 0709 13:20:49.583310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:50.800357 (+1217047us) service_pool.cc:224] Handling call
0709 13:20:50.800709 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:50.831582 (+ 30873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:50.831813 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:50.830713 16235 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:20:50.834758 16203 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:50.834986 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:50.836218 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:50.837015 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:50.837707 15598 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18.
I20260709 13:20:50.838721 15642 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:50.838943 16203 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:50.839188 15525 raft_consensus.cc:1240] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 18. Current term is 19. Ops: [17.14-18.15]
I20260709 13:20:50.839965 15642 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:50.842104 16205 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:50.846202 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:50.825793 21413 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:50.850011 21413 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:50.852084 16203 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:50.852633 15598 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "50726e0d396948b79be43aece292acb5"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:59198
I20260709 13:20:50.853345 15598 raft_consensus.cc:606] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:50.854696 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:50.856734 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:50.837893 21474 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:50.858242 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:50.859089 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:50.859905 15493 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:50.860662 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:50.773981 21389 consensus_queue.cc:260] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:50.850538 21413 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:50.850960 21304 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:50.862529 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:50.863039 21335 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:50.863601 21320 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:50.865417 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:50.867996 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:50.868400 15676 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18.
W20260709 13:20:50.861781 21510 log.cc:927] Time spent T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:20:50.872169 21348 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:50.838443 21461 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:50.850142 21369 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:50.881209 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:50.838061 21316 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:50.857297 21452 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:50.866490 21175 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:50.857376 21507 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:50.883096 21363 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:50.895808 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:50.902093 21513 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:50.902796 21500 consensus_queue.cc:1059] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:20:50.909273 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:50.885183 21348 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:50.874879 21430 consensus_queue.cc:1048] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
W20260709 13:20:50.915478 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:50.920179 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:50.913966 15526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56840 (request call id 1905) took 1834 ms. Trace:
I20260709 13:20:50.921741 15675 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 22 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:20:50.924497 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:50.925868 15642 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:50.920900 15526 rpcz_store.cc:276] 0709 13:20:49.079962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:49.103607 (+ 23645us) service_pool.cc:224] Handling call
0709 13:20:50.913779 (+1810172us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b5400070dd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:50.913891 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":322756992}
I20260709 13:20:50.927170 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:50.927982 15567 connection.cc:256] RPC call timeout handler was delayed by -1.5522727750001195s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2315s for a response
W20260709 13:20:50.929049 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:50.929207 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:50.929633 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:50.903975 21175 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:50.916841 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:50.931617 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:50.933470 15642 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:50.937064 15567 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:50.938223 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:50.911993 21452 raft_consensus.cc:697] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:50.939632 15526 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
W20260709 13:20:50.937100 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:50.941699 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:50.943094 15642 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:50.918141 21348 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:50.919193 21513 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:50.947484 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2421) took 1076 ms. Trace:
I20260709 13:20:50.897867 21314 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:50.921056 21439 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:50.946847 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:50.948719 15522 rpcz_store.cc:276] 0709 13:20:49.871244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:50.813448 (+942204us) service_pool.cc:224] Handling call
0709 13:20:50.813769 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:50.813988 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:20:50.884557 (+ 70569us) log.cc:844] Serialized 14 byte log entry
0709 13:20:50.945685 (+ 61128us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:50.945738 (+    53us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:50.945790 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:50.945853 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:50.947239 (+  1386us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":124059,"raft.queue_time_us":2086,"thread_start_us":131056,"threads_started":2}
I20260709 13:20:50.951603 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:50.952411 15522 raft_consensus.cc:2410] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:20:50.953174 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:50.938910 21500 consensus_queue.cc:1243] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Peer dd1a14c3173d49dc9b55bb116e3783c9 log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:20:50.966661 15668 raft_consensus.cc:1240] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:50.936468 21304 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:50.968084 15668 raft_consensus.cc:1275] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:20:50.970151 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:50.971309 21335 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:50.971881 21335 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:50.974144 21335 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:50.940550 21451 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:50.936468 21098 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:20:50.981884 16231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57244 (request call id 1698) took 3371 ms (client timeout 1794 ms). Trace:
W20260709 13:20:50.982369 16231 rpcz_store.cc:269] 0709 13:20:47.610207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.612563 (+  2356us) service_pool.cc:224] Handling call
0709 13:20:50.981650 (+3369087us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b5400720e54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:50.981808 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3632212224}
I20260709 13:20:50.986392 16234 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 22 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 13:20:50.986821 16234 pending_rounds.cc:85] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9: Aborting all ops after (but not including) 15
I20260709 13:20:50.986255 21452 consensus_queue.cc:237] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:50.987031 16234 pending_rounds.cc:107] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 13:20:50.987280 16234 raft_consensus.cc:2889] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:50.989339 16234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1723) took 1519 ms. Trace:
I20260709 13:20:50.990851 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2422) took 1119 ms. Trace:
I20260709 13:20:50.991243 15523 rpcz_store.cc:276] 0709 13:20:49.871512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:50.353126 (+481614us) service_pool.cc:224] Handling call
0709 13:20:50.990654 (+637528us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b54002fa3d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:50.990788 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47170816}
I20260709 13:20:50.992457 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:50.993556 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:50.955892 15522 raft_consensus.cc:2393] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 10: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:51.013026 15523 raft_consensus.cc:2393] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 17: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:50.956775 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:50.989699 16234 rpcz_store.cc:276] 0709 13:20:49.469878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:49.492121 (+ 22243us) service_pool.cc:224] Handling call
0709 13:20:50.986035 (+1493914us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:50.987720 (+  1685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:50.989104 (+  1384us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b5400720e50. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:50.989266 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3435016064}
W20260709 13:20:50.958478 21367 log.cc:927] Time spent T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 1.466s	user 0.012s	sys 0.010s
I20260709 13:20:51.015451 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:50.994565 16232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1719) took 1536 ms. Trace:
I20260709 13:20:50.939352 21175 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:50.627148 15520 rpcz_store.cc:276] 0709 13:20:49.070089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:50.394856 (+1324767us) service_pool.cc:224] Handling call
0709 13:20:50.395109 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:50.395287 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:20:50.511277 (+115990us) log.cc:844] Serialized 14 byte log entry
0709 13:20:50.626259 (+114982us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:50.626325 (+    66us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:50.626369 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:50.626434 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:50.626556 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4365,"thread_start_us":214845,"threads_started":2}
I20260709 13:20:50.999866 21439 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.002733 21314 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:50.951766 21513 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:50.947760 21318 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:50.991295 15524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1908) took 1739 ms. Trace:
I20260709 13:20:51.004868 15674 raft_consensus.cc:1240] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:20:51.006839 15493 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:50.971339 16204 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:20:51.011775 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 68764.2us
I20260709 13:20:51.033001 15642 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:51.034504 15642 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:51.035821 15642 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:51.030171 15524 rpcz_store.cc:276] 0709 13:20:49.252286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:50.680828 (+1428542us) service_pool.cc:224] Handling call
0709 13:20:50.686130 (+  5302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:50.686493 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:50.686523 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:50.687018 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 13:20:50.878151 (+191133us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:50.878194 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:50.878267 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:50.991082 (+112815us) raft_consensus.cc:1685] finished
0709 13:20:50.991103 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:50.991242 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5874,"raft.queue_time_us":90,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":9898,"thread_start_us":150716,"threads_started":1,"wal-append.queue_time_us":221862}
W20260709 13:20:51.012060 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 54299.1us
I20260709 13:20:51.016530 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:51.017675 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:50.972360 21116 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:51.018400 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:51.016232 16232 rpcz_store.cc:276] 0709 13:20:49.457677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:49.460846 (+  3169us) service_pool.cc:224] Handling call
0709 13:20:49.461091 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:49.622361 (+161270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:50.994337 (+1371976us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54006f13d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:50.994508 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323283840}
I20260709 13:20:51.005260 21177 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:51.047477 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 76209.5us
I20260709 13:20:51.015352 21098 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:20:51.055593 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 48875.3us
I20260709 13:20:51.049705 15520 raft_consensus.cc:1240] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:51.058713 15520 raft_consensus.cc:1275] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:20:51.059105 15520 pending_rounds.cc:85] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 9
I20260709 13:20:51.059347 15520 pending_rounds.cc:107] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:20:51.059584 15520 raft_consensus.cc:2889] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:51.053108 15522 raft_consensus.cc:1275] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:20:51.027442 21430 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:51.077394 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 2414) took 1494 ms. Trace:
I20260709 13:20:51.067094 15526 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 17.
I20260709 13:20:51.080699 15567 leader_election.cc:400] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 13. Current term is 14.
I20260709 13:20:51.081203 15567 leader_election.cc:403] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:51.048044 21451 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:51.077991 15521 rpcz_store.cc:276] 0709 13:20:49.583002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:50.724718 (+1141716us) service_pool.cc:224] Handling call
0709 13:20:50.725081 (+   363us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:50.731656 (+  6575us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:50.731690 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:50.732427 (+   737us) log.cc:844] Serialized 52 byte log entry
0709 13:20:50.889501 (+157074us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:51.061714 (+172213us) log.cc:844] Serialized 14 byte log entry
0709 13:20:51.075492 (+ 13778us) log.cc:844] Serialized 14 byte log entry
0709 13:20:51.077047 (+  1555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:51.077125 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:51.077163 (+    38us) raft_consensus.cc:1685] finished
0709 13:20:51.077175 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:51.077331 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20244,"prepare.queue_time_us":114317,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":20714,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":15651,"thread_start_us":224557,"threads_started":2,"wal-append.queue_time_us":171086}
I20260709 13:20:51.095953 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:51.096731 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:51.086992 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:51.088044 21182 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:51.090009 21104 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:51.104696 21465 raft_consensus.cc:3055] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:51.092550 15567 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:51.105144 21465 raft_consensus.cc:740] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:51.105979 21465 consensus_queue.cc:260] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:51.107293 15567 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:51.051791 21304 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:51.095333 15598 raft_consensus.cc:1240] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 13. Current term is 14. Ops: [12.7-13.8]
I20260709 13:20:51.095187 15641 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:51.098819 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:51.099464 15520 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:20:51.063540 21335 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:51.103650 21407 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:51.094264 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:51.052057 21416 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:20:51.064567 21390 consensus_queue.cc:1059] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.078s
I20260709 13:20:51.065454 21427 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:51.172192 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:51.131315 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:51.175261 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:51.131697 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:51.140605 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1909) took 1586 ms. Trace:
I20260709 13:20:51.138064 16234 raft_consensus.cc:1240] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:51.171591 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:51.158738 15670 raft_consensus.cc:1275] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:20:51.177234 15519 rpcz_store.cc:276] 0709 13:20:49.554132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:50.710741 (+1156609us) service_pool.cc:224] Handling call
0709 13:20:50.814056 (+103315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:50.814283 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:20:50.947565 (+133282us) log.cc:844] Serialized 14 byte log entry
0709 13:20:51.056066 (+108501us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:51.056114 (+    48us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:51.056155 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:51.056254 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:51.140429 (+ 84175us) spinlock_profiling.cc:243] Waited 87.2 ms on lock 0x7b54003ce1d0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:51.140543 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3330,"prepare.queue_time_us":189219,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":41,"raft.queue_time_us":12587,"spinlock_wait_cycles":231555968,"thread_start_us":236518,"threads_started":2,"wal-append.queue_time_us":141588}
I20260709 13:20:51.150988 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 20 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:51.137822 16232 raft_consensus.cc:1240] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:51.176578 15594 raft_consensus.cc:2393] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 12: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:51.176008 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:51.179705 15493 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:51.138278 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:51.177968 15668 raft_consensus.cc:2393] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 13: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:51.064741 21505 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:51.179580 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:51.183935 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 20 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:51.184764 16238 raft_consensus.cc:2410] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 20 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:20:51.182854 15491 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:20:51.181325 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 86237.6us
I20260709 13:20:51.183012 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:51.176975 15673 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:20:51.187222 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:51.186549 15641 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:51.079893 21431 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:51.193020 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:51.194275 15491 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:51.190145 15668 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:51.149179 21100 consensus_queue.cc:1059] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:20:51.172564 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:51.192315 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:51.197809 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:51.195838 15641 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:51.196280 15668 raft_consensus.cc:1240] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:51.198658 15669 raft_consensus.cc:1240] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:51.107858 21465 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:51.156697 21389 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:51.202296 15673 raft_consensus.cc:1240] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:51.202121 15641 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:51.121471 21439 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:51.148850 21369 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:51.203274 21505 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:51.208135 15491 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:51.204968 16236 raft_consensus.cc:2393] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 13: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:51.107910 21266 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:51.154269 21481 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:51.143000 21500 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:51.150112 21116 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:51.193086 21412 raft_consensus.cc:3055] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:51.228570 15641 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:51.188437 21453 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:20:51.217257 15493 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:51.085556 21430 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:51.242111 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 134890us
I20260709 13:20:51.200289 21434 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:51.198910 21349 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:51.193397 21426 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.135304 21442 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:51.148435 21413 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.193337 21348 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.202685 21304 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:51.183413 21335 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:51.253604 15641 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:51.114530 21177 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.134s
I20260709 13:20:51.284976 21430 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:51.250424 21369 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.287879 21369 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:51.255733 21500 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.292052 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:51.293413 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:51.251961 21116 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.294646 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:51.295922 21116 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:51.296633 21390 consensus_queue.cc:1059] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:51.297647 21390 raft_consensus.cc:3055] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:51.298049 21390 raft_consensus.cc:740] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:51.298991 21390 consensus_queue.cc:260] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.300396 21390 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:51.259531 21349 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:51.302349 21349 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:51.307365 21320 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:51.309052 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:51.255636 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 47574.1us
I20260709 13:20:51.250665 21412 raft_consensus.cc:740] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:51.312326 15670 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:51.208585 21482 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:51.258939 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:51.259956 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:51.256901 15567 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:51.270520 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:51.270428 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:51.239981 21098 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:51.247430 21201 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:51.184935 21427 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.323866 21482 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:51.325219 21482 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:51.325999 21427 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:51.326256 15522 raft_consensus.cc:1240] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:51.324703 16235 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 22.
I20260709 13:20:51.327265 21348 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:51.271071 21474 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:51.270823 21316 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:51.328380 21201 raft_consensus.cc:2804] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:51.328074 21412 consensus_queue.cc:260] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.328873 21201 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:51.329243 21201 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:51.329519 21412 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:51.338512 21104 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.339138 21104 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.339618 21412 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:51.281667 15641 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:20:51.340663 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 59114.6us
I20260709 13:20:51.328361 21316 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:51.343092 21316 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:51.344132 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:51.344398 21428 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:51.344980 21428 raft_consensus.cc:697] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:51.345458 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:51.345841 21428 consensus_queue.cc:237] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:51.347242 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:51.348425 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:51.348892 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:51.349882 21104 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:51.350586 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 2393) took 2996 ms. Trace:
I20260709 13:20:51.350965 15528 rpcz_store.cc:276] 0709 13:20:48.353985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:48.355299 (+  1314us) service_pool.cc:224] Handling call
0709 13:20:49.970396 (+1615097us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:50.013818 (+ 43422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:51.350378 (+1336560us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b5400023350. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:51.350499 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3713479552}
W20260709 13:20:51.327996 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 71270.5us
I20260709 13:20:51.352607 21513 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.353093 15567 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:51.300257 21175 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.256503 21481 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:51.280450 21335 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:51.355795 21175 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:51.242902 15566 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:51.354086 21492 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:51.358842 16240 raft_consensus.cc:1240] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:51.360740 15641 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:51.361966 15567 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:51.358693 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 115871us
I20260709 13:20:51.293692 21500 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:51.278793 21426 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.367136 16238 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:20:51.279884 21304 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:51.366051 15566 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:51.369496 21320 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:51.371840 21426 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:51.376469 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:51.380367 15567 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:51.355661 21427 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:51.373486 21466 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:20:51.391083 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 173991us
I20260709 13:20:51.388322 15567 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:51.378175 21304 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:51.379045 21468 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:51.359025 21316 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:51.394145 15493 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:51.386250 21320 raft_consensus.cc:697] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
W20260709 13:20:51.404099 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 125060us
I20260709 13:20:51.404882 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:51.405596 15520 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:20:51.406603 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:51.410043 15519 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:20:51.392722 21268 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:51.408186 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:51.391193 21466 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:51.407146 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:51.419333 16204 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:51.421240 15527 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:20:51.386900 21339 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:51.418015 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:51.386675 21177 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:51.381995 21412 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.422045 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:51.397243 21426 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:51.423307 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:51.402326 21103 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:51.425678 15521 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:51.423887 15641 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:51.439324 15641 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:51.387368 21427 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.394359 21276 consensus_queue.cc:1059] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 13:20:51.437325 21500 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:51.443912 21427 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:51.394495 21419 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.026s
I20260709 13:20:51.444572 21427 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:51.443830 21500 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:51.446982 21500 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:51.419183 21492 consensus_queue.cc:1059] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:20:51.457312 15491 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:51.457458 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:51.464718 21513 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:51.466028 21201 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.466504 21413 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:51.462716 21465 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:51.467299 21465 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.425737 21466 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:51.470630 21098 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:51.471029 21098 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:51.471387 21098 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:51.429376 21445 consensus_queue.cc:1048] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:20:51.430835 21442 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:20:51.463632 21532 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:51.470319 15524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1922) took 1197 ms. Trace:
I20260709 13:20:51.477764 15524 rpcz_store.cc:276] 0709 13:20:50.272833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:51.051552 (+778719us) service_pool.cc:224] Handling call
0709 13:20:51.054292 (+  2740us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:51.055550 (+  1258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:51.055582 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:51.065236 (+  9654us) log.cc:844] Serialized 26 byte log entry
0709 13:20:51.373106 (+307870us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:51.373149 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:51.373242 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:51.470094 (+ 96852us) raft_consensus.cc:1685] finished
0709 13:20:51.470115 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:51.470258 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":890,"raft.queue_time_us":10116,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":7565,"thread_start_us":272541,"threads_started":1,"wal-append.queue_time_us":347463}
I20260709 13:20:51.485002 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:51.486060 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:51.486666 15670 raft_consensus.cc:2393] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 11: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:51.489104 15519 raft_consensus.cc:1240] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:51.491418 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:51.481752 21109 consensus_queue.cc:1048] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:20:51.494889 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:51.495733 15677 raft_consensus.cc:2393] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 19: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:51.496580 20887 raft_consensus.cc:3055] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:51.496989 20887 raft_consensus.cc:740] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:51.497766 20887 consensus_queue.cc:260] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:51.498927 20887 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:51.469471 21465 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:51.470434 21407 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 13. Current term is 14.
I20260709 13:20:51.485551 15676 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
W20260709 13:20:51.502004 21266 consensus_queue.cc:1175] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3ef186ec12554a1d87814af0a0ace9a4" responder_term: 14 status { last_received { term: 13 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 13. Current term is 14. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:20:51.504201 21316 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:51.470432 21416 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:51.505940 21416 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:51.506382 21416 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:51.486466 21532 raft_consensus.cc:697] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:51.509527 21434 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:51.509351 21532 consensus_queue.cc:237] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:51.509878 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:51.510897 21434 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:51.510248 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 68977.2us
I20260709 13:20:51.514073 21442 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:51.514470 15527 raft_consensus.cc:1240] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:51.516369 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:51.492072 15520 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:51.469415 21349 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.523910 15521 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:20:51.518504 15491 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:51.526276 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:51.510527 15595 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:51.525206 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:51.531416 15524 raft_consensus.cc:1240] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 12. Current term is 13. Ops: []
W20260709 13:20:51.535112 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 48794.9us
I20260709 13:20:51.535307 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:51.531076 15597 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 16.
I20260709 13:20:51.536170 16204 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:51.532371 15641 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:51.539407 15601 raft_consensus.cc:1240] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:20:51.509727 21268 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:51.539783 16238 raft_consensus.cc:606] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:51.499063 21468 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:51.551144 16238 raft_consensus.cc:614] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:51.549715 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:51.545814 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:51.552906 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:51.559403 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:51.562364 15642 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:51.574457 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 81491.3us
I20260709 13:20:51.575323 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:51.576663 15642 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:51.579286 15642 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:51.591692 15595 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 19.
I20260709 13:20:51.552779 21316 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:51.593565 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:51.596845 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 67095.8us
I20260709 13:20:51.558584 21451 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:51.575130 21500 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:51.528070 21442 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:51.607966 16204 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:51.578306 21428 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.584182 21439 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:51.577320 21533 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:51.580732 21452 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:51.547017 21436 consensus_queue.cc:1059] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:20:51.600101 21316 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:51.574682 21276 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:20:51.547217 21507 consensus_queue.cc:1048] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:20:51.605468 21500 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:51.676155 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 52025.2us
I20260709 13:20:51.581197 21177 consensus_queue.cc:1059] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:20:51.620590 21453 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:51.664355 15525 raft_consensus.cc:1275] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:20:51.681213 15525 pending_rounds.cc:85] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 7
I20260709 13:20:51.681494 15525 pending_rounds.cc:107] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:20:51.681821 15525 raft_consensus.cc:2889] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:51.549124 21320 consensus_queue.cc:237] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:51.600131 21103 consensus_queue.cc:1059] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 13:20:51.620821 21533 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:51.689407 21533 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:51.600852 21482 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.665659 21529 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:51.631631 21316 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:51.678413 21500 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:51.618669 21428 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:51.617062 21268 consensus_queue.cc:1059] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:20:51.738865 21103 raft_consensus.cc:3055] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:51.739279 21103 raft_consensus.cc:740] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:51.740041 21103 consensus_queue.cc:260] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.737622 21098 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:51.714130 21177 raft_consensus.cc:3055] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:51.716861 21482 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:51.718349 15567 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:51.718418 21428 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:51.720039 16205 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:51.681404 21448 consensus_queue.cc:1059] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:20:51.745208 21194 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:51.745771 21194 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:51.746152 21194 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:51.747268 21201 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:51.748003 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:51.748555 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:51.720885 21407 consensus_queue.cc:1059] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:51.749624 21407 raft_consensus.cc:3055] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:51.749891 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:51.751087 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:51.751758 21492 raft_consensus.cc:3055] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:51.752125 21492 raft_consensus.cc:740] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:51.752279 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:51.753017 21492 consensus_queue.cc:260] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:51.753602 15668 raft_consensus.cc:1240] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:51.722043 21466 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:51.755724 21466 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:51.755877 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:51.756229 15594 raft_consensus.cc:1240] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:51.756644 15522 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:51.760322 21414 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:51.760973 21414 raft_consensus.cc:697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:51.762024 21412 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:51.761871 21414 consensus_queue.cc:237] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.752043 21098 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:51.765659 21529 consensus_queue.cc:1059] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:51.766287 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:51.766628 21529 raft_consensus.cc:3055] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:51.767009 21529 raft_consensus.cc:740] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:51.768213 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:51.724499 21416 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:51.769230 15671 raft_consensus.cc:1240] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:51.726289 21453 consensus_queue.cc:1048] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:51.770346 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:51.771435 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:51.771992 15522 raft_consensus.cc:2468] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:51.772651 15523 raft_consensus.cc:1275] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:20:51.772970 16231 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:20:51.721386 21192 raft_consensus.cc:3055] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:51.773821 21435 raft_consensus.cc:3055] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:51.774135 21192 raft_consensus.cc:740] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:51.775250 21434 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:51.775017 21192 consensus_queue.cc:260] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.738801 21268 raft_consensus.cc:3055] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:51.685616 21304 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.776275 21268 raft_consensus.cc:740] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:51.776484 21192 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:51.665220 21229 consensus_queue.cc:1059] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:20:51.777752 21349 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:51.723882 21500 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.778581 21229 raft_consensus.cc:3055] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:51.778961 21229 raft_consensus.cc:740] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:51.779680 21500 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:51.736902 16234 raft_consensus.cc:1240] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:51.779783 21229 consensus_queue.cc:260] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.780921 16231 raft_consensus.cc:606] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:51.781073 21229 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:51.778247 21435 raft_consensus.cc:740] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:20:51.662693 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 54824.8us
I20260709 13:20:51.689838 21335 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.782613 21435 consensus_queue.cc:260] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.783599 16233 raft_consensus.cc:1240] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:51.783882 21435 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:51.744520 21177 raft_consensus.cc:740] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:51.748587 15670 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:51.767817 21529 consensus_queue.cc:260] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.743224 21418 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:51.743561 21533 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.784796 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:51.827312 16239 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:51.785842 15594 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:20:51.735244 21442 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:51.723874 15600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 1621) took 1367 ms. Trace:
I20260709 13:20:51.787482 15520 raft_consensus.cc:1240] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:51.787667 16231 raft_consensus.cc:614] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:20:51.791703 16203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:51.772190 15674 raft_consensus.cc:2393] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 14: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:51.792544 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:51.834180 16203 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:51.771317 21416 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:51.823937 21349 consensus_queue.cc:1059] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:20:51.782655 21545 log.cc:927] Time spent T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:20:51.778020 21304 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:51.831437 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:51.795348 15675 raft_consensus.cc:1240] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:51.836030 15668 raft_consensus.cc:1240] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:51.836040 15527 raft_consensus.cc:1240] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:20:51.842476 16203 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:51.832968 15600 rpcz_store.cc:276] 0709 13:20:50.356694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:50.363784 (+  7090us) service_pool.cc:224] Handling call
0709 13:20:50.364094 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:50.487490 (+123396us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:20:50.516431 (+ 28941us) log.cc:844] Serialized 14 byte log entry
0709 13:20:50.712555 (+196124us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:50.712607 (+    52us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:20:50.712653 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:51.722555 (+1009902us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b1800320620. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce18ec2a6 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419
0709 13:20:51.723646 (+  1091us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54000a8f54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:51.723694 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:51.723812 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10930,"prepare.queue_time_us":43068,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":621,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":2969,"spinlock_wait_cycles":288772736,"thread_start_us":195263,"threads_started":1,"wal-append.queue_time_us":267874,"wal-append.run_cpu_time_us":1210,"wal-append.run_wall_time_us":1035458}
I20260709 13:20:51.785135 21407 raft_consensus.cc:740] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:20:51.834128 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 48846.2us
I20260709 13:20:51.822361 21451 raft_consensus.cc:697] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:51.857486 15493 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:51.846150 15670 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 19.
W20260709 13:20:51.845517 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 266279us
I20260709 13:20:51.861704 15642 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:51.835664 21492 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:51.869304 15493 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:51.834908 21413 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 13:20:51.828629 21313 consensus_queue.cc:1059] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:20:51.823602 21500 consensus_queue.cc:1059] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:20:51.838182 21513 consensus_queue.cc:1048] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:20:51.865892 21390 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:51.879817 15642 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:51.883540 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:51.883744 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:51.888432 15522 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
I20260709 13:20:51.894941 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:51.895924 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:51.897063 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:20:51.897063 15523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1923) took 1622 ms. Trace:
I20260709 13:20:51.899394 15523 rpcz_store.cc:276] 0709 13:20:50.274690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:51.051621 (+776931us) service_pool.cc:224] Handling call
0709 13:20:51.051924 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:51.773003 (+721079us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:51.896857 (+123854us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b54003ccdd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:51.897004 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1562007808}
I20260709 13:20:51.871587 21532 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:51.875136 21439 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:51.865868 21100 consensus_queue.cc:1059] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:20:51.875376 21349 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:51.907194 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:20:51.901185 21539 log.cc:927] Time spent T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.305s	user 0.001s	sys 0.000s
I20260709 13:20:51.913467 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:51.867373 21268 consensus_queue.cc:260] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.917028 21268 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:51.918056 21536 log.cc:927] Time spent T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:20:51.888047 21407 consensus_queue.cc:260] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:51.869603 21177 consensus_queue.cc:260] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:51.924499 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:51.914618 21547 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:51.926081 21194 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.928328 21194 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:51.912837 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:51.916277 15493 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:51.904681 21390 raft_consensus.cc:697] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:51.955199 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:51.955796 21390 consensus_queue.cc:237] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:51.958361 21335 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:51.955835 16233 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:51.947431 21533 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:51.948292 21349 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:51.951633 21532 consensus_queue.cc:1059] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:20:51.952986 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:51.953537 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:51.966679 15594 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:51.967509 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:51.968145 15602 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:20:51.966768 15491 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:51.953948 21447 consensus_queue.cc:1059] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:51.963194 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:51.972322 15567 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:51.975304 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:51.976014 15597 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
I20260709 13:20:51.977443 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:51.971263 15641 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:51.978220 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:51.979889 15597 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:51.966140 21430 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:51.983964 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:51.984858 15599 raft_consensus.cc:1275] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:20:51.987833 16233 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:20:51.912742 21451 consensus_queue.cc:237] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:51.996817 15642 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:51.994396 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:51.971494 21500 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:51.991429 15568 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:51.951599 21316 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:52.005848 15598 raft_consensus.cc:1275] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:20:51.982584 21439 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:20:52.023443 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:52.037561 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:51.995478 21547 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:51.999830 21304 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:20:52.043627 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 71422us
I20260709 13:20:52.043469 15522 raft_consensus.cc:2368] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 15. Current term is 16.
I20260709 13:20:52.062165 21431 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.062778 21431 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:52.063658 21445 raft_consensus.cc:3055] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:52.064149 21445 raft_consensus.cc:740] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:52.066408 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:52.063694 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:52.067605 15528 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:52.067852 21465 raft_consensus.cc:3055] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:52.068250 21465 raft_consensus.cc:740] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:52.069060 21465 consensus_queue.cc:260] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:52.064998 21445 consensus_queue.cc:260] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:52.070330 21109 raft_consensus.cc:3055] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:52.070741 21109 raft_consensus.cc:740] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:52.071648 21109 consensus_queue.cc:260] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:52.067214 15520 raft_consensus.cc:2393] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 12: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:52.078449 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:52.081279 16233 raft_consensus.cc:2393] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 15: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:52.078734 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:20:52.074126 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 82793.5us
I20260709 13:20:52.070689 21445 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:52.086987 21229 consensus_queue.cc:1059] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:52.006460 21549 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:52.092764 15642 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:52.072862 21231 raft_consensus.cc:2804] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:52.113997 21231 raft_consensus.cc:697] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:52.077162 15528 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 15.
I20260709 13:20:52.114945 21231 consensus_queue.cc:237] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:52.096460 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:52.064865 21431 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:52.121356 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 264408us
I20260709 13:20:52.122409 15521 raft_consensus.cc:1240] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:52.043912 21547 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:52.124522 15642 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:52.125874 15594 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 15.
I20260709 13:20:52.127065 15597 raft_consensus.cc:2468] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:20:52.128124 15668 raft_consensus.cc:1240] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:52.145258 16203 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:52.146802 16205 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:52.152101 16203 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:52.157408 15601 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:20:52.091845 21313 raft_consensus.cc:2804] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:52.159456 16203 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:52.161362 16203 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:52.165577 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:52.168545 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:52.169237 15519 raft_consensus.cc:3060] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:52.143947 21547 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:52.151433 21465 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:52.163031 15642 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:52.151154 21429 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:52.151373 21416 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:52.159265 21533 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:52.181847 15525 raft_consensus.cc:1240] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:52.161722 21116 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:52.162060 21546 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:52.184230 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:52.185241 15594 raft_consensus.cc:1240] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:52.145624 21427 consensus_queue.cc:1059] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:20:52.187103 15642 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:52.188385 15597 raft_consensus.cc:2410] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:20:52.193775 15675 raft_consensus.cc:1240] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:52.199642 15519 raft_consensus.cc:2468] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:20:52.170296 21313 raft_consensus.cc:697] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:52.202278 21532 consensus_queue.cc:1059] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:20:52.176409 21175 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:52.210649 15521 raft_consensus.cc:1275] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:20:52.171638 21558 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:20:52.249792 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 62766.6us
W20260709 13:20:52.323683 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:52.203001 21452 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:52.194983 21429 raft_consensus.cc:697] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:52.335016 15642 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:52.184630 21390 consensus_queue.cc:1059] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.095s
I20260709 13:20:52.338806 21320 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:52.339512 21320 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:52.343631 21098 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:52.202026 21453 consensus_queue.cc:1048] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.132s
I20260709 13:20:52.350839 21468 raft_consensus.cc:3055] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:52.351343 21468 raft_consensus.cc:740] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:52.352190 21468 consensus_queue.cc:260] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:52.322154 20887 consensus_queue.cc:1059] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:20:52.359468 20887 raft_consensus.cc:3055] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:52.359869 20887 raft_consensus.cc:740] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:20:52.360011 21555 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 13:20:52.360646 20887 consensus_queue.cc:260] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:52.174152 21549 consensus_queue.cc:1048] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.163s
I20260709 13:20:52.364259 21103 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:52.327234 21533 raft_consensus.cc:3055] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:52.366276 21533 raft_consensus.cc:740] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:52.367137 21533 consensus_queue.cc:260] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:52.368351 21533 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:52.370327 21313 consensus_queue.cc:237] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:52.326762 21546 raft_consensus.cc:2804] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:52.377254 15602 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 18.
I20260709 13:20:52.380414 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:52.387701 15670 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:20:52.394804 15670 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 16.
I20260709 13:20:52.407086 15491 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:52.408468 21431 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:52.409070 21431 raft_consensus.cc:697] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:52.410027 21431 consensus_queue.cc:237] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:52.395718 21435 consensus_meta.cc:287] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.611s	user 0.006s	sys 0.005s
I20260709 13:20:52.422116 21109 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:52.435236 21505 raft_consensus.cc:3055] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:52.452032 21505 raft_consensus.cc:740] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:52.395584 21546 raft_consensus.cc:697] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:52.452940 21505 consensus_queue.cc:260] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:52.438634 21466 raft_consensus.cc:3055] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:52.444233 21116 consensus_queue.cc:1048] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:20:52.454308 21466 raft_consensus.cc:740] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:52.455221 15528 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:20:52.455503 21452 consensus_queue.cc:1048] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:52.455320 21466 consensus_queue.cc:260] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:52.456687 21466 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:52.441452 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:52.441053 21427 consensus_queue.cc:1059] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:20:52.447783 21560 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.460449 21427 raft_consensus.cc:3055] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:52.460166 21560 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:52.453044 21546 consensus_queue.cc:237] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:52.460842 21427 raft_consensus.cc:740] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:52.461697 21427 consensus_queue.cc:260] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:52.463063 21427 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:52.445048 15641 leader_election.cc:400] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 15. Current term is 16.
I20260709 13:20:52.468775 15641 leader_election.cc:403] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:52.470376 15519 raft_consensus.cc:1275] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:20:52.467711 21465 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:20:52.488358 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:52.495694 21560 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:52.486060 21429 consensus_queue.cc:237] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:52.490010 21453 consensus_queue.cc:1048] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:20:52.501133 21413 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:52.505719 21320 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:52.510106 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:52.510814 15519 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:20:52.501730 21413 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:52.513525 21413 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:52.395694 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 223254us
I20260709 13:20:52.516733 16203 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:52.517387 20887 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:52.517652 21427 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:52.521562 21500 raft_consensus.cc:3055] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:52.524134 21500 raft_consensus.cc:740] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:52.524998 21500 consensus_queue.cc:260] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:52.478591 21452 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 15. Current term is 16.
I20260709 13:20:52.522300 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:52.526355 15671 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 8.
I20260709 13:20:52.522497 21465 raft_consensus.cc:697] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:52.527787 15491 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:52.529253 16203 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:52.555759 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:52.558776 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:52.559675 16233 raft_consensus.cc:2393] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 12: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:52.564761 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:52.565516 16231 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 15.
I20260709 13:20:52.570607 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:52.571435 16239 raft_consensus.cc:2393] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 7: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
W20260709 13:20:52.580962 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:52.562471 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:52.584722 16233 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 17.
W20260709 13:20:52.602775 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 207761us
I20260709 13:20:52.605767 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:20:52.526662 21349 consensus_meta.cc:287] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.559s	user 0.007s	sys 0.002s
I20260709 13:20:52.635164 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:52.635905 16231 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:52.603874 21500 raft_consensus.cc:3060] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:20:52.649873 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 46357.8us
I20260709 13:20:52.650851 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:52.651851 16233 raft_consensus.cc:2410] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:20:52.658516 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:52.659148 16236 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:52.660991 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:52.661765 15671 raft_consensus.cc:2393] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 18: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:52.662635 15670 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:52.664064 16231 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 18.
I20260709 13:20:52.648345 16239 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 16.
I20260709 13:20:52.599514 21558 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:52.668141 15493 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:52.665131 15670 raft_consensus.cc:1240] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:52.658074 21349 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:52.686466 21316 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:52.647007 21389 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:20:52.688434 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 160741us
I20260709 13:20:52.654274 21431 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:52.684276 21539 log.cc:927] Time spent T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:20:52.691416 15493 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:52.698032 15673 raft_consensus.cc:1240] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:52.709059 15493 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:52.689296 21389 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:52.694370 21316 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:52.694624 21349 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:52.730823 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 50476.6us
I20260709 13:20:52.738299 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:52.739118 16232 raft_consensus.cc:2393] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 9: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:52.698277 21098 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:52.717031 21389 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:52.707979 21453 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:52.735208 21314 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:52.700644 21431 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:52.762023 16203 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:52.767366 21507 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:52.769719 21390 consensus_queue.cc:1059] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:52.770947 21429 raft_consensus.cc:3055] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:52.771322 21429 raft_consensus.cc:740] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:52.772182 21429 consensus_queue.cc:260] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:52.773341 21429 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:52.773904 15670 raft_consensus.cc:1275] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:52.699405 21465 consensus_queue.cc:237] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:52.778086 21546 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.778743 21546 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:52.779199 21507 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:52.780092 21430 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:52.780582 21430 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:52.784400 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:52.735450 21349 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:52.784983 21547 consensus_meta.cc:287] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.605s	user 0.010s	sys 0.000s
I20260709 13:20:52.755883 21098 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:52.786479 21507 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:20:52.786861 21320 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.806669 21334 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:52.839955 21577 log.cc:927] Time spent T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
W20260709 13:20:52.840569 21584 log.cc:927] Time spent T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:20:52.781118 21546 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:52.784703 21588 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.819828 21571 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.762143 21580 consensus_queue.cc:1059] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.124s
I20260709 13:20:52.762223 21453 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:52.830513 21513 consensus_queue.cc:1059] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.079s
I20260709 13:20:52.881057 21547 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:52.886792 21320 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:52.915215 21349 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:52.961493 21575 log.cc:927] Time spent T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
W20260709 13:20:52.965109 21578 log.cc:927] Time spent T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 13:20:52.893006 21334 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:52.974821 21334 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:52.906607 21571 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:52.975692 21558 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:52.977210 21571 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:52.904800 21588 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:52.980865 21588 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:52.971414 21582 log.cc:927] Time spent T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:20:52.971607 21320 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:52.972083 21583 log.cc:927] Time spent T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:20:52.986747 21431 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:52.988178 21547 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:53.020252 21570 log.cc:927] Time spent T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.234s	user 0.000s	sys 0.000s
I20260709 13:20:53.023784 21389 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.026785 21453 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:53.067042 21430 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.069693 21430 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 23 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:53.138965 15519 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.004s	sys 0.032s
W20260709 13:20:53.139935 15519 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.004s	sys 0.032s
W20260709 13:20:53.151728 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:52.802219 16236 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 16.
I20260709 13:20:53.161006 20887 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:53.162487 20887 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.165794 20887 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:53.166522 20887 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.169191 21098 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:53.152064 15601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34102 (request call id 1645) took 1189 ms. Trace:
I20260709 13:20:53.246474 21389 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:53.247664 21453 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.248631 21453 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:53.252141 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 542997us
I20260709 13:20:53.252983 21500 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:53.254235 21098 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:53.255215 21098 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:53.257278 21576 log.cc:927] Time spent T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.417s	user 0.004s	sys 0.011s
I20260709 13:20:53.260179 21077 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.261301 21077 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.264304 21077 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:53.265460 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:53.265991 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.266918 21533 raft_consensus.cc:3055] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:53.267412 21533 raft_consensus.cc:740] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:53.269138 21533 consensus_queue.cc:260] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.271741 21533 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:53.272032 21549 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.272969 21549 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.277100 21413 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.277379 21580 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.278033 21413 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.278369 21580 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.280164 21339 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:53.280889 21413 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:53.281095 21580 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:53.264099 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:53.295754 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:53.296489 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:53.297297 15598 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:53.295823 15602 raft_consensus.cc:1275] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:20:53.296520 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:53.304396 21529 raft_consensus.cc:3060] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:53.307739 21453 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:53.281107 21339 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.167202 20887 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:53.309732 21500 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:53.178328 15601 rpcz_store.cc:276] 0709 13:20:51.962683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:51.962962 (+   279us) service_pool.cc:224] Handling call
0709 13:20:53.151568 (+1188606us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b540043ba54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:53.151821 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":238340864}
I20260709 13:20:53.315900 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:53.260432 21547 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:53.318733 21547 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.320300 15524 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:20:53.321024 21547 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:53.321508 21452 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:53.322332 21452 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.325060 21452 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:53.328523 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:53.329753 21549 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:53.331007 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:53.333698 20887 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:53.326442 15568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.334498 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:53.327198 21339 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:53.328653 21439 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:53.336690 15524 raft_consensus.cc:1275] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:20:53.337109 21439 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:53.338802 15521 raft_consensus.cc:1275] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:20:53.349640 21505 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:53.349836 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:53.350867 15526 raft_consensus.cc:2393] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 10: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:53.352306 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:53.353892 15520 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:20:53.326689 21098 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:53.365578 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:53.386840 15598 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 18.
I20260709 13:20:53.393292 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:53.393122 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:53.336614 21339 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:53.394991 15524 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:20:53.408948 15523 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:53.410157 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:53.408396 15521 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 17.
W20260709 13:20:53.414386 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 249690us
I20260709 13:20:53.431679 15493 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:53.434630 21339 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:53.437198 21468 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:53.437557 21339 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:53.426390 15567 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.572s (SENT) [suppressed 27 similar messages]
I20260709 13:20:53.449628 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:53.452620 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:53.452961 15671 raft_consensus.cc:2410] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:20:53.438063 21389 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:53.453418 21175 raft_consensus.cc:3055] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:53.445766 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:53.454420 21430 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 13:20:53.454746 15672 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:20:53.454030 21175 raft_consensus.cc:740] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:20:53.458484 15528 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 2445) took 1293 ms (client timeout 1572 ms). Trace:
I20260709 13:20:53.458256 21175 consensus_queue.cc:260] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:53.458899 15528 rpcz_store.cc:269] 0709 13:20:52.165157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:52.165299 (+   142us) service_pool.cc:224] Handling call
0709 13:20:53.458154 (+1292855us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54003cd2d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:53.458317 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":538500480}
I20260709 13:20:53.459483 21175 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:20:53.461102 21579 log.cc:927] Time spent T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.737s	user 0.008s	sys 0.022s
I20260709 13:20:53.464021 15674 raft_consensus.cc:1240] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:53.439792 21407 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:53.454442 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:53.470912 15493 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:53.475421 15523 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 18.
I20260709 13:20:53.437832 21533 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:20:53.482815 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:53.490558 15675 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:20:53.501288 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:53.502194 15598 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
I20260709 13:20:53.507577 15671 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:20:53.507980 21601 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:53.511154 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:53.516850 21596 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.517637 21596 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.517201 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2455) took 1003 ms. Trace:
I20260709 13:20:53.507577 21560 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:53.519795 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:53.527163 15675 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:20:53.515141 21098 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.528249 15526 raft_consensus.cc:2410] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:20:53.527895 21547 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:53.530097 21098 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:53.530534 21547 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.532416 21098 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:53.532678 21547 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:53.512187 21601 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.514955 21452 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:53.539995 21452 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.542625 15601 raft_consensus.cc:1275] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:20:53.520953 15519 rpcz_store.cc:276] 0709 13:20:52.514283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:52.514793 (+   510us) service_pool.cc:224] Handling call
0709 13:20:52.515104 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:52.526960 (+ 11856us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:20:53.140673 (+613713us) log.cc:844] Serialized 14 byte log entry
0709 13:20:53.427969 (+287296us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:53.494281 (+ 66312us) log.cc:844] Serialized 26 byte log entry
0709 13:20:53.496815 (+  2534us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:53.496854 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:53.498540 (+  1686us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:53.516927 (+ 18387us) raft_consensus.cc:1685] finished
0709 13:20:53.516946 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:53.517136 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27055,"prepare.queue_time_us":190628,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":22770,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":11605,"spinlock_wait_cycles":41630464,"thread_start_us":804422,"threads_started":2,"wal-append.queue_time_us":341542}
I20260709 13:20:53.529175 16203 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:53.544685 16204 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:53.538365 21560 raft_consensus.cc:697] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
W20260709 13:20:53.548123 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 99920.1us
W20260709 13:20:53.549127 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:53.549237 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 132098us
I20260709 13:20:53.554742 21547 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:53.555440 21547 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:53.559191 21531 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.559881 21531 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.562953 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:53.563810 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.563342 21560 consensus_queue.cc:237] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:53.564347 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.566160 21597 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:53.566617 21597 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:53.566991 21597 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:53.569428 21389 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.570078 21389 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.572305 21389 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:53.557657 21547 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:53.579687 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 145847us
I20260709 13:20:53.581717 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:53.582278 21601 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:53.582428 15676 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:53.583577 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:53.584306 15601 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 18.
I20260709 13:20:53.587527 21596 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:53.588943 21533 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:53.589555 21533 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.579190 16204 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:53.591877 21533 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:53.592533 21533 raft_consensus.cc:2804] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:53.592902 21533 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:53.577435 21452 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:53.598177 21177 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:53.598908 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:53.599712 16232 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:53.600042 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:53.601079 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:53.554275 21175 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:53.610666 21513 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:53.609469 21175 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:53.612680 21175 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:53.614557 21314 consensus_meta.cc:287] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.849s	user 0.012s	sys 0.008s
I20260709 13:20:53.615057 21314 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:53.615816 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:53.617008 21100 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.617679 21100 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:53.615087 21335 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:53.619033 21335 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.619561 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:53.620352 16236 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 18.
I20260709 13:20:53.619110 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:53.623140 16233 raft_consensus.cc:2393] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 17: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:53.625454 15568 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:53.632515 20887 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:53.632962 20887 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:53.633334 20887 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:53.633906 21335 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:53.634936 15673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49094 (request call id 2462) took 1245 ms. Trace:
I20260709 13:20:53.635355 15673 rpcz_store.cc:276] 0709 13:20:52.380375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:52.380824 (+   449us) service_pool.cc:224] Handling call
0709 13:20:52.381101 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:52.701163 (+320062us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:53.621654 (+920491us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b54001a8454. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:53.625962 (+  4308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":728396544}
I20260709 13:20:53.636906 21531 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:53.606540 21505 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:53.593263 21533 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:53.640709 15598 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:20:53.643419 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:53.643885 21453 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:53.644599 21453 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.647488 21453 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:53.649433 21435 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.650148 21182 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:53.650830 21182 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.652988 21339 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.653815 21339 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:53.655292 16205 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:53.656190 21320 consensus_queue.cc:1048] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:53.657323 21445 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.658703 15601 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:20:53.659943 21445 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:53.662312 21445 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:53.649367 15642 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:53.665980 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:53.657500 21427 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:53.670456 15602 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:20:53.673313 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:53.649767 21100 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:53.674432 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:53.674746 21595 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:53.676432 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:53.664772 21182 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:53.679258 21427 raft_consensus.cc:697] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:53.679827 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:53.680498 21427 consensus_queue.cc:237] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.680953 15420 catalog_manager.cc:5697] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 8 to 11, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 11 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:53.656224 21339 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:53.674696 16234 raft_consensus.cc:2410] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:20:53.686529 16232 raft_consensus.cc:2393] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 9: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:53.686362 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:53.688406 15598 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
I20260709 13:20:53.674335 21435 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.690795 21435 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:53.671940 21531 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:53.694368 20887 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:53.696583 16234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57244 (request call id 1749) took 1910 ms (client timeout 1739 ms). Trace:
I20260709 13:20:53.691572 21561 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:53.692358 21531 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.637904 21505 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.719058 16235 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 23 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 22. Current term is 23. Ops: []
I20260709 13:20:53.704085 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:53.722560 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:53.717530 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:53.718547 15601 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:20:53.704521 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:53.715229 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:53.770324 15598 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 7.
I20260709 13:20:53.704012 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:53.769214 16236 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:53.714941 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:53.673336 15642 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:53.725610 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:53.696714 20887 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:53.751188 15519 raft_consensus.cc:1275] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:53.748751 16204 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:53.701948 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:53.711207 15420 catalog_manager.cc:5697] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:53.711386 21100 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:53.780084 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 62303.5us
I20260709 13:20:53.781495 15567 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:53.712868 15421 catalog_manager.cc:5244] ChangeConfig:REMOVE_PEER RPC for tablet b99eaeae24b44aa78cfa8c90caca288e with cas_config_opid_index 5: aborting the task: latest config opid_index 8; task opid_index 5
W20260709 13:20:53.780081 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 56590.1us
I20260709 13:20:53.705703 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:53.765619 16234 rpcz_store.cc:269] 0709 13:20:51.785871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:51.786204 (+   333us) service_pool.cc:224] Handling call
0709 13:20:53.696333 (+1910129us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b54005136d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:53.696491 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4178929920}
I20260709 13:20:53.787477 15675 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
I20260709 13:20:53.697391 21334 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.711136 21229 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:53.803406 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 71076us
I20260709 13:20:53.778628 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:53.699132 21426 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:53.795861 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 47233.4us
W20260709 13:20:53.801872 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 131504us
I20260709 13:20:53.807570 21468 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:53.808151 16237 raft_consensus.cc:1275] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:20:53.808560 15421 catalog_manager.cc:5244] ChangeConfig:REMOVE_PEER RPC for tablet 0af5649725244ae599dfc6ed7aeac5f1 with cas_config_opid_index 5: aborting the task: latest config opid_index 7; task opid_index 5
I20260709 13:20:53.808503 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:53.814026 21549 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:53.814613 21549 raft_consensus.cc:697] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:53.815706 21549 consensus_queue.cc:237] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:53.807974 21466 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:53.809032 16205 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:53.808878 21531 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:53.818243 21369 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:53.810118 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:53.819789 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:53.824651 16236 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:53.812088 16231 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
I20260709 13:20:53.811964 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:53.825934 15528 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 17.
I20260709 13:20:53.779867 21597 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.739884 21546 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:53.804140 21335 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:53.837078 21304 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:53.837534 21304 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:53.808499 21334 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.714264 21439 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.845795 15670 raft_consensus.cc:2393] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 23: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:53.783475 21100 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.847544 16232 raft_consensus.cc:2410] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:20:53.857142 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 55688us
I20260709 13:20:53.849761 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:53.850960 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:53.855952 15568 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:53.885046 15598 raft_consensus.cc:2410] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:20:53.841679 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:53.887228 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:53.854208 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:53.887648 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:53.837494 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:53.838234 21304 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:53.843578 21116 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:20:53.893335 21505 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:53.894619 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:53.895437 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:53.895666 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 58265.9us
I20260709 13:20:53.896078 15675 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:53.897778 16239 raft_consensus.cc:2368] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 17. Current term is 18.
I20260709 13:20:53.896379 15642 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:53.837893 21513 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:53.907070 15521 raft_consensus.cc:2393] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 17: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:53.845407 21439 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 23 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:53.910122 21533 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.917979 16233 raft_consensus.cc:2393] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 8: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:53.918741 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:53.920081 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:53.890810 21453 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:53.938170 15420 catalog_manager.cc:5697] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 7 to 13, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 13 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:53.938862 16205 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:53.939111 21427 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:53.939749 21427 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.940418 21597 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:53.842133 21595 consensus_queue.cc:1048] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:20:53.940780 21430 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:53.941344 21414 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 13:20:53.941998 21595 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:53.942759 21434 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:53.897310 21320 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:53.846140 21426 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.943178 21434 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:53.943619 21434 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:53.944926 21426 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:53.907301 21546 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:53.948163 21546 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:53.948694 21453 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.951089 21453 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:53.951304 16231 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:20:53.857316 21182 consensus_queue.cc:1048] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:20:53.925202 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:53.925381 15421 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58802 (request call id 261) took 47092 ms (client timeout 15000 ms). Trace:
I20260709 13:20:53.926590 21103 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:53.930454 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:53.961414 15677 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:20:53.930824 21442 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:53.894646 21468 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:53.932294 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:53.935268 21175 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.936482 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:53.937378 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:53.864363 21561 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.942555 21595 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:53.920269 15600 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 17.
I20260709 13:20:53.820072 21229 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:53.913507 21513 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:53.856145 21339 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.056s
I20260709 13:20:53.960080 15669 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:53.955754 15566 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:53.894068 15641 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:53.922433 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:53.980906 16235 raft_consensus.cc:1275] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:20:53.973238 16240 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:53.983871 15598 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:20:53.856936 21466 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.102s
I20260709 13:20:53.984354 16235 pending_rounds.cc:85] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9: Aborting all ops after (but not including) 8
I20260709 13:20:53.990280 21447 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.991127 21339 raft_consensus.cc:3055] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:20:53.972299 21442 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:53.990442 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 183903us
W20260709 13:20:53.999544 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 46689.3us
I20260709 13:20:54.000155 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:54.001572 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:53.999112 15668 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:20:54.013319 16238 raft_consensus.cc:2393] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 10: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
W20260709 13:20:54.017511 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.996518 16235 pending_rounds.cc:107] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:20:54.019850 16235 raft_consensus.cc:2889] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:54.036132 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:54.036846 15597 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:20:53.987149 21513 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:53.980921 21445 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.068365 15672 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 23 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 22. Current term is 23. Ops: []
I20260709 13:20:53.995157 21407 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:53.985680 21103 raft_consensus.cc:697] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:20:53.999987 21605 log.cc:927] Time spent T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:20:53.999035 21439 raft_consensus.cc:2764] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:20:54.072544 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:53.995954 21560 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:54.082129 15642 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:54.017975 21339 raft_consensus.cc:740] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:54.077184 15566 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:54.013065 21597 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.019939 21442 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:54.067598 15528 raft_consensus.cc:2376] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14. Re-sending same reply.
I20260709 13:20:54.036777 15420 catalog_manager.cc:5697] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 6 to 12. New cstate: current_term: 12 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:54.085841 21560 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:54.090191 21447 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:54.090404 21445 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:54.092290 21560 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:54.092610 21445 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:54.093415 21339 consensus_queue.cc:260] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:54.094755 21339 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:54.095216 21109 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:54.095898 21109 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:54.085844 21549 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:54.098354 21549 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.098714 21549 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:54.099865 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:54.100639 15675 raft_consensus.cc:2393] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 14: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:54.101336 21266 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:54.101976 21266 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:54.103317 21445 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:54.103729 21445 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.104143 21266 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:54.104090 21445 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:54.115262 21427 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:54.117277 21229 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:54.119077 21175 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:54.122035 21549 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:54.126013 21549 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:54.126315 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:54.127115 21445 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:54.128407 21103 consensus_queue.cc:237] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:54.129563 21445 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:54.130144 21100 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:54.131793 21561 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:54.133502 21595 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:54.134071 21597 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:54.134480 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:54.135365 15676 raft_consensus.cc:2393] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 16: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:54.136439 21597 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:54.136896 21533 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:54.095105 21538 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:54.138090 21538 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:54.138995 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:54.074492 21320 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:54.140558 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:54.141224 15596 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:54.077095 21407 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.142606 21320 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:54.142614 21407 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:54.148382 21304 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:54.150089 21600 log.cc:927] Time spent T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:20:54.150444 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:53.960618 15421 rpcz_store.cc:269] 0709 13:20:06.832613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.842020 (+  9407us) service_pool.cc:224] Handling call
0709 13:20:53.925233 (+47083213us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:45.196088 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:45.389039 (+192951us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:45.400135 (+ 11096us) write_op.cc:435] Acquired schema lock
0709 13:20:45.400156 (+    21us) tablet.cc:662] Decoding operations
0709 13:20:45.402602 (+  2446us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:45.402703 (+   101us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:45.402729 (+    26us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:20:45.403140 (+   411us) tablet.cc:701] Row locks acquired
0709 13:20:45.403188 (+    48us) write_op.cc:260] PREPARE: finished
0709 13:20:45.403372 (+   184us) write_op.cc:270] Start()
0709 13:20:45.403556 (+   184us) write_op.cc:276] Timestamp: P: 1783603245403282 usec, L: 0
0709 13:20:45.403592 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 13:20:45.406443 (+  2851us) log.cc:844] Serialized 913 byte log entry
0709 13:20:46.972706 (+1566263us) spinlock_profiling.cc:243] Waited 8.98 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:20:47.499952 (+527246us) op_driver.cc:464] REPLICATION: finished
0709 13:20:48.471677 (+971725us) write_op.cc:301] APPLY: starting
0709 13:20:48.494287 (+ 22610us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:48.765180 (+270893us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:48.773568 (+  8388us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:49.188706 (+415138us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:49.189007 (+   301us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 13:20:49.189052 (+    45us) write_op.cc:312] APPLY: finished
0709 13:20:49.189366 (+   314us) log.cc:844] Serialized 33 byte log entry
0709 13:20:49.373157 (+183791us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:49.387107 (+ 13950us) write_op.cc:454] Released schema lock
0709 13:20:49.394452 (+  7345us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:49.448924 (+ 54472us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:49.466173 (+ 17249us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3053290,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":878568,"apply.run_cpu_time_us":26353,"apply.run_wall_time_us":1025475,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12503,"num_ops":2,"prepare.queue_time_us":517882,"prepare.run_cpu_time_us":26177,"prepare.run_wall_time_us":442003,"raft.queue_time_us":399610,"raft.run_cpu_time_us":65419,"raft.run_wall_time_us":618755,"replication_time_us":2065950,"spinlock_wait_cycles":20653824,"thread_start_us":1060535,"threads_started":5,"wal-append.queue_time_us":413420,"wal-append.run_cpu_time_us":33516,"wal-append.run_wall_time_us":3839440}]]}
W20260709 13:20:54.153476 21608 log.cc:927] Time spent T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:20:54.095537 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:54.155391 15598 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:20:54.094977 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:54.166433 15677 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:54.086582 15519 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:54.151120 15594 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 16.
I20260709 13:20:54.171708 15599 raft_consensus.cc:1275] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:20:54.168679 15521 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:54.127152 15670 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:54.093779 21568 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.210124 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:54.212316 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:54.216169 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:54.216929 15519 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 7.
I20260709 13:20:54.092290 21447 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:54.238530 15420 catalog_manager.cc:5697] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 12 to 14, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 14 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:54.206286 21109 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:54.251973 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:54.164119 21434 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:54.239720 15526 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:54.254110 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:54.262109 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:54.260324 15522 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:20:54.275061 15521 raft_consensus.cc:1275] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:20:54.280221 21434 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:54.287053 21538 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:54.291014 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:54.291718 15524 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:54.292819 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:54.293409 15524 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 17.
I20260709 13:20:54.221220 21568 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:54.294268 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:54.295677 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:54.296386 15525 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:20:54.299045 15420 catalog_manager.cc:5697] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 7 to 8, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 8 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:54.300695 15420 catalog_manager.cc:5697] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:54.295753 21568 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:54.302438 15596 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:20:54.308059 15594 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 23.
W20260709 13:20:54.310771 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 190627us
I20260709 13:20:54.313148 16203 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:54.314600 15597 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18.
I20260709 13:20:54.317140 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:54.318269 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:54.273115 21320 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.333817 15642 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:54.336114 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:54.337250 15524 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:20:54.337524 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:54.337857 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:54.340232 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:54.341008 16203 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:54.340909 15521 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
I20260709 13:20:54.280221 21334 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:54.358834 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:54.443547 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:54.444347 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 80317.1us
I20260709 13:20:54.344117 21434 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:54.344319 21439 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:54.343991 21568 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:54.446584 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:54.498071 15521 raft_consensus.cc:2393] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 13: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:54.344583 21595 consensus_queue.cc:1048] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.108s
I20260709 13:20:54.442735 21103 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:54.503348 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 162416us
W20260709 13:20:54.496466 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 49961.9us
I20260709 13:20:54.447966 21304 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:54.507026 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:54.500732 15641 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:54.508531 15677 raft_consensus.cc:2393] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 10: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:54.450924 21529 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:20:54.449052 21434 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:54.345840 21320 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:54.460403 21439 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:54.510679 15597 raft_consensus.cc:2393] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 14: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
W20260709 13:20:54.515623 21614 log.cc:927] Time spent T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:20:54.513772 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:54.543015 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:54.547503 16205 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:54.543800 15521 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:20:54.553355 16205 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:54.553694 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:54.541136 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:54.569401 15594 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:20:54.529649 21439 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:54.524065 21434 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:54.562849 21369 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:54.547626 21620 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:20:54.561262 21529 raft_consensus.cc:2804] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:54.608144 15642 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:54.533792 15524 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:54.582746 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:54.589383 21621 log.cc:927] Time spent T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:20:54.511423 21103 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:54.625721 21103 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:54.564378 21304 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:54.607609 21369 raft_consensus.cc:697] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:20:54.613633 21617 log.cc:927] Time spent T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:20:54.615298 15420 catalog_manager.cc:5697] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 9 to 13, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 13 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:54.618707 21407 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:54.620570 21320 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:54.621263 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 172368us
I20260709 13:20:54.653096 15641 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:54.597556 21434 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:54.654772 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:54.657377 21369 consensus_queue.cc:237] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:54.662545 21407 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:54.664232 15596 raft_consensus.cc:2393] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 12: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:54.664109 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:54.668259 15521 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:20:54.671336 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:54.627434 21529 raft_consensus.cc:697] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:54.665727 21434 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:54.677264 15493 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:54.649030 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:54.691354 15524 raft_consensus.cc:2393] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 12: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:54.693148 15526 raft_consensus.cc:1275] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 13:20:54.697527 15526 pending_rounds.cc:85] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 8
I20260709 13:20:54.697793 15526 pending_rounds.cc:107] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 13:20:54.698041 15526 raft_consensus.cc:2889] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:54.698231 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:54.698947 15594 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:54.700140 15420 catalog_manager.cc:5697] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 10 to 13, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 13 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:54.674845 21620 raft_consensus.cc:697] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 23 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:54.715200 21620 consensus_queue.cc:237] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:54.719780 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:54.720577 15603 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
I20260709 13:20:54.723371 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:54.724133 15641 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:54.679098 21304 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:54.726536 21434 consensus_queue.cc:1048] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:54.726897 21529 consensus_queue.cc:237] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:54.728248 21434 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:54.729339 21304 raft_consensus.cc:697] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:54.730459 21304 consensus_queue.cc:237] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:54.740171 15568 leader_election.cc:400] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 17. Current term is 18.
I20260709 13:20:54.740634 15568 leader_election.cc:403] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:54.696252 21629 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:54.734755 21442 consensus_meta.cc:287] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.587s	user 0.008s	sys 0.002s
I20260709 13:20:54.755131 21407 consensus_queue.cc:1048] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:54.755894 15420 catalog_manager.cc:5697] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 8 to 12, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 12 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:20:54.760080 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:54.760449 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 1965) took 1013 ms. Trace:
I20260709 13:20:54.761509 21314 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.772607 15594 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 15.
I20260709 13:20:54.766247 21468 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:54.766538 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 89344.4us
I20260709 13:20:54.766556 21389 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:54.767982 21339 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 17. Current term is 18.
W20260709 13:20:54.769706 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:54.762894 21104 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:54.774724 21426 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:54.776651 15493 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:54.781572 15420 catalog_manager.cc:5697] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 10 to 18, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 18 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:54.771700 15527 rpcz_store.cc:276] 0709 13:20:53.747085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:53.747210 (+   125us) service_pool.cc:224] Handling call
0709 13:20:53.747641 (+   431us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:53.748137 (+   496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:53.748167 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:53.748885 (+   718us) log.cc:844] Serialized 52 byte log entry
0709 13:20:53.859610 (+110725us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:20:53.958057 (+ 98447us) log.cc:844] Serialized 14 byte log entry
0709 13:20:53.970093 (+ 12036us) log.cc:844] Serialized 14 byte log entry
0709 13:20:54.759677 (+789584us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b1800393ca0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1898c0a 00007f5ceafd8219 00007f5ce1920a40 00007f5ce191b4e4 00007f5ce191b455 00007f5ce4a03060 00007f5ce4a15716 00007f5ce4a26287 00007f5ce4a26208
0709 13:20:54.759893 (+   216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:54.759972 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:54.760100 (+   128us) raft_consensus.cc:1685] finished
0709 13:20:54.760115 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:54.760278 (+   163us) inbound_call.cc:177] Queueing success response
0709 13:20:54.764037 (+  3759us) spinlock_profiling.cc:243] Waited 6.9 ms on lock 0x7b1800393ca0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce191e557 00007f5ce19207c1 00007f5ce1920779 00007f5ce1920709 00007f5ce19206d1 00007f5ce191f9cd 000000000038f354 000000000038f199 00007f5ce190183a
Metrics: {"mutex_wait_us":41801,"prepare.queue_time_us":754464,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":58,"raft.queue_time_us":797976,"raft.run_cpu_time_us":369,"raft.run_wall_time_us":16962,"spinlock_wait_cycles":20383232,"thread_start_us":163622,"threads_started":2,"wal-append.queue_time_us":196752}
W20260709 13:20:54.781265 21513 consensus_meta.cc:287] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.701s	user 0.011s	sys 0.000s
W20260709 13:20:54.800416 15601 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.009s	sys 0.020s
W20260709 13:20:54.800904 15601 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.009s	sys 0.020s
I20260709 13:20:54.780321 21549 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:54.797282 21104 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:54.793093 21468 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:54.778577 21314 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:54.776479 21442 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:54.777985 21389 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:20:54.804852 21513 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:54.818679 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 210619us
I20260709 13:20:54.798043 21426 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:54.837813 21314 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:54.839094 21389 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.837672 21104 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:54.852038 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:54.849668 16205 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:54.857406 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:54.860056 16233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57244 (request call id 1778) took 1156 ms. Trace:
I20260709 13:20:54.873052 16233 rpcz_store.cc:276] 0709 13:20:53.703429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:53.704255 (+   826us) service_pool.cc:224] Handling call
0709 13:20:54.859866 (+1155611us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54006f13d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:54.859996 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":324952448}
W20260709 13:20:54.865053 21618 log.cc:927] Time spent T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.349s	user 0.004s	sys 0.006s
I20260709 13:20:54.818992 21632 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:54.873986 15600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34102 (request call id 1661) took 1148 ms. Trace:
I20260709 13:20:54.815887 21468 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.876156 15566 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:54.878358 16236 raft_consensus.cc:1240] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:54.883193 15600 rpcz_store.cc:276] 0709 13:20:53.725565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:53.778161 (+ 52596us) service_pool.cc:224] Handling call
0709 13:20:54.873798 (+1095637us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54000a8f54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:54.873906 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":262041472}
I20260709 13:20:54.882588 21334 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:54.887531 21334 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.815224 21549 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:54.893806 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 109350us
I20260709 13:20:54.899619 15567 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:54.907878 21339 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:54.908142 15567 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:54.888195 21389 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:54.909386 15567 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:54.885816 21468 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:54.890096 21334 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:54.920655 15641 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:54.931916 15603 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:20:54.936718 15641 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:54.916105 21320 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:54.918936 21466 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:54.945350 15596 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:20:54.924186 21407 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:54.935602 21104 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.899142 21632 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:54.938045 21435 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:54.942646 21633 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.957605 15420 catalog_manager.cc:5697] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 6 to 9, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 9 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:54.953987 21104 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:54.961113 21314 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.961608 21466 raft_consensus.cc:697] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:54.997507 15491 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:54.999327 16204 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:54.996352 15420 catalog_manager.cc:5697] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:20:55.030615 16204 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:55.035674 21620 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:55.036170 21620 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:55.036550 21620 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:55.031391 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:55.001125 21561 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.965938 21560 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:20:55.042394 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:55.018638 21529 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.044087 16239 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 17.
I20260709 13:20:55.048583 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:55.050302 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:55.051501 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 178063us
I20260709 13:20:55.051877 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:54.970517 21633 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:55.096340 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 196415us
I20260709 13:20:55.107401 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:55.108901 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:55.109467 15594 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:20:55.045348 21513 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:55.108544 21175 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.108050 15601 raft_consensus.cc:1275] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:20:55.018479 21314 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:55.136835 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:55.021178 21439 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:55.140400 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 142961us
W20260709 13:20:55.106150 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 158772us
I20260709 13:20:55.167464 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:55.168892 15491 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:55.150575 15641 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:55.095687 21561 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:55.053970 21529 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.140175 21320 raft_consensus.cc:697] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:55.184228 15672 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
I20260709 13:20:55.161806 21426 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:55.199339 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 168877us
I20260709 13:20:55.201454 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:55.201622 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:55.202291 16235 raft_consensus.cc:2368] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13.
I20260709 13:20:55.204056 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:55.177852 21549 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:55.207358 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 298023us
I20260709 13:20:55.206496 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:55.205711 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:55.209298 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:55.178807 21653 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:55.205400 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:55.176754 21116 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:55.160894 21175 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.188131 21389 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:55.193652 21560 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:20:55.214650 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 64138.9us
I20260709 13:20:55.216004 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:55.220534 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:55.216324 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:55.218178 21635 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:55.219022 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:55.209573 15567 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:55.226300 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:55.226722 16239 raft_consensus.cc:2393] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 23: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:55.213873 21116 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:55.233135 16235 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 16.
I20260709 13:20:55.213446 21426 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:55.245471 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:55.250069 16239 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:20:55.246726 21116 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:55.252501 16240 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:20:55.252542 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:55.226961 21103 raft_consensus.cc:2764] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:20:55.228721 21647 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.257349 16238 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 19.
W20260709 13:20:55.259112 21641 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:20:55.260320 21595 consensus_queue.cc:1048] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:55.231992 21654 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:55.260764 21659 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:55.249827 21632 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:55.264930 16236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57258 (request call id 1274) took 1333 ms. Trace:
I20260709 13:20:55.266150 16234 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:20:55.273674 21648 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:55.368070 21648 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:55.368498 21648 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:55.237507 21513 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:55.370999 21626 log.cc:927] Time spent T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.190s	user 0.002s	sys 0.000s
I20260709 13:20:55.370693 21513 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.371268 21453 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:55.371735 15642 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:20:55.279093 21636 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:55.372989 21636 raft_consensus.cc:697] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:55.283174 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:20:55.286412 21638 log.cc:927] Time spent T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
W20260709 13:20:55.287711 21631 log.cc:927] Time spent T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.397s	user 0.001s	sys 0.001s
I20260709 13:20:55.291038 21320 consensus_queue.cc:237] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.294659 21650 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.391497 21650 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.392305 21649 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:55.344187 21435 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:55.393651 21435 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:55.345973 21646 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:55.394974 21646 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.395025 21636 consensus_queue.cc:237] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.348747 21442 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:55.188566 21466 consensus_queue.cc:237] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:55.367502 21560 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:55.282061 21639 log.cc:927] Time spent T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
W20260709 13:20:55.403090 21634 log.cc:927] Time spent T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.487s	user 0.003s	sys 0.000s
I20260709 13:20:55.405153 21654 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:55.412201 21652 log.cc:927] Time spent T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:20:55.352917 21647 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.415844 15598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 2560) took 1182 ms. Trace:
I20260709 13:20:55.416270 15598 rpcz_store.cc:276] 0709 13:20:54.233579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:54.233758 (+   179us) service_pool.cc:224] Handling call
0709 13:20:54.255946 (+ 22188us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:54.256352 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:54.256382 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:54.935055 (+678673us) spinlock_profiling.cc:243] Waited 5.09 ms on lock 0x7b2c000fb608. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce3304a11 00007f5ce3314372 00007f5ce4a46a01 00007f5ce4a0b43a 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2
0709 13:20:54.942337 (+  7282us) log.cc:844] Serialized 26 byte log entry
0709 13:20:55.145405 (+203068us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:55.145438 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:55.145499 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:55.415510 (+270011us) raft_consensus.cc:1685] finished
0709 13:20:55.415534 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:55.415756 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":121,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":15468,"spinlock_wait_cycles":11704192,"thread_start_us":175853,"threads_started":1,"wal-append.queue_time_us":393618}
I20260709 13:20:55.383253 21659 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.414299 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57310 (request call id 2471) took 1127 ms. Trace:
I20260709 13:20:55.371855 21453 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:55.392891 21649 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.365403 16236 rpcz_store.cc:276] 0709 13:20:53.931088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:53.970509 (+ 39421us) service_pool.cc:224] Handling call
0709 13:20:53.970848 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:54.883667 (+912819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:55.264615 (+380948us) spinlock_profiling.cc:243] Waited 907 ms on lock 0x7b5400532854. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:55.264799 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2086648192}
I20260709 13:20:55.456001 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:55.456563 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:55.457929 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 86277.1us
I20260709 13:20:55.466442 15674 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:55.458529 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 233007us
I20260709 13:20:55.467610 15567 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:55.459436 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:55.476586 15568 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:55.418479 15528 rpcz_store.cc:276] 0709 13:20:54.287084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:54.290011 (+  2927us) service_pool.cc:224] Handling call
0709 13:20:54.290378 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:54.766193 (+475815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:54.766224 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:54.766759 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 13:20:54.915434 (+148675us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:54.915480 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:54.915555 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:55.413997 (+498442us) raft_consensus.cc:1685] finished
0709 13:20:55.414019 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:55.414196 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":475329,"raft.queue_time_us":475621,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":86,"thread_start_us":148253,"threads_started":1,"wal-append.queue_time_us":271866}
I20260709 13:20:55.484580 15567 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:55.489737 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:55.490655 16234 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:55.491446 21658 consensus_queue.cc:1048] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:55.491662 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:55.492321 16235 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 16.
I20260709 13:20:55.493533 21654 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:55.483232 21546 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:55.495533 21633 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:55.496567 21407 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:55.496997 21407 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:55.497346 21407 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:55.493745 21561 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:55.493413 21650 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:55.490578 21447 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:55.499051 21447 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:55.499426 21447 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:55.502666 21647 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:55.504848 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:55.505652 15670 raft_consensus.cc:2393] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 12: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:55.507490 21447 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.505582 15676 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:20:55.513746 21546 raft_consensus.cc:697] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:55.514866 21546 consensus_queue.cc:237] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:55.515638 21658 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:55.516193 21658 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.517030 21620 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.519958 21658 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:55.521126 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:55.509718 21447 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:55.522461 21447 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:55.522840 21447 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:55.523175 21447 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:55.398247 21560 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.531141 21175 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:55.531721 21560 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:55.531721 21314 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:55.532891 21104 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:55.538754 21513 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:55.522015 15671 raft_consensus.cc:2410] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 17 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:20:55.541041 21529 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:55.541304 21649 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:55.483505 21426 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.543655 21426 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.542784 21447 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:55.547472 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:55.551783 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:55.552492 15677 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 15.
I20260709 13:20:55.553581 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:55.561645 16203 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:55.545715 21426 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:55.567034 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 535684us
I20260709 13:20:55.563225 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:55.568207 15677 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:20:55.566219 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:55.566588 15598 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:20:55.569105 15668 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
I20260709 13:20:55.551038 21453 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:55.583103 15493 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:20:55.577339 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 408904us
I20260709 13:20:55.585150 16234 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 15.
I20260709 13:20:55.577586 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:55.570622 16203 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:55.587083 16203 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:55.580365 16205 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:55.595304 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:55.596046 16235 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:55.611625 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:55.613873 15677 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:55.614687 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:55.571576 21656 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:55.573164 21104 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.615545 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:55.603389 21658 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:55.575204 21659 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:55.574776 21447 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:55.573482 21439 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:55.603482 21660 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:20:55.624511 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 44260.1us
W20260709 13:20:55.623065 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 146026us
I20260709 13:20:55.626499 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:55.629295 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:55.639364 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:55.640101 15523 raft_consensus.cc:2393] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 14: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:55.630342 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:55.631045 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 146555us
I20260709 13:20:55.618775 21656 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:55.634625 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:55.635278 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:55.620222 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:55.644491 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:55.633484 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:55.645625 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:55.646119 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:55.643733 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:55.644138 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:55.603224 21637 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:20:55.648504 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:55.626929 21658 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:55.650566 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:55.660559 15525 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:55.646934 15521 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 15.
I20260709 13:20:55.651636 15567 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:55.652916 15568 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:55.626209 21529 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.654264 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:55.654992 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:55.662353 15522 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:55.630759 21646 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:55.627245 21369 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:55.656536 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:55.679651 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:55.657266 15528 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:55.621054 21663 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:55.659001 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:55.655768 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:55.684727 15524 raft_consensus.cc:2393] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 13: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:55.675922 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:55.678669 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 85088.9us
I20260709 13:20:55.612725 21648 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.680954 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:55.685732 15603 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:55.640875 21426 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:55.640686 21561 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:55.682192 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:55.639645 21660 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:55.649582 21656 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:55.663527 21658 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:55.709381 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:55.687835 15677 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:20:55.689199 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:55.687714 16240 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 19.
I20260709 13:20:55.693766 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:55.691727 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:55.679921 15600 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:20:55.632079 21104 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:55.714535 15528 raft_consensus.cc:2393] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 12: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:55.657768 21640 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 13:20:55.693740 21426 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.695360 21666 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:55.696000 21314 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:55.700420 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:55.748571 15524 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:20:55.675132 21389 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:55.702726 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:55.684239 21636 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.686177 21663 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:55.686733 21529 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.707269 21674 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:55.719314 15525 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:20:55.738811 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:55.707497 21675 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:20:55.720577 21334 consensus_meta.cc:287] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.786s	user 0.017s	sys 0.000s
I20260709 13:20:55.691949 21660 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:55.707494 21561 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:55.718194 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 56160.7us
I20260709 13:20:55.766608 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:55.771350 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:55.772423 15524 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:20:55.776773 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:55.753248 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 101690us
I20260709 13:20:55.788508 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:55.792402 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:55.793099 15520 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:55.797267 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:55.761895 21663 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:55.775307 21620 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:55.800966 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:55.772228 21674 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:55.814168 21674 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:55.815651 21407 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.816579 15674 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:20:55.797686 15568 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:55.805575 15567 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:55.754122 21314 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:55.800065 15493 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:55.812389 15524 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
I20260709 13:20:55.813287 15526 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:20:55.818058 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:55.831624 15493 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:55.832643 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:55.774652 21389 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.838376 21680 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:55.838861 21680 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:55.839236 21680 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:55.839752 21407 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 18 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:55.811599 21675 consensus_queue.cc:1048] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:20:55.841755 21104 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:55.801419 21681 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:20:55.783804 21334 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.759824 21666 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.846169 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:55.888609 15673 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:55.846372 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:55.840824 21682 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:20:55.899474 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 67921.9us
I20260709 13:20:55.779556 21636 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.849023 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:55.904572 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 85854.2us
I20260709 13:20:55.849151 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:55.851538 21468 consensus_meta.cc:287] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.920s	user 0.017s	sys 0.000s
W20260709 13:20:55.855515 21665 log.cc:927] Time spent T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.157s	user 0.000s	sys 0.002s
I20260709 13:20:55.904843 15669 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 19.
I20260709 13:20:55.856771 21674 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:55.866263 15668 raft_consensus.cc:1240] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:55.912015 21674 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:55.880072 21314 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:55.882614 21648 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:55.884274 21561 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:55.887199 21426 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:55.835858 21560 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.914084 21560 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.916375 21560 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:55.890421 16235 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 19.
I20260709 13:20:55.903666 15493 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
W20260709 13:20:55.856515 21679 log.cc:927] Time spent T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
W20260709 13:20:55.917455 21673 log.cc:927] Time spent T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.222s	user 0.002s	sys 0.000s
W20260709 13:20:55.923555 21116 consensus_meta.cc:287] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.651s	user 0.011s	sys 0.004s
I20260709 13:20:55.871481 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:55.924297 15521 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:55.923676 21636 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:55.837425 21649 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:55.929121 15597 raft_consensus.cc:1275] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:20:55.936383 16233 raft_consensus.cc:2393] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 17: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:55.925590 21175 consensus_queue.cc:1059] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:20:55.904384 21104 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:20:55.950400 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 46815.8us
I20260709 13:20:55.962462 15524 raft_consensus.cc:1275] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:20:55.980962 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 161523us
I20260709 13:20:55.954312 21104 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:55.926136 21468 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.985811 15493 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:55.961843 21649 raft_consensus.cc:697] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:55.987468 15641 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:55.934579 21116 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:55.997416 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:20:55.976593 21435 consensus_meta.cc:287] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.582s	user 0.011s	sys 0.007s
I20260709 13:20:55.971204 21636 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:55.971635 21560 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.974591 21689 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.018366 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:55.960963 21635 consensus_queue.cc:1048] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:20:56.022123 15641 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:55.989071 21104 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:55.994925 21339 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.988323 21314 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:56.047895 15641 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:56.022081 21620 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:55.984153 21561 consensus_queue.cc:1059] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:20:55.981050 21426 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
W20260709 13:20:56.070375 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 176121us
W20260709 13:20:56.002126 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 283669us
I20260709 13:20:56.015710 21619 raft_consensus.cc:3055] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:56.016984 21529 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:56.084847 15642 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:56.085822 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:56.083446 21434 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:56.066740 21339 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.099915 15521 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 8.
I20260709 13:20:56.101042 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:56.052996 21546 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.105952 15642 leader_election.cc:400] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13.
I20260709 13:20:56.056521 21304 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:56.092197 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:56.023598 21389 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:56.094051 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:56.107717 15567 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:56.094270 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:56.111697 15567 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:56.019306 21636 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:56.106303 15642 leader_election.cc:403] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:56.109298 15594 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:56.097600 21339 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:56.019644 21435 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:56.115077 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:56.119776 15601 raft_consensus.cc:2393] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 8: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:56.114192 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:56.023320 21560 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:56.025856 21689 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:56.132171 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:56.132889 15668 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:56.108794 21304 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:56.122257 15642 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:56.125452 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:56.088881 21619 raft_consensus.cc:740] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:56.054847 21680 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.070251 21314 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:56.142562 15493 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:56.117146 21636 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:56.127265 21633 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:56.148644 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:56.108321 21682 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:20:56.151845 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:56.149353 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:56.148334 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:56.123939 21334 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:56.156378 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 151741us
I20260709 13:20:56.163069 15598 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 18.
I20260709 13:20:56.192349 15669 raft_consensus.cc:1240] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:56.147284 21304 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:56.193812 15521 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
W20260709 13:20:56.195631 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 53166.2us
I20260709 13:20:56.119809 21658 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.113250 21546 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.200111 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:56.155723 21314 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:56.146575 21659 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13.
I20260709 13:20:56.201951 15601 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 23 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:20:56.117501 21649 consensus_queue.cc:237] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:56.147959 21666 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:56.234696 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:56.205725 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 119350us
I20260709 13:20:56.188174 21690 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:56.204538 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:56.213883 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:20:56.217343 21686 log.cc:927] Time spent T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:20:56.237617 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:56.197400 21692 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:56.238237 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:56.246142 16236 raft_consensus.cc:2393] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 14: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:56.203248 21447 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:56.237267 15641 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:56.240090 15594 raft_consensus.cc:3060] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:56.244256 16234 raft_consensus.cc:2393] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 15: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:56.245360 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:56.246609 16238 raft_consensus.cc:2393] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 18: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:56.242692 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:56.248955 16232 raft_consensus.cc:2393] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 19: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:20:56.250185 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 138574us
I20260709 13:20:56.210223 21633 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:56.251721 15567 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:56.252727 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:56.259341 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 106522us
I20260709 13:20:56.260919 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:56.259605 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:56.267717 15523 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
W20260709 13:20:56.276777 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 77943.9us
I20260709 13:20:56.277594 15493 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:56.244508 21334 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.284476 15491 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:56.285456 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:56.279347 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:56.279944 16231 raft_consensus.cc:2393] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 9: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:56.281523 15641 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:56.282382 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:56.239548 21690 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:56.293764 16233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57244 (request call id 1786) took 1265 ms. Trace:
I20260709 13:20:56.294342 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:56.294224 16233 rpcz_store.cc:276] 0709 13:20:55.028622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:55.038171 (+  9549us) service_pool.cc:224] Handling call
0709 13:20:56.293575 (+1255404us) spinlock_profiling.cc:243] Waited 879 ms on lock 0x7b5400640ad4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:56.293705 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2021421824}
I20260709 13:20:56.295131 15493 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:56.297170 15599 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:20:56.293133 21633 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:56.299645 21633 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:56.300000 21633 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:56.302260 21435 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:56.258903 21116 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:56.336426 15520 consensus_meta.cc:287] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.537s	user 0.013s	sys 0.000s
I20260709 13:20:56.336654 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:56.337826 15520 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18.
W20260709 13:20:56.338143 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 252372us
I20260709 13:20:56.338829 15526 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:20:56.213104 21619 consensus_queue.cc:260] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.342015 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:56.292956 21690 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:56.348289 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:56.349040 16236 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 23 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:20:56.349092 15677 raft_consensus.cc:2393] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 19: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:56.344496 15528 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:56.287516 21561 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.351738 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:56.352937 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:56.347308 21640 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.278004 21447 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.359022 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:56.360662 15594 raft_consensus.cc:2468] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 19.
I20260709 13:20:56.362079 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:56.361653 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:56.342466 21698 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:56.366847 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:20:56.368153 21705 log.cc:927] Time spent T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:20:56.370786 21658 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:56.372567 15668 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 17.
W20260709 13:20:56.373589 21660 consensus_meta.cc:287] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.579s	user 0.004s	sys 0.004s
W20260709 13:20:56.378954 21696 log.cc:927] Time spent T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:20:56.374802 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:56.355412 21104 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.378623 21660 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.289422 21334 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.380432 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:56.402602 15594 raft_consensus.cc:2393] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 18: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:56.375937 21659 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:56.400818 15595 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 16.
I20260709 13:20:56.407083 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:56.352489 21646 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.359767 21619 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:56.377004 21663 consensus_meta.cc:287] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.550s	user 0.008s	sys 0.000s
W20260709 13:20:56.397081 21697 log.cc:927] Time spent T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:20:56.362732 21529 consensus_queue.cc:1175] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3ef186ec12554a1d87814af0a0ace9a4" responder_term: 14 status { last_received { term: 13 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 13. Current term is 14. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:20:56.436982 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 152583us
I20260709 13:20:56.367677 21698 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:56.364022 21716 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:56.443964 15491 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:56.362630 21561 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.441313 21560 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:56.364789 21640 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.446604 21698 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:56.435127 21708 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:20:56.473816 21656 consensus_meta.cc:287] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.757s	user 0.011s	sys 0.000s
W20260709 13:20:56.513016 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 69143.1us
I20260709 13:20:56.513855 15491 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:56.476734 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 215871us
I20260709 13:20:56.516515 15642 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:56.517303 21704 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:56.483186 21710 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:56.419080 21659 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.474295 21546 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:56.470988 21691 log.cc:927] Time spent T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.321s	user 0.000s	sys 0.001s
I20260709 13:20:56.477723 21680 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:56.441874 21663 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:56.511953 21682 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:56.522387 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 231846us
I20260709 13:20:56.524417 16233 raft_consensus.cc:2393] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 17: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:56.526461 15491 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:56.528548 21389 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:56.437448 21646 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:56.503880 21697 log.cc:927] Time spent T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:20:56.552974 21633 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.558200 21334 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:56.555092 21434 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:20:56.552397 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:56.548136 21713 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:20:56.604199 21717 log.cc:927] Time spent T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7: Append to log took a long time: real 0.074s	user 0.003s	sys 0.000s
I20260709 13:20:56.545341 21546 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:56.538936 21703 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:56.548808 21711 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:56.508054 21689 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:20:56.563908 21104 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 20 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:56.559437 21640 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:56.523188 21716 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:56.585170 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 58781.8us
I20260709 13:20:56.615826 21709 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:20:56.616187 15491 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:56.610610 21711 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:56.616874 21711 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:56.617404 21709 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:56.586153 15641 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:56.592594 21646 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:56.594026 15525 raft_consensus.cc:1275] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:20:56.595678 21447 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:56.596081 15524 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
I20260709 13:20:56.600631 21659 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:56.608544 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:56.609786 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 314878us
I20260709 13:20:56.542796 21704 raft_consensus.cc:697] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:56.626060 15528 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 18.
I20260709 13:20:56.523808 21656 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.588992 21680 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:56.638101 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 52036.9us
I20260709 13:20:56.610322 21703 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:56.639338 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 122883us
I20260709 13:20:56.641265 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:56.634824 21699 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.644210 15528 raft_consensus.cc:2393] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 18: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:56.616391 21689 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:56.577221 21636 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.640501 15493 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:56.648706 15642 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:56.645531 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 203301us
I20260709 13:20:56.650000 15491 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:56.651589 15491 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:56.641052 16204 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:56.606372 21546 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.657979 16204 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:56.544708 21710 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.652942 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2480) took 1021 ms. Trace:
I20260709 13:20:56.650861 15493 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:56.656240 16239 raft_consensus.cc:2368] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 16. Current term is 17.
I20260709 13:20:56.655534 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:56.660131 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:56.660753 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:56.662153 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:56.662437 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:56.662755 15677 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:56.663437 15671 raft_consensus.cc:2368] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 14. Current term is 15.
I20260709 13:20:56.663317 15523 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
I20260709 13:20:56.664775 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:56.666339 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:56.666894 15671 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 17.
I20260709 13:20:56.668128 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:56.669019 21636 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:56.585640 21682 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.675185 15528 raft_consensus.cc:1275] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:20:56.645583 21703 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:56.673323 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:56.631379 21648 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:20:56.676857 16204 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:56.678133 21304 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.682250 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:56.678109 21646 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:56.693059 15677 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:20:56.678054 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:56.687187 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:56.659168 15522 rpcz_store.cc:276] 0709 13:20:55.631272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:55.705946 (+ 74674us) service_pool.cc:224] Handling call
0709 13:20:55.706246 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:55.706628 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:55.706655 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:55.707098 (+   443us) log.cc:844] Serialized 26 byte log entry
0709 13:20:55.983253 (+276155us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:55.983292 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:55.983358 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:56.652713 (+669355us) raft_consensus.cc:1685] finished
0709 13:20:56.652746 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:56.652880 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":332,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"thread_start_us":275798,"threads_started":1,"wal-append.queue_time_us":407295}
I20260709 13:20:56.689049 16204 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:56.689683 15493 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:56.674279 21635 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:56.703115 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:56.655282 21389 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:56.654672 21716 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:56.654911 21709 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:56.656373 21659 raft_consensus.cc:2804] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:56.717890 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:56.710909 15601 raft_consensus.cc:2393] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 13: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:56.714529 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:56.672953 21680 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:56.693799 21692 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.704003 21334 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:56.701328 21646 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:56.694171 21660 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:56.737125 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:56.719322 16204 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:56.707894 21713 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:56.701750 21104 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:56.713524 21389 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:56.658718 21689 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:56.719197 21709 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:56.707494 21619 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:56.752508 15642 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:56.651801 21699 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.753221 15601 raft_consensus.cc:1240] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:56.713122 21690 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:56.748653 15522 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:20:56.719851 21620 consensus_meta.cc:287] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.631s	user 0.006s	sys 0.005s
I20260709 13:20:56.730186 21680 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:56.770725 15601 raft_consensus.cc:1240] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13. Ops: []
W20260709 13:20:56.770336 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 106764us
I20260709 13:20:56.747917 21646 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:56.777045 15567 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:56.778455 15602 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 23.
I20260709 13:20:56.758493 21320 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:56.758814 21389 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:56.757768 21709 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:56.754650 21659 raft_consensus.cc:697] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:56.781211 15597 raft_consensus.cc:1240] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:56.759618 21513 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:20:56.782681 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 63694.8us
I20260709 13:20:56.757074 21334 consensus_queue.cc:1048] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:20:56.789798 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:56.751612 21692 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.795008 21738 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.043s
I20260709 13:20:56.796515 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:56.768553 21620 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:56.768208 21434 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:56.769984 21680 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:56.799387 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:56.799407 15567 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:56.805054 16204 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:56.805334 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:56.783613 21633 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:56.768352 21104 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.794920 21656 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:56.824267 15521 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:20:56.816587 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 64155.2us
I20260709 13:20:56.828127 15601 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:56.829494 15642 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:56.810161 21719 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:56.834616 15567 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:56.835114 16204 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:56.843647 21468 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:56.846621 21659 consensus_queue.cc:237] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.850175 15522 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:56.851083 21561 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:56.854609 21546 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:56.851347 15522 raft_consensus.cc:1240] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:56.857705 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:56.855674 21719 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:56.846522 21104 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:56.834843 21658 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:56.855700 21620 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:56.867841 16239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53976 (request call id 1251) took 1209 ms. Trace:
I20260709 13:20:56.883968 16239 rpcz_store.cc:276] 0709 13:20:55.658409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:55.658730 (+   321us) service_pool.cc:224] Handling call
0709 13:20:56.862877 (+1204147us) spinlock_profiling.cc:243] Waited 972 ms on lock 0x7b5400724a54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:56.867785 (+  4908us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2234639104}
I20260709 13:20:56.867055 21546 raft_consensus.cc:2764] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:20:56.891099 21666 raft_consensus.cc:3055] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:56.891592 21666 raft_consensus.cc:740] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:56.892454 21666 consensus_queue.cc:260] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.899101 15519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2481) took 1267 ms. Trace:
I20260709 13:20:56.899528 15519 rpcz_store.cc:276] 0709 13:20:55.631702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:55.708445 (+ 76743us) service_pool.cc:224] Handling call
0709 13:20:55.708709 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:55.843381 (+134672us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:20:56.021009 (+177628us) log.cc:844] Serialized 14 byte log entry
0709 13:20:56.242587 (+221578us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:56.242638 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:56.242680 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:56.898832 (+656152us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b54002de1d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:56.898892 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:56.899037 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":98051,"prepare.queue_time_us":252597,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":201710,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":43947,"spinlock_wait_cycles":319055360,"thread_start_us":320090,"threads_started":3,"wal-append.queue_time_us":606503}
I20260709 13:20:56.900444 21703 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:56.901417 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:56.903517 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:56.904296 15519 raft_consensus.cc:2393] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 14: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:56.905893 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:56.907692 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:56.863139 21513 consensus_queue.cc:1048] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:20:56.914131 21655 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:56.916088 21692 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:56.781255 21704 consensus_queue.cc:237] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:56.870082 21719 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:56.920037 21733 consensus_queue.cc:1048] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:20:56.870213 16234 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:56.870395 15521 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 15.
W20260709 13:20:56.878142 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 48601.3us
I20260709 13:20:56.923388 15566 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:56.923660 21439 consensus_queue.cc:1059] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:56.923524 15491 leader_election.cc:400] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 14. Current term is 15.
I20260709 13:20:56.924612 21648 raft_consensus.cc:3055] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:56.878680 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:56.882735 16237 raft_consensus.cc:1240] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:56.880239 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:56.924621 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:56.926827 16238 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 10.
I20260709 13:20:56.882445 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:56.924858 15491 leader_election.cc:403] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:20:56.928226 15597 raft_consensus.cc:1275] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:20:56.928846 15491 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:56.929268 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:56.930147 21636 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 14. Current term is 15.
W20260709 13:20:56.882151 21314 consensus_meta.cc:287] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.648s	user 0.007s	sys 0.003s
I20260709 13:20:56.878368 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:56.925401 21648 raft_consensus.cc:740] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:56.925446 21732 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:56.883066 21710 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:56.922518 21633 consensus_queue.cc:1059] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:56.930827 21314 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.927110 16239 raft_consensus.cc:2393] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 15: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:56.924823 15566 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:56.933085 15522 raft_consensus.cc:2393] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 15: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:56.934959 15525 raft_consensus.cc:1240] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:56.936031 15491 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:56.935089 15642 leader_election.cc:400] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 16. Current term is 17.
I20260709 13:20:56.936650 16235 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
I20260709 13:20:56.879923 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:56.939980 15642 leader_election.cc:403] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:56.940497 16231 raft_consensus.cc:2368] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 16. Current term is 17.
I20260709 13:20:56.942586 15491 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:56.893740 21666 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:56.930451 21561 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:56.951403 15642 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:56.925592 21699 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:56.959554 15642 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:56.932250 21689 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:56.947402 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 113198us
I20260709 13:20:56.969754 15642 leader_election.cc:400] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 16. Current term is 17.
I20260709 13:20:56.971244 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:56.971949 15671 raft_consensus.cc:2368] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 16. Current term is 17.
I20260709 13:20:56.951239 21636 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:56.944672 21104 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:56.922101 21546 consensus_queue.cc:1059] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:20:56.955487 21561 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:56.951918 21692 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:56.954542 21732 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:56.994195 21732 raft_consensus.cc:697] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:56.994947 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:56.995736 15671 raft_consensus.cc:2393] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 17: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:56.995188 21732 consensus_queue.cc:237] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:56.961345 21620 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:56.991452 21739 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:56.998530 21719 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:56.993301 21561 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:56.999818 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:57.000533 15668 raft_consensus.cc:2393] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 14: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:57.000775 21719 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:57.001346 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:57.002159 15673 raft_consensus.cc:2410] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:20:57.002960 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:57.004107 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:56.999154 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:56.968037 21439 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:57.006417 15674 raft_consensus.cc:2393] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 15: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:20:56.942173 21738 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.040s
I20260709 13:20:57.010819 21636 raft_consensus.cc:697] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:57.017570 21755 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:57.013268 21715 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:57.009111 21439 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:57.013688 21737 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:57.026335 21682 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:57.026647 21745 log.cc:927] Time spent T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 13:20:57.029497 21513 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:57.031730 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:57.068815 16231 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:56.931900 21648 consensus_queue.cc:260] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:56.980934 21711 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:57.052887 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:56.976718 21633 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 13:20:57.029376 21739 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:20:57.070181 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 50044.8us
I20260709 13:20:56.967386 21698 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:57.048589 16235 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:20:57.100482 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 127879us
I20260709 13:20:57.057823 21439 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:57.147856 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:57.164705 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:57.046185 21719 consensus_queue.cc:1048] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:20:57.066427 21715 raft_consensus.cc:697] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:57.182547 16240 raft_consensus.cc:2368] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 14. Current term is 15.
I20260709 13:20:57.183179 15594 raft_consensus.cc:1240] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:57.052991 21709 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:57.072202 21703 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:57.066320 21755 raft_consensus.cc:697] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:57.204509 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:57.189985 21675 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.130s	user 0.003s	sys 0.020s
I20260709 13:20:57.149389 21751 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:57.207018 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:57.073608 21513 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:57.165838 21648 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:57.167243 21738 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:57.208678 21715 consensus_queue.cc:237] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:57.207751 21675 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.151s	user 0.004s	sys 0.020s
I20260709 13:20:57.077212 21646 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:57.211683 21755 consensus_queue.cc:237] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:57.224550 16238 raft_consensus.cc:1240] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:57.221655 16237 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:57.167838 21739 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:57.128715 21561 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:57.221167 21703 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:57.228797 21314 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 20 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:57.245522 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:57.230815 21760 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:57.255048 21760 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:57.199079 21636 consensus_queue.cc:237] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:57.252599 21434 raft_consensus.cc:3055] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:57.233279 21738 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:57.251341 21659 raft_consensus.cc:3055] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:57.258996 21663 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:57.259202 21698 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:57.260462 21709 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:57.249675 21761 log.cc:927] Time spent T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:20:57.249029 16238 raft_consensus.cc:2368] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 16. Current term is 17.
I20260709 13:20:57.265745 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:57.313122 15521 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:57.246912 16203 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:57.290486 21659 raft_consensus.cc:740] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:57.245493 21680 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:57.258996 21711 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:57.440464 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 193627us
I20260709 13:20:57.275389 21434 raft_consensus.cc:740] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:57.247117 21703 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:57.458076 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:57.462013 15528 raft_consensus.cc:2393] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 19: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:57.222258 21751 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:57.448093 21700 log.cc:927] Time spent T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.792s	user 0.001s	sys 0.003s
I20260709 13:20:57.469082 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:57.473531 15521 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:20:57.478549 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:57.484247 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:57.486475 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:57.488152 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:57.489454 21703 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:57.492174 15675 raft_consensus.cc:1275] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:20:57.495070 21731 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:57.495682 21731 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:57.500129 21680 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:57.502867 21737 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:57.503684 21659 consensus_queue.cc:260] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:57.505059 21659 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:57.440434 21620 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 16. Current term is 17.
I20260709 13:20:57.449753 21663 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:57.458541 21646 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:57.462044 21334 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:20:57.517405 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 64843.7us
I20260709 13:20:57.476318 21658 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:57.476955 21698 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:57.523878 15641 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:57.520870 16203 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:57.525707 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:57.517200 21737 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:57.532517 21646 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:20:57.533422 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:57.534027 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:57.535207 21680 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:57.539201 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:57.540122 15595 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:57.540719 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:57.547600 15641 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:57.554402 15595 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:20:57.535540 15493 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:57.535665 21703 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:57.536237 15567 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:57.519613 21663 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:57.517699 21752 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:57.560220 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:57.531320 21620 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:57.563329 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:57.509032 16237 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 20.
I20260709 13:20:57.564940 15493 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:57.502894 21439 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:57.563964 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:57.525768 21334 raft_consensus.cc:697] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:57.576793 15598 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 15: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:57.562373 16203 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:57.624540 15596 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:20:57.628559 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 63709.2us
I20260709 13:20:57.545729 21739 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:57.741888 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:57.742506 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:57.740296 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 177942us
I20260709 13:20:57.523725 21658 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:57.573848 21769 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:57.757957 16203 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:57.759614 16203 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:57.748514 15519 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:20:57.526327 21698 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:57.623566 21339 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:57.576858 21765 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:57.625137 21513 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:57.546002 21737 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:57.522987 21434 consensus_queue.cc:260] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:57.665469 21711 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:57.552013 21680 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:57.753373 21561 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:57.761374 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:57.777074 16231 consensus_meta.cc:287] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.708s	user 0.017s	sys 0.005s
I20260709 13:20:57.778311 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:20:57.763547 21686 log.cc:927] Time spent T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 13:20:57.792431 16239 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.053s	sys 0.061s
I20260709 13:20:57.751824 21779 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:57.792773 16239 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.053s	sys 0.061s
I20260709 13:20:57.791471 16231 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:20:57.797025 15519 raft_consensus.cc:1240] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:57.774682 21769 raft_consensus.cc:697] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
W20260709 13:20:57.785601 21666 consensus_meta.cc:287] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.830s	user 0.007s	sys 0.011s
I20260709 13:20:57.741689 21703 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:57.781328 21619 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:57.785208 21513 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:57.786950 21339 raft_consensus.cc:697] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:57.800808 21568 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:57.786656 21764 consensus_queue.cc:1059] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:20:57.820060 15526 raft_consensus.cc:1240] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:20:57.837230 15601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 2583) took 1129 ms. Trace:
I20260709 13:20:57.842195 15601 rpcz_store.cc:276] 0709 13:20:56.707651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:56.774300 (+ 66649us) service_pool.cc:224] Handling call
0709 13:20:56.774591 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:56.830302 (+ 55711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:57.836932 (+1006630us) spinlock_profiling.cc:243] Waited 53 ms on lock 0x7b540025b554. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:57.837159 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":121835904}
I20260709 13:20:57.781435 21765 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:57.844512 21751 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:57.742664 21752 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:57.879843 15596 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 19.
I20260709 13:20:57.880563 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2484) took 2219 ms. Trace:
I20260709 13:20:57.808153 21738 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:57.884054 15677 raft_consensus.cc:1240] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:57.885772 15526 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:20:57.882757 15527 rpcz_store.cc:276] 0709 13:20:55.632461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:55.740097 (+107636us) service_pool.cc:224] Handling call
0709 13:20:55.758300 (+ 18203us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:55.770058 (+ 11758us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:20:55.903022 (+132964us) log.cc:844] Serialized 14 byte log entry
0709 13:20:56.149898 (+246876us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:56.150459 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 13:20:56.150603 (+   144us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:56.187758 (+ 37155us) log.cc:844] Serialized 14 byte log entry
0709 13:20:56.196067 (+  8309us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:56.196148 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:57.829108 (+1632960us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b2400130f28. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce17a49c4 00007f5ce17a2acb 00007f5ce195ddc9 00007f5ce195da55 00007f5ce195cfb0 00007f5ce45d3488 00007f5ce45d3be8 00007f5ce45ab398 00007f5ce45a676a 00007f5ce45a5bb6
0709 13:20:57.851148 (+ 22040us) raft_consensus.cc:1685] finished
0709 13:20:57.851172 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:57.851340 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":140472,"prepare.queue_time_us":285763,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":2866,"raft.queue_time_us":394141,"raft.run_cpu_time_us":566,"raft.run_wall_time_us":47144,"spinlock_wait_cycles":26992384,"thread_start_us":319020,"threads_started":3,"wal-append.queue_time_us":308556}
I20260709 13:20:57.803916 21779 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:57.891433 15566 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:57.817065 21334 consensus_queue.cc:237] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:57.901602 15675 raft_consensus.cc:1275] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:20:57.887322 21754 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.019s	sys 0.038s
W20260709 13:20:57.914651 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 155098us
I20260709 13:20:57.896274 21742 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:57.919878 16203 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:57.922955 21751 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:57.923640 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:57.939374 21658 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:20:57.939951 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:57.907217 21339 consensus_queue.cc:237] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:57.897583 21769 consensus_queue.cc:237] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:57.940379 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 177779us
W20260709 13:20:57.913902 21754 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.020s	sys 0.039s
I20260709 13:20:57.928979 21439 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:57.964982 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:57.943970 21698 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:57.960474 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:57.962146 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:57.942047 21726 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.953260 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:57.959347 15493 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:57.974188 16233 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
W20260709 13:20:57.986461 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:57.952435 21756 log.cc:927] Time spent T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.748s	user 0.000s	sys 0.002s
I20260709 13:20:57.987988 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:57.988664 16236 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:20:57.989874 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:57.990926 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:57.991204 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:57.991693 16236 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:57.992057 15493 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:57.943564 21619 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:57.994449 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:57.991866 16240 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 23.
I20260709 13:20:57.997447 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:57.958114 21737 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:57.997818 15493 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:57.990947 21777 log.cc:927] Time spent T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:20:58.004114 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:58.009639 16234 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:58.004773 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:58.005182 15493 leader_election.cc:400] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 14. Current term is 15.
I20260709 13:20:58.011072 16237 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 10.
W20260709 13:20:58.005626 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 204999us
I20260709 13:20:58.006183 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:57.982225 21680 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:57.981065 21709 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:57.946292 21751 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:20:58.013437 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:57.987651 21666 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:58.023470 21789 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:20:58.024406 21774 log.cc:927] Time spent T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:20:58.025185 21776 log.cc:927] Time spent T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:20:58.025856 21783 log.cc:927] Time spent T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:20:58.026113 21787 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:20:58.026649 21782 log.cc:927] Time spent T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:20:58.000849 21703 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:58.030369 21663 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:58.053957 15567 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:20:58.055828 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 133979us
I20260709 13:20:57.990146 21726 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:57.998793 21561 consensus_queue.cc:1059] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:20:58.044092 21680 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:57.998684 21711 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:20:58.065260 16236 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 18.
I20260709 13:20:58.061723 21737 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:58.091709 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 192508us
I20260709 13:20:58.085073 16234 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 20.
W20260709 13:20:58.082264 21648 consensus_meta.cc:287] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.833s	user 0.014s	sys 0.005s
I20260709 13:20:58.107075 15567 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:20:58.107769 21659 consensus_meta.cc:287] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.602s	user 0.011s	sys 0.000s
I20260709 13:20:58.082212 21765 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:58.117446 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 142371us
I20260709 13:20:58.099732 21698 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:20:58.123275 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 110063us
I20260709 13:20:58.090476 21752 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:58.128288 16204 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:58.129849 15493 leader_election.cc:400] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 16. Current term is 17.
I20260709 13:20:58.130389 15493 leader_election.cc:403] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:20:58.131448 16239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57258 (request call id 1294) took 1077 ms. Trace:
I20260709 13:20:58.133796 15493 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:58.138325 15668 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:58.099123 15568 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:58.138486 15599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 2574) took 1939 ms. Trace:
I20260709 13:20:58.139492 15599 rpcz_store.cc:276] 0709 13:20:56.199138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:56.200194 (+  1056us) service_pool.cc:224] Handling call
0709 13:20:56.290909 (+ 90715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:56.297757 (+  6848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:58.135300 (+1837543us) spinlock_profiling.cc:243] Waited 78.6 ms on lock 0x7b540030d250. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:58.138418 (+  3118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":180772608}
I20260709 13:20:58.143224 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:58.123517 21698 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:58.133935 16239 rpcz_store.cc:276] 0709 13:20:57.054255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:57.054378 (+   123us) service_pool.cc:224] Handling call
0709 13:20:57.054748 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:57.147214 (+ 92466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:57.147248 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:57.187289 (+ 40041us) log.cc:844] Serialized 26 byte log entry
0709 13:20:57.795746 (+608457us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:57.795797 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:57.795861 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:58.131202 (+335341us) raft_consensus.cc:1685] finished
0709 13:20:58.131225 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:58.131377 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41544,"raft.queue_time_us":41742,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":50390,"thread_start_us":575312,"threads_started":1,"wal-append.queue_time_us":710400}
I20260709 13:20:58.081691 21787 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:58.148984 21389 consensus_meta.cc:287] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 1.343s	user 0.011s	sys 0.011s
I20260709 13:20:58.154824 21389 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:58.156051 21787 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:58.156700 21787 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:58.157025 21787 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:58.157012 21389 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:58.157636 21787 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:58.133062 21786 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:58.147475 21703 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:58.109253 21737 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.145709 15493 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:58.159216 15521 raft_consensus.cc:2393] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 13: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:58.137632 21680 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:20:58.184372 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 50015.4us
W20260709 13:20:58.188786 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 89768.7us
I20260709 13:20:58.187539 15493 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:58.222543 21466 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:58.234684 21650 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:58.242930 21650 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.245229 21650 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:58.235136 21339 consensus_queue.cc:1059] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:20:58.250068 21690 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:58.251473 21798 raft_consensus.cc:2804] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:58.252043 21798 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:58.252471 21798 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:58.159220 21698 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:58.209409 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 61488.9us
I20260709 13:20:58.144832 21794 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:58.268590 21466 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 13:20:58.278595 21435 raft_consensus.cc:3055] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:58.279073 21435 raft_consensus.cc:740] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:58.280016 21435 consensus_queue.cc:260] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:58.281304 21435 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:58.158397 21786 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.284231 21786 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:58.285043 15568 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:58.286039 21796 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:58.286435 21796 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:58.286540 15568 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:58.286820 21796 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:58.287731 15568 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:58.289611 21690 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:20:58.190423 21703 raft_consensus.cc:697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:58.290009 21690 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:58.290418 21690 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:58.290496 21751 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:58.187321 21790 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:58.294472 21790 raft_consensus.cc:697] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:58.295459 21790 consensus_queue.cc:237] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.301100 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:58.302872 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:58.303629 15672 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:20:58.308079 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:58.222779 21715 consensus_queue.cc:1059] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:20:58.310691 21792 raft_consensus.cc:3055] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:58.311251 21792 raft_consensus.cc:740] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:58.311775 21434 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:58.312129 21792 consensus_queue.cc:260] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:58.315327 15528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 2492) took 1608 ms. Trace:
I20260709 13:20:58.315805 15528 rpcz_store.cc:276] 0709 13:20:56.707055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:56.714229 (+  7174us) service_pool.cc:224] Handling call
0709 13:20:58.315049 (+1600820us) spinlock_profiling.cc:243] Waited 740 ms on lock 0x7b540029bdd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:58.315253 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1701341440}
I20260709 13:20:58.316949 15522 raft_consensus.cc:1240] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:20:58.307348 16203 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:58.159992 21752 consensus_queue.cc:1048] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.110s
W20260709 13:20:58.323448 15524 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57300 (request call id 2491) took 2181 ms (client timeout 2092 ms). Trace:
W20260709 13:20:58.328244 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 50630us
W20260709 13:20:58.326121 15524 rpcz_store.cc:269] 0709 13:20:56.141982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:56.343821 (+201839us) service_pool.cc:224] Handling call
0709 13:20:58.323271 (+1979450us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b5400187754. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:58.323382 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":340778752}
I20260709 13:20:58.332849 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:58.333627 15673 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 19.
I20260709 13:20:58.336942 15596 raft_consensus.cc:1240] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:58.339308 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:58.340989 15597 raft_consensus.cc:1240] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:20:58.341024 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:58.343815 15675 raft_consensus.cc:1240] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:58.344699 16239 raft_consensus.cc:1240] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:58.344882 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:58.362833 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:58.331099 21513 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:58.270889 21304 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:58.364727 16236 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:20:58.335489 21633 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.385520 16203 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:58.369267 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:58.330983 21786 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:20:58.372746 21802 log.cc:927] Time spent T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:20:58.373616 21801 log.cc:927] Time spent T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 13:20:58.379339 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:20:58.378890 21793 log.cc:927] Time spent T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:20:58.290542 21407 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:58.386754 15673 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:20:58.366633 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 129063us
I20260709 13:20:58.394256 21786 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:58.394212 16237 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:58.399893 15603 raft_consensus.cc:1240] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
W20260709 13:20:58.401367 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.402524 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:58.323040 21692 consensus_queue.cc:1048] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:20:58.403187 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:58.406126 15676 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:58.403298 15672 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 15.
I20260709 13:20:58.405140 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:58.407243 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:58.407877 16240 raft_consensus.cc:2393] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 17: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:58.408401 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:58.408985 15672 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:58.419312 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.382273 21726 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:58.341990 21675 consensus_queue.cc:1059] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:20:58.429126 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:58.429901 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:58.436038 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:58.437282 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:58.438741 15668 raft_consensus.cc:1240] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11. Ops: []
W20260709 13:20:58.431732 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 244342us
I20260709 13:20:58.440294 15493 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:58.441776 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 56349.6us
I20260709 13:20:58.402501 21786 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:58.401010 21689 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:58.449349 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:58.455130 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.455502 15597 raft_consensus.cc:1275] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:20:58.428450 21389 raft_consensus.cc:3055] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:58.456540 16237 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 11.
I20260709 13:20:58.445245 21175 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.441705 21718 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:58.462872 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:58.464109 15602 raft_consensus.cc:1275] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:20:58.404148 21633 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:58.467819 15603 raft_consensus.cc:1240] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:58.465013 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:58.466207 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:58.403996 21513 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.468633 15524 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:20:58.470458 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:58.470659 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:58.434633 21765 consensus_queue.cc:1059] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.035s
I20260709 13:20:58.460083 21598 raft_consensus.cc:2764] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:20:58.460325 21468 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:58.479879 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:58.480356 21468 raft_consensus.cc:697] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:58.474838 21304 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:20:58.464118 21689 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.483039 21304 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.473946 15519 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:20:58.477545 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:58.459478 21389 raft_consensus.cc:740] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:58.488024 15675 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:58.454989 21769 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.020s
I20260709 13:20:58.481318 21468 consensus_queue.cc:237] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.497056 15567 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:58.497746 16240 raft_consensus.cc:1240] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:20:58.473896 15597 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 19.
I20260709 13:20:58.499563 16232 raft_consensus.cc:1240] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:20:58.469758 21718 raft_consensus.cc:697] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:58.464766 21175 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.507843 15567 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:58.508498 15673 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 20.
I20260709 13:20:58.504530 21682 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:58.493003 21680 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 16. Current term is 17.
I20260709 13:20:58.456575 21703 consensus_queue.cc:237] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.518801 15676 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 15.
I20260709 13:20:58.512790 15672 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 18.
I20260709 13:20:58.512943 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:58.523136 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:58.523351 21680 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:58.523888 21680 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.479086 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:58.526445 15524 raft_consensus.cc:1275] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:20:58.520988 21175 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:58.529212 21680 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:58.534955 15567 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:58.535944 21785 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:58.536370 21785 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:58.536691 21785 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:58.536701 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:58.539563 15523 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56840 (request call id 1988) took 2342 ms (client timeout 1958 ms). Trace:
W20260709 13:20:58.539986 15523 rpcz_store.cc:269] 0709 13:20:56.197524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:56.335999 (+138475us) service_pool.cc:224] Handling call
0709 13:20:58.539371 (+2203372us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b54003d9ad4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:58.539475 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":748078464}
I20260709 13:20:58.540985 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:58.543273 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:58.543612 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:58.543933 15674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 3003) took 1079 ms. Trace:
I20260709 13:20:58.544359 15674 rpcz_store.cc:276] 0709 13:20:57.465070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:57.467206 (+  2136us) service_pool.cc:224] Handling call
0709 13:20:57.490602 (+ 23396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:57.507881 (+ 17279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:57.507911 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:57.507949 (+    38us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:57.507978 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:58.543724 (+1035746us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b54001540d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:58.543767 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:58.543880 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39241600}
I20260709 13:20:58.545117 15523 raft_consensus.cc:1240] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:58.545284 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:58.546382 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:58.527983 15675 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 12.
I20260709 13:20:58.546516 21659 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:58.538182 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:58.547523 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:58.549337 21708 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:58.552654 21785 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:58.554113 21648 consensus_queue.cc:1059] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:58.554991 21439 raft_consensus.cc:3055] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:58.555369 21439 raft_consensus.cc:740] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
W20260709 13:20:58.556380 21780 log.cc:927] Time spent T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.493s	user 0.000s	sys 0.001s
I20260709 13:20:58.556217 21439 consensus_queue.cc:260] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:58.557426 21439 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:58.558315 21792 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:58.562539 21731 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:58.565510 21810 raft_consensus.cc:3055] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:58.565991 21810 raft_consensus.cc:740] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:58.566802 21810 consensus_queue.cc:260] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:58.568109 15670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 3005) took 1102 ms. Trace:
I20260709 13:20:58.496441 21787 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.569234 21304 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:58.571098 21787 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:58.571316 21803 log.cc:927] Time spent T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 13:20:58.553265 21779 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:20:58.573770 21807 log.cc:927] Time spent T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:20:58.584985 21796 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.585536 16232 raft_consensus.cc:1240] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:58.586413 15526 raft_consensus.cc:1240] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:58.588008 15525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2494) took 1872 ms. Trace:
I20260709 13:20:58.588380 15525 rpcz_store.cc:276] 0709 13:20:56.715895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:56.822555 (+106660us) service_pool.cc:224] Handling call
0709 13:20:56.822846 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:56.940289 (+117443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:58.587771 (+1647482us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54003cdcd4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:20:58.587946 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":257027456}
I20260709 13:20:58.589072 21731 consensus_queue.cc:1048] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:20:58.589351 21648 consensus_queue.cc:1048] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:58.509810 21561 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:58.590797 21561 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.591214 21731 consensus_queue.cc:1059] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:58.592753 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:58.594426 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:58.520872 21389 consensus_queue.cc:260] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.595568 21389 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:58.596146 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:58.598984 21513 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:58.600189 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:58.527871 21689 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:58.514814 21726 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:58.601562 21726 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:58.602020 21726 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:58.603292 15568 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:58.604176 21682 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:58.605681 15568 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:58.606096 21760 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:58.607750 21439 raft_consensus.cc:3055] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:58.513384 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:58.610423 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:58.621863 15677 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:20:58.610992 15568 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:58.613829 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:58.623241 15520 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:20:58.615803 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.622602 15519 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 8.
I20260709 13:20:58.654492 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:58.657634 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:58.624548 15568 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:58.568522 15670 rpcz_store.cc:276] 0709 13:20:57.465687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:57.465794 (+   107us) service_pool.cc:224] Handling call
0709 13:20:57.466148 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:57.474998 (+  8850us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:57.475031 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:57.483896 (+  8865us) log.cc:844] Serialized 26 byte log entry
0709 13:20:57.887053 (+403157us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:57.887096 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:57.887163 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:58.555802 (+668639us) spinlock_profiling.cc:243] Waited 24.9 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:20:58.567808 (+ 12006us) raft_consensus.cc:1685] finished
0709 13:20:58.567830 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:58.568049 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8402,"raft.queue_time_us":8542,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"spinlock_wait_cycles":57354624,"thread_start_us":402736,"threads_started":1,"wal-append.queue_time_us":512864}
I20260709 13:20:58.509897 21737 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:58.592998 21561 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:58.680420 15603 raft_consensus.cc:1240] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:20:58.683429 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:58.654209 21765 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:20:58.685174 15494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.621352 21699 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:58.689177 15567 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:58.692772 15520 raft_consensus.cc:2468] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:20:58.656164 21513 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:20:58.655019 21682 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:58.621291 21654 consensus_queue.cc:1059] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 13:20:58.696206 21715 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:58.655269 21789 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.608335 21690 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.707468 21304 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.682572 21674 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:58.722478 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:58.722493 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:58.621722 21439 raft_consensus.cc:740] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:58.725013 21654 raft_consensus.cc:3055] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:58.728051 21786 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.696311 21718 consensus_queue.cc:237] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.709198 21798 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.684775 21765 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:58.685077 21742 consensus_queue.cc:1059] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:20:58.734606 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:20:58.742412 21788 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.001s	sys 0.007s
I20260709 13:20:58.743697 15599 raft_consensus.cc:2468] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
I20260709 13:20:58.752019 15597 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 24 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 23. Current term is 24. Ops: []
I20260709 13:20:58.733110 21633 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:58.761289 16234 raft_consensus.cc:1240] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:58.764472 21796 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:58.754937 21788 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.001s	sys 0.008s
I20260709 13:20:58.768340 15527 raft_consensus.cc:1240] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:58.746798 21654 raft_consensus.cc:740] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:58.753053 21765 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:20:58.784186 21806 log.cc:927] Time spent T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.356s	user 0.000s	sys 0.003s
I20260709 13:20:58.766487 21742 consensus_queue.cc:1243] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer dd1a14c3173d49dc9b55bb116e3783c9 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:20:58.786379 21654 consensus_queue.cc:260] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.786514 21439 consensus_queue.cc:260] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:58.750936 21698 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.800534 21698 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:58.802764 21798 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:58.787672 21654 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:58.795259 21690 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 24 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:58.808792 15491 leader_election.cc:400] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 election: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 16. Current term is 17.
I20260709 13:20:58.809906 21787 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.810437 21787 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.743335 21304 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.814589 15491 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:58.816988 15491 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:58.735136 21789 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.819859 15491 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:58.827441 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 219331us
I20260709 13:20:58.827759 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:58.828402 16240 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:58.830833 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.831192 15491 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:20:58.833621 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:58.820078 21561 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:58.820135 21636 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:58.838492 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:58.833878 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:58.845523 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:58.845716 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:58.847841 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:58.848539 15599 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:20:58.848189 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:58.850497 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:58.834749 21680 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:58.851958 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:58.852432 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:58.844846 21721 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:20:58.858002 21824 log.cc:927] Time spent T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:20:58.851612 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:58.835402 21786 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:58.840989 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:58.865952 15525 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 17.
I20260709 13:20:58.820973 21439 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:58.842541 21561 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:58.868342 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:58.870900 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:58.860226 16205 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:58.840070 15493 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:58.872785 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:58.873639 15600 raft_consensus.cc:2410] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:20:58.873435 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:58.875407 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:58.875702 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:58.860716 21721 raft_consensus.cc:697] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:58.877403 21721 consensus_queue.cc:237] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:58.881321 15596 raft_consensus.cc:1240] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 18. Current term is 19. Ops: [16.10-18.11]
I20260709 13:20:58.862911 21726 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.888125 21726 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:58.888871 15601 raft_consensus.cc:1240] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:58.894902 21304 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:58.895596 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:58.896380 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:58.897118 15599 raft_consensus.cc:2393] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 13: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:58.896384 15603 raft_consensus.cc:2393] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 14: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:58.898473 15597 raft_consensus.cc:3055] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:58.899087 15597 raft_consensus.cc:740] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:58.901578 16205 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:58.897585 21787 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:20:58.906906 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 66921.2us
I20260709 13:20:58.908918 15493 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:58.900575 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:58.886174 21704 raft_consensus.cc:3055] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:58.899863 15597 consensus_queue.cc:260] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.918313 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:58.919199 15597 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:58.869685 21561 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:58.919548 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:58.920780 15493 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:58.921814 16205 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:58.922010 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:58.923389 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:58.924120 15596 raft_consensus.cc:2393] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 16: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:58.924219 15566 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:20:58.935709 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 95034.5us
I20260709 13:20:58.937031 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:58.938160 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:58.939931 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:58.936466 15641 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:58.942765 15677 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:20:58.943856 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:58.904556 21788 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:58.945705 15670 raft_consensus.cc:2393] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 13: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:58.949529 15519 raft_consensus.cc:1240] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:58.952898 15528 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:58.954197 21680 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:58.954757 21680 raft_consensus.cc:697] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:58.965622 21680 consensus_queue.cc:237] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:58.972429 21765 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.974113 15602 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 18.
I20260709 13:20:58.974637 21765 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 20 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:58.956406 21658 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:20:58.975306 15525 raft_consensus.cc:2393] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 12: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:58.975435 21658 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:58.975903 21658 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:20:58.976044 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 51918.8us
I20260709 13:20:58.977029 21407 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:58.977572 21407 raft_consensus.cc:697] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:58.978471 21407 consensus_queue.cc:237] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:58.944428 21711 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:58.957489 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:58.918725 21704 raft_consensus.cc:740] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:58.946552 21304 consensus_queue.cc:1048] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:20:58.987277 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:58.981879 21711 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:58.989594 21711 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:58.990700 16240 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:20:59.039541 16232 raft_consensus.cc:1240] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:58.928323 21787 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:58.992354 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:59.040760 16231 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 15.
I20260709 13:20:58.994474 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:59.041785 15596 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:20:59.001479 15597 raft_consensus.cc:1275] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 13:20:59.002118 21742 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.002400 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:59.006906 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:59.007735 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:59.010456 15675 raft_consensus.cc:1240] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:20:59.044036 15597 pending_rounds.cc:85] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting all ops after (but not including) 9
I20260709 13:20:59.011119 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:58.986274 21827 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:59.045135 21827 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.046041 21769 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:59.046347 21820 log.cc:927] Time spent T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.215s	user 0.003s	sys 0.000s
I20260709 13:20:59.012593 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:59.010985 21561 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:59.048375 21827 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:59.019260 21785 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:59.023479 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:59.006910 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 2495) took 1266 ms. Trace:
I20260709 13:20:59.025075 21339 consensus_queue.cc:1059] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:59.027179 15677 raft_consensus.cc:1240] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:59.050246 15521 rpcz_store.cc:276] 0709 13:20:57.740714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:57.741205 (+   491us) service_pool.cc:224] Handling call
0709 13:20:59.006719 (+1265514us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b54002de1d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:20:59.006851 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":945118592}
I20260709 13:20:59.028594 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:59.030807 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 92759.1us
I20260709 13:20:59.056273 15527 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:20:59.056859 15642 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:59.031309 21731 consensus_queue.cc:1059] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:59.032992 21789 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:59.034348 15523 raft_consensus.cc:1275] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:20:59.037014 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 100635us
I20260709 13:20:59.059155 15641 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:59.044394 15597 pending_rounds.cc:107] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:20:59.048504 15601 raft_consensus.cc:2393] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 13: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:58.911556 21513 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:58.983424 21704 consensus_queue.cc:260] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.051448 15493 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:59.052165 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:59.053468 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:58.920430 21798 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:59.053815 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:59.062654 16204 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:59.062654 16238 raft_consensus.cc:2393] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 15: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:59.067420 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:59.084183 15493 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:20:59.087364 15567 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.952s (SENT) [suppressed 5 similar messages]
I20260709 13:20:59.060369 15597 raft_consensus.cc:2889] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:59.089303 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:59.091614 15520 raft_consensus.cc:1275] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:20:59.055414 21389 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:59.104585 15525 raft_consensus.cc:1240] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:59.104261 16233 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:20:59.105281 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 50187.1us
W20260709 13:20:59.105520 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 46440.6us
W20260709 13:20:59.198606 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 90585.7us
W20260709 13:20:59.114378 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 62747.6us
I20260709 13:20:59.126205 15642 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:59.127161 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:59.202651 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:59.106626 15493 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:59.111706 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:20:59.204452 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 92120.8us
I20260709 13:20:59.048849 21742 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.206588 15641 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:59.207171 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 144599us
W20260709 13:20:59.207967 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 81887.5us
I20260709 13:20:59.209811 15642 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:59.201022 16205 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:59.203905 15601 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 16.
I20260709 13:20:59.215315 21648 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:59.085798 21789 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:59.215385 21175 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:59.216719 21175 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:59.217056 21175 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:59.217139 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:59.217748 15602 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:59.089744 21827 raft_consensus.cc:2804] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:59.218668 21827 raft_consensus.cc:697] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:59.219017 21561 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:59.106695 21798 raft_consensus.cc:697] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:59.219614 21827 consensus_queue.cc:237] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:59.220345 21798 consensus_queue.cc:237] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:59.108314 21704 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:59.104230 21711 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:59.230939 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:59.231773 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:59.232002 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:59.232533 16234 raft_consensus.cc:1240] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:59.199931 21769 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:20:59.235548 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 128996us
I20260709 13:20:59.236140 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:20:59.227736 21755 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:59.241676 16205 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:59.241987 21389 raft_consensus.cc:2804] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:59.245851 21389 raft_consensus.cc:697] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:59.246807 21389 consensus_queue.cc:237] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.244663 15669 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:20:59.238353 16231 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:59.245687 21764 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:59.250259 21788 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:59.247253 21810 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:59.241897 21334 consensus_queue.cc:1059] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.011s
I20260709 13:20:59.252713 21334 raft_consensus.cc:3055] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:59.253082 21334 raft_consensus.cc:740] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:59.239228 15493 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:59.253957 21334 consensus_queue.cc:260] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.253693 16205 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:59.255430 21334 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:59.257058 21788 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:59.258138 21561 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:59.258625 21561 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:59.258998 21561 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:59.261768 16205 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:59.262665 15493 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:59.226521 21825 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:59.227870 21659 raft_consensus.cc:2804] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:59.237811 15642 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:59.272544 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:59.273392 15600 raft_consensus.cc:2393] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 15: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:20:59.198210 21529 consensus_queue.cc:1059] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:20:59.230967 21711 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:59.266821 15641 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:59.206168 21731 consensus_queue.cc:1059] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:20:59.226356 21765 consensus_queue.cc:1059] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:20:59.274739 21654 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:59.104820 21304 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.161s
I20260709 13:20:59.275184 16231 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:20:59.242136 21655 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:59.279896 16205 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:59.237264 21779 consensus_queue.cc:1048] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:20:59.243618 21703 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:20:59.266330 21755 raft_consensus.cc:697] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
W20260709 13:20:59.304091 21468 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.812s	user 0.013s	sys 0.032s
I20260709 13:20:59.276270 21659 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:59.303851 21711 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:20:59.342361 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 104647us
W20260709 13:20:59.334887 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 247532us
I20260709 13:20:59.399489 15567 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:20:59.275560 21825 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:59.335749 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 55945.6us
W20260709 13:20:59.342167 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 75457.8us
I20260709 13:20:59.399144 15642 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:59.402952 16237 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:20:59.349206 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.303927 21788 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:59.406279 15526 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:59.408241 15642 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:59.398043 21468 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.906s	user 0.014s	sys 0.040s
I20260709 13:20:59.411049 15528 raft_consensus.cc:1240] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:20:59.411563 16205 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:59.416601 15602 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 20.
I20260709 13:20:59.312026 21369 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.393726 21787 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:59.409090 21648 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:59.387655 21769 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:59.387801 21768 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:59.423770 16205 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:20:59.351393 21175 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.345247 21529 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:59.402604 21659 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:59.424944 21721 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:20:59.451606 21175 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:59.439610 21466 consensus_queue.cc:1059] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:20:59.454916 21466 raft_consensus.cc:3055] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:59.455284 21466 raft_consensus.cc:740] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:59.456236 16239 raft_consensus.cc:2393] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 15: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:20:59.407985 21718 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:20:59.458285 21718 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:59.435567 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 230178us
I20260709 13:20:59.349867 21764 consensus_queue.cc:1059] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 13:20:59.421288 21785 raft_consensus.cc:2804] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:59.461050 21785 raft_consensus.cc:697] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:59.461958 21785 consensus_queue.cc:237] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:59.463400 15491 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:59.466836 15491 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:59.426680 21786 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:59.350052 21804 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.083s
I20260709 13:20:59.433209 21765 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:59.446239 15567 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:59.469475 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.470327 15491 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:59.442198 21706 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:20:59.480134 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 56453.1us
I20260709 13:20:59.440203 21369 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.490276 21739 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:59.491355 21711 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.491976 21658 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.478559 21407 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:59.498807 21468 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:59.499927 21718 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:59.473598 21659 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.472820 21426 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:59.502616 21659 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:59.504199 21561 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.505473 15525 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:20:59.506423 21789 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:59.510746 21706 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.512872 21787 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:59.513099 21706 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:59.519457 15567 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:59.524981 15526 raft_consensus.cc:1275] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:20:59.528708 16205 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:59.513419 21787 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:59.469878 21742 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:59.447952 21792 consensus_queue.cc:1059] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
W20260709 13:20:59.540350 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 132204us
W20260709 13:20:59.534670 21689 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:20:59.532171 21798 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:59.585769 15602 raft_consensus.cc:1240] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 24 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 23. Current term is 24. Ops: []
I20260709 13:20:59.590147 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:59.590591 15676 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:20:59.461902 21755 consensus_queue.cc:237] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.591958 15598 raft_consensus.cc:1240] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:20:59.592567 21719 raft_consensus.cc:3055] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:59.592974 21719 raft_consensus.cc:740] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:59.593726 21719 consensus_queue.cc:260] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:59.594924 21719 raft_consensus.cc:3060] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:59.602837 21825 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:59.604434 21711 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:59.578589 21658 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 24 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:59.619506 21466 consensus_queue.cc:260] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.646446 21466 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:59.621601 21561 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:59.584776 21788 consensus_queue.cc:1048] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:20:59.528169 21752 consensus_queue.cc:1048] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:20:59.627061 21711 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.627403 21708 raft_consensus.cc:3055] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:59.539978 21175 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:59.534511 21529 consensus_queue.cc:1059] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.078s
I20260709 13:20:59.588222 21659 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:20:59.642784 21619 raft_consensus.cc:3055] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:59.616384 15674 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 15.
I20260709 13:20:59.590258 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:20:59.662113 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 128054us
I20260709 13:20:59.555320 21714 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:59.671842 15602 raft_consensus.cc:1240] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:59.673317 15601 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:59.674639 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:59.675271 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:20:59.683693 15603 raft_consensus.cc:3060] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:59.684041 15599 raft_consensus.cc:2393] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 24: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
W20260709 13:20:59.690600 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 100449us
I20260709 13:20:59.704602 15597 raft_consensus.cc:1240] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:59.691816 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:59.709872 15603 raft_consensus.cc:2468] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 15.
I20260709 13:20:59.664714 21708 raft_consensus.cc:740] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:59.666570 21619 raft_consensus.cc:740] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:59.718542 16205 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:20:59.674006 21468 raft_consensus.cc:697] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:20:59.722254 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:59.720419 21764 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:59.726750 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:59.727523 15526 raft_consensus.cc:2393] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 15: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:59.731633 21708 consensus_queue.cc:260] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:20:59.732273 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:59.732911 15528 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 17.
W20260709 13:20:59.733994 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 214640us
I20260709 13:20:59.689059 21855 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.712997 21719 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:59.667181 21711 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.735630 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:20:59.744228 15526 raft_consensus.cc:2393] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 13: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:59.715571 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:59.733177 21708 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:59.739524 21719 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:59.756011 21719 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:59.759131 21787 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.761600 21787 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:59.742825 21855 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:59.748710 21468 consensus_queue.cc:237] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:20:59.749295 21812 log.cc:927] Time spent T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 1.250s	user 0.002s	sys 0.004s
I20260709 13:20:59.749779 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:59.762432 15566 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:59.763545 21765 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:59.764920 21619 consensus_queue.cc:260] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.719743 21769 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.720180 21751 consensus_queue.cc:1059] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 13:20:59.809351 21659 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.809974 21659 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.812435 21659 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:59.776623 21849 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:59.814079 21769 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:59.816272 15669 raft_consensus.cc:1275] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:20:59.816645 15669 pending_rounds.cc:85] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting all ops after (but not including) 7
I20260709 13:20:59.816912 15669 pending_rounds.cc:107] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:20:59.817163 15669 raft_consensus.cc:2889] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:59.735209 21529 raft_consensus.cc:2804] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:20:59.744582 21704 consensus_meta.cc:287] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.522s	user 0.008s	sys 0.003s
I20260709 13:20:59.789258 16204 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:20:59.846053 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 56885.6us
I20260709 13:20:59.797465 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:59.847864 15521 raft_consensus.cc:2393] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 15: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:20:59.800731 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:59.747884 21654 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:20:59.806566 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 44232.4us
I20260709 13:20:59.834784 21619 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:59.817337 21822 raft_consensus.cc:3055] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:20:59.841408 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:59.842186 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:20:59.848517 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 70536us
I20260709 13:20:59.854889 15566 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:20:59.870244 15526 raft_consensus.cc:1240] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:59.871824 15601 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:20:59.872942 15528 raft_consensus.cc:1240] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:59.822505 21714 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:59.816671 21769 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:20:59.873631 15491 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:59.874661 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:20:59.795126 21755 consensus_queue.cc:1059] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.053s
I20260709 13:20:59.825194 21815 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.824383 21849 raft_consensus.cc:697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
W20260709 13:20:59.869134 21811 log.cc:927] Time spent T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 1.384s	user 0.009s	sys 0.002s
I20260709 13:20:59.882741 15491 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:20:59.846278 21858 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.852962 21764 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:20:59.856753 21529 raft_consensus.cc:697] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:59.865341 21742 raft_consensus.cc:3055] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:59.899847 21742 raft_consensus.cc:740] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:59.865552 21779 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:59.865113 21822 raft_consensus.cc:740] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:59.859318 21765 consensus_queue.cc:1059] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:20:59.887687 21721 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:59.887346 21466 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:20:59.918678 15601 raft_consensus.cc:1275] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:20:59.920475 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:20:59.921746 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:20:59.923679 21788 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.957911 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:20:59.902629 21858 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.902287 21853 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:59.924332 15528 raft_consensus.cc:1275] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:20:59.926717 15670 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:20:59.926250 21849 consensus_queue.cc:237] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.927479 21786 raft_consensus.cc:3055] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:59.927920 21855 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:59.928925 21728 raft_consensus.cc:3055] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:59.929607 21175 raft_consensus.cc:3055] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:59.980173 21175 raft_consensus.cc:740] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:59.974261 21304 consensus_queue.cc:1048] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:20:59.981114 21175 consensus_queue.cc:260] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.931761 21731 raft_consensus.cc:3055] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:59.907449 21407 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.982075 21731 raft_consensus.cc:740] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:20:59.982283 21407 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:59.977200 21728 raft_consensus.cc:740] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:20:59.983778 21728 consensus_queue.cc:260] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.933704 21711 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:20:59.472280 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:59.978917 21513 raft_consensus.cc:493] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:59.984494 21407 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:20:59.938287 15668 raft_consensus.cc:1275] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:20:59.987458 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:59.988256 15668 raft_consensus.cc:2410] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:20:59.989481 15668 raft_consensus.cc:1275] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:20:59.938761 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:59.940184 15672 raft_consensus.cc:1240] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:59.940374 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:59.990732 15674 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:20:59.991379 15672 raft_consensus.cc:1275] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:20:59.991760 15672 pending_rounds.cc:85] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4: Aborting all ops after (but not including) 7
I20260709 13:20:59.991966 15672 pending_rounds.cc:107] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:20:59.992182 15672 raft_consensus.cc:2889] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:59.884815 21654 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 13:20:59.992354 21742 consensus_queue.cc:260] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.904908 21779 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:59.993659 21742 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:59.994031 21779 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:59.994786 21513 raft_consensus.cc:515] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:20:59.965807 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:00.001241 21466 consensus_queue.cc:1048] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:21:00.002146 21175 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:00.001305 15566 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:21:00.007231 21466 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:00.007668 21466 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:00.007994 21466 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:00.008581 15670 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:20:59.961000 21737 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.968247 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:20:59.886263 21815 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:59.951742 21721 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:20:59.857975 21719 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:00.010958 15520 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
I20260709 13:20:59.960358 21726 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.022575 15602 raft_consensus.cc:1240] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:20:59.968032 15676 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:21:00.026367 15676 pending_rounds.cc:85] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting all ops after (but not including) 10
I20260709 13:21:00.026782 15676 pending_rounds.cc:107] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:21:00.027107 15676 raft_consensus.cc:2889] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:59.905257 21764 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:00.033869 15528 raft_consensus.cc:1275] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:20:59.973588 21786 raft_consensus.cc:740] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:20:59.975737 21853 raft_consensus.cc:697] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:21:00.046609 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:00.026489 21721 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:59.974617 21658 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.066s
W20260709 13:21:00.158787 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:00.159128 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:00.159591 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.042429 21339 consensus_queue.cc:1059] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.124s
I20260709 13:21:00.032893 21561 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.120s
I20260709 13:20:59.969965 21788 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:00.028842 21708 consensus_queue.cc:1059] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.135s
I20260709 13:21:00.026574 21737 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:20:59.977761 21529 consensus_queue.cc:237] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:00.036860 21389 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.132s
I20260709 13:21:00.040688 21654 consensus_queue.cc:1048] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.128s
I20260709 13:21:00.037417 21787 consensus_queue.cc:1048] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.133s
I20260709 13:21:00.036422 21726 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:00.195713 21765 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.191915 21769 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.010200 21822 consensus_queue.cc:260] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:00.177579 21864 log.cc:927] Time spent T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 13:21:00.206295 16234 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
I20260709 13:21:00.157617 21680 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:21:00.221278 21858 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:00.223587 21866 log.cc:927] Time spent T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.188s	user 0.000s	sys 0.000s
I20260709 13:21:00.223377 21786 consensus_queue.cc:260] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:00.225066 21786 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:00.229756 15525 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:21:00.230247 15525 pending_rounds.cc:85] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 9
I20260709 13:21:00.230468 15525 pending_rounds.cc:107] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:21:00.230683 15525 raft_consensus.cc:2889] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:00.231215 21737 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:00.233634 15525 raft_consensus.cc:3060] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:00.240447 15525 raft_consensus.cc:1275] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:21:00.240217 21779 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:00.250360 16240 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53976 (request call id 1276) took 1197 ms. Trace:
I20260709 13:21:00.250808 16240 rpcz_store.cc:276] 0709 13:20:59.052841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:59.054338 (+  1497us) service_pool.cc:224] Handling call
0709 13:20:59.054639 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:59.082861 (+ 28222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:59.082891 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:59.090135 (+  7244us) log.cc:844] Serialized 26 byte log entry
0709 13:20:59.914339 (+824204us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b1800364ea0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab0d1 00007f5ce49c6a0f
0709 13:20:59.978769 (+ 64430us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:20:59.978809 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:59.979015 (+   206us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:00.250042 (+271027us) raft_consensus.cc:1685] finished
0709 13:21:00.250062 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:00.250292 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28244,"raft.queue_time_us":49521,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":91117,"spinlock_wait_cycles":3946880,"thread_start_us":62590,"threads_started":1,"wal-append.queue_time_us":831821}
I20260709 13:21:00.252979 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:00.260180 15602 raft_consensus.cc:1275] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:21:00.260586 15602 pending_rounds.cc:85] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting all ops after (but not including) 9
I20260709 13:21:00.260838 15602 pending_rounds.cc:107] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:21:00.261155 15602 raft_consensus.cc:2889] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:00.262481 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:00.260710 15596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 2605) took 1149 ms. Trace:
I20260709 13:21:00.272802 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:00.274315 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:00.276129 16232 raft_consensus.cc:1240] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:00.270561 15596 rpcz_store.cc:276] 0709 13:20:59.110911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:59.203571 (+ 92660us) service_pool.cc:224] Handling call
0709 13:20:59.203883 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:59.248837 (+ 44954us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:59.248872 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:59.249388 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 13:20:59.444477 (+195089us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:20:59.444521 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:00.260301 (+815780us) spinlock_profiling.cc:243] Waited 31.8 ms on lock 0x7b540016ba54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:21:00.260351 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:00.260394 (+    43us) raft_consensus.cc:1685] finished
0709 13:21:00.260407 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:00.260638 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7374,"raft.queue_time_us":27142,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":62,"spinlock_wait_cycles":73219328,"thread_start_us":194679,"threads_started":1,"wal-append.queue_time_us":362613}
I20260709 13:21:00.287962 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:00.293597 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:00.294621 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:00.295744 21708 consensus_queue.cc:1059] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:00.296563 15597 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:21:00.298434 15596 raft_consensus.cc:2393] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 15: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:21:00.299176 15641 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:00.302330 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:00.215472 21769 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:00.215891 21765 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:00.305727 15641 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:00.307952 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:00.313709 21304 consensus_queue.cc:1059] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:21:00.316313 21737 consensus_queue.cc:1048] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:21:00.322809 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:00.324244 15597 raft_consensus.cc:1275] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:21:00.321844 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:00.326679 15567 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:00.328536 21389 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.329082 21389 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:00.331256 21389 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:00.326056 15672 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 17.
I20260709 13:21:00.296625 21708 raft_consensus.cc:3055] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:00.355772 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:00.217015 21853 consensus_queue.cc:237] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:00.369613 15676 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
W20260709 13:21:00.324282 21860 log.cc:927] Time spent T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.364s	user 0.000s	sys 0.001s
W20260709 13:21:00.384130 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.377224 16239 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:21:00.402544 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 91177.5us
I20260709 13:21:00.352754 21649 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:00.304498 21729 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:21:00.291303 21848 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 13:21:00.352934 21828 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:00.428941 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:00.435806 15491 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:00.369818 21708 raft_consensus.cc:740] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
W20260709 13:21:00.424119 21869 log.cc:927] Time spent T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.270s	user 0.000s	sys 0.001s
I20260709 13:21:00.417225 21719 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:00.445591 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:00.420271 21815 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:00.367772 21466 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:00.457175 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:00.441491 21711 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:00.369289 21714 consensus_queue.cc:1048] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 13:21:00.444916 21726 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:21:00.454241 21867 log.cc:927] Time spent T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.291s	user 0.001s	sys 0.000s
I20260709 13:21:00.459719 21721 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:00.463404 21711 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:00.482193 16233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1842) took 1454 ms. Trace:
I20260709 13:21:00.482693 16233 rpcz_store.cc:276] 0709 13:20:59.028145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:59.029182 (+  1037us) service_pool.cc:224] Handling call
0709 13:20:59.029440 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:59.201113 (+171673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:00.474697 (+1273584us) spinlock_profiling.cc:243] Waited 55.6 ms on lock 0x7b54001e1754. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:21:00.482111 (+  7414us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":127809792}
I20260709 13:21:00.483865 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:21:00.488224 16204 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.010s	sys 0.018s
W20260709 13:21:00.488605 16204 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.010s	sys 0.018s
W20260709 13:21:00.488895 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 540940us
I20260709 13:21:00.506649 15493 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:00.483631 21711 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:00.483186 21764 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:00.522315 16233 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:21:00.527199 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:00.524238 21779 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:00.523207 21680 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:21:00.529639 21764 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:00.530215 21764 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:00.527901 16239 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:21:00.532258 21764 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:00.514524 21828 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:00.535372 21828 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:00.518383 21703 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:00.537482 21828 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:00.507913 21708 consensus_queue.cc:260] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:00.540781 21708 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:00.515157 21389 consensus_queue.cc:1048] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:21:00.554695 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:00.555316 15523 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:00.556725 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 125048us
I20260709 13:21:00.559734 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:00.560343 15672 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:21:00.561160 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:00.561487 15566 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:00.561703 15672 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:21:00.563336 15566 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:00.549649 21765 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:00.566439 21731 consensus_queue.cc:260] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:00.567713 21731 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:00.570007 21714 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:00.570467 21714 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:00.570787 21714 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:00.571131 21466 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:00.572171 21389 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:00.572770 21389 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:00.575060 21389 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:00.583395 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:00.553929 21769 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:00.571810 21466 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:00.585285 21466 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:00.585796 21466 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:21:00.589581 21742 consensus_meta.cc:287] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.595s	user 0.006s	sys 0.001s
I20260709 13:21:00.595001 15602 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:21:00.592763 21721 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:21:00.599647 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.606655 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:00.607359 15670 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:00.611629 15523 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:21:00.626919 15567 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:00.632052 21854 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:00.632776 21854 raft_consensus.cc:697] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:21:00.634495 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.633737 21854 consensus_queue.cc:237] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:00.641031 21369 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:00.649253 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:00.649996 15674 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 17.
I20260709 13:21:00.651139 15675 raft_consensus.cc:1240] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:00.652680 15493 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:00.634315 21682 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.656953 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:00.660285 21439 raft_consensus.cc:3055] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:00.660744 21439 raft_consensus.cc:740] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:00.631467 21714 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:00.670023 21703 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:00.670401 21714 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:21:00.671211 21887 log.cc:927] Time spent T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.115s	user 0.002s	sys 0.000s
W20260709 13:21:00.674090 21883 log.cc:927] Time spent T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.119s	user 0.002s	sys 0.000s
I20260709 13:21:00.661520 21439 consensus_queue.cc:260] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:00.686959 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:00.689575 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:00.686219 21439 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:00.691903 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:00.692431 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:00.693359 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:00.693554 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:00.695178 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:00.695839 15670 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:21:00.676447 15493 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:00.679809 21466 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:00.680531 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:00.682224 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:00.683135 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:00.676064 21513 leader_election.cc:290] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:00.702806 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:00.731418 15673 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:00.708020 15674 raft_consensus.cc:1240] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:21:00.708640 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e93d4a9ac6e844da87dcd8c6bfa7549b" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:00.709852 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:00.715235 15669 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:00.715700 15525 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:00.734153 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:00.735687 21765 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:21:00.743479 16203 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:00.744884 21513 consensus_queue.cc:1059] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:21:00.745268 21655 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:00.746052 21513 raft_consensus.cc:3055] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:21:00.746472 21513 raft_consensus.cc:740] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:00.657691 21529 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:00.709236 21894 log.cc:927] Time spent T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.092s	user 0.001s	sys 0.001s
W20260709 13:21:00.768024 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.765005 15525 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18.
I20260709 13:21:00.763788 15642 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:00.731181 21636 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:00.777472 16204 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:00.753423 15596 raft_consensus.cc:1240] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:00.751645 21654 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:00.739032 21785 consensus_queue.cc:1048] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:21:00.774228 21334 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:00.764544 15673 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:21:00.796186 15566 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:21:00.709378 21682 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:00.798532 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 58178.9us
I20260709 13:21:00.739111 21808 consensus_queue.cc:1059] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.051s
W20260709 13:21:00.808854 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.785986 21885 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:00.805140 15493 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:00.795194 21765 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:00.821964 15493 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:00.780681 21529 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:00.813989 21888 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:00.807466 21853 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.818596 21765 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:00.803329 21334 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:00.829062 21808 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:00.828249 21885 raft_consensus.cc:697] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:00.839025 21703 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:21:00.826314 21719 consensus_queue.cc:1048] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:21:00.815827 21849 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:21:00.815928 21636 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
W20260709 13:21:00.882113 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.868359 21808 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:00.892612 15670 raft_consensus.cc:1275] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:21:00.847103 21765 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:00.847342 21888 raft_consensus.cc:697] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:00.866394 21848 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:00.913470 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.849494 21853 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:00.894382 21877 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:21:00.925272 21901 log.cc:927] Time spent T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:21:00.926239 21899 log.cc:927] Time spent T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
W20260709 13:21:00.930667 21896 log.cc:927] Time spent T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:21:00.923692 21877 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:00.945470 21334 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:00.955144 21808 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:00.956669 21714 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.966840 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:00.958137 21682 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:00.959807 21529 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:00.963462 15677 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:21:00.971158 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:00.971392 15566 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:00.967552 15674 raft_consensus.cc:2393] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 9: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:21:00.969305 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:00.969246 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:00.970458 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:00.913874 21848 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:21:00.974104 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:00.973188 16233 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:21:00.937958 21466 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:00.977695 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:00.978350 15597 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 20.
I20260709 13:21:00.938651 21906 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:00.935271 21711 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:00.975867 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:00.984898 15670 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:00.979592 16205 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:00.980916 16203 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:00.981597 15669 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18.
I20260709 13:21:00.969022 21877 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:00.997531 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:01.001037 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:00.999161 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:01.011597 15669 raft_consensus.cc:1240] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:00.979377 21682 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:01.032826 15670 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:21:01.036496 15524 raft_consensus.cc:2393] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 18: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:21:00.923635 21888 consensus_queue.cc:237] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:01.004944 21339 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.994201 21529 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:01.000213 21856 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:00.973325 21714 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:01.079500 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 86220.6us
I20260709 13:21:01.034674 21682 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:01.082348 21339 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:01.107685 21853 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:01.099881 15602 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:21:00.944200 21885 consensus_queue.cc:237] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:01.120882 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 122694us
I20260709 13:21:01.077298 21856 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:01.123432 15568 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:00.995218 21906 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:01.140749 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 308063us
I20260709 13:21:01.150648 15597 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:21:01.115509 21682 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:01.150797 15568 leader_election.cc:304] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:01.157799 21369 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:01.158151 21920 consensus_queue.cc:1059] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:21:01.126506 21856 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:01.170253 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:01.172941 21711 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:01.141381 21752 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:01.177014 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:01.182468 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:01.186197 21339 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:01.189713 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:01.189785 15520 raft_consensus.cc:1240] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:21:01.170925 21726 raft_consensus.cc:2749] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:01.171761 21916 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:01.156042 21808 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:21:01.197751 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:01.197849 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:01.215309 21916 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:01.335561 15643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:01.369663 21849 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:01.370275 21636 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:21:01.370625 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 171435us
I20260709 13:21:01.371598 21389 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:21:01.372076 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 200417us
I20260709 13:21:01.373642 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:01.374485 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:01.377729 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:01.170548 21369 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:01.380640 15641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:01.384013 21334 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:01.399828 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:01.408133 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:21:01.408962 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:01.396584 21439 consensus_meta.cc:287] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.705s	user 0.017s	sys 0.000s
I20260709 13:21:01.417860 21682 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:01.422394 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:01.423260 21916 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:01.389220 21659 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:21:01.399055 21879 log.cc:927] Time spent T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.844s	user 0.004s	sys 0.011s
W20260709 13:21:01.401239 21912 log.cc:927] Time spent T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.244s	user 0.001s	sys 0.006s
I20260709 13:21:01.435555 21728 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:01.435433 21659 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:01.437387 21369 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:01.437569 21619 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:21:01.440126 21765 consensus_meta.cc:287] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.520s	user 0.009s	sys 0.012s
I20260709 13:21:01.439685 21513 consensus_queue.cc:260] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:01.396507 21849 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:01.405004 21848 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:01.462404 21848 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:01.462946 21848 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:01.465126 21848 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:01.449505 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:01.403990 21389 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:01.444671 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:01.456893 15528 raft_consensus.cc:1275] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:21:01.432740 21714 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:01.398861 21636 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:01.466871 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:01.473901 15520 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:21:01.450488 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:01.474622 15526 raft_consensus.cc:2393] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 14: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:21:01.409322 21334 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:01.483844 15491 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:21:01.483728 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 81253.8us
I20260709 13:21:01.487064 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:01.481546 21513 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:21:01.462464 21659 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:01.490798 15669 raft_consensus.cc:1240] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:01.487735 15528 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:21:01.458643 21619 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:01.490561 21919 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:21:01.494833 21898 log.cc:927] Time spent T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.752s	user 0.007s	sys 0.008s
I20260709 13:21:01.466216 21765 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:01.507951 21334 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:01.539609 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:01.547281 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:21:01.547508 15677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49100 (request call id 3042) took 1601 ms (client timeout 1557 ms). Trace:
I20260709 13:21:01.490232 21904 consensus_queue.cc:1048] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:21:01.550295 15670 raft_consensus.cc:1240] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 8. Current term is 9. Ops: []
W20260709 13:21:01.548648 15677 rpcz_store.cc:269] 0709 13:20:59.937223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:59.938479 (+  1256us) service_pool.cc:224] Handling call
0709 13:21:01.538001 (+1599522us) spinlock_profiling.cc:243] Waited 973 ms on lock 0x7b54003881d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:21:01.538137 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2238236544}
I20260709 13:21:01.554587 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:01.555542 15670 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 15.
I20260709 13:21:01.555768 15669 raft_consensus.cc:1240] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:01.497118 21808 consensus_queue.cc:1048] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:21:01.558421 21369 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:01.559041 21369 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:01.561149 21369 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:01.563760 21765 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:01.564541 15674 raft_consensus.cc:2393] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 18: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:21:01.565543 16203 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:01.567900 21389 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:01.506404 21906 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:01.569450 21849 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:01.570583 16203 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:01.494879 21703 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:21:01.566612 21636 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:01.566092 21877 consensus_meta.cc:287] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.556s	user 0.006s	sys 0.006s
I20260709 13:21:01.578015 21682 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:01.577895 21877 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:01.578514 21788 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:01.581044 21619 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:01.581207 21877 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:01.583266 15521 raft_consensus.cc:1240] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:21:01.583885 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:01.497473 21911 consensus_queue.cc:1048] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.082s
W20260709 13:21:01.584941 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 179341us
I20260709 13:21:01.597864 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:01.584545 16234 raft_consensus.cc:3060] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:01.607245 21934 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:21:01.609349 15568 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:01.610365 21934 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:01.610821 21934 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:01.611208 21934 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:01.574847 21851 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:01.614162 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:01.615628 15642 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:21:01.609251 21900 log.cc:927] Time spent T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.865s	user 0.002s	sys 0.000s
I20260709 13:21:01.566113 21925 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:01.622807 21931 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:01.570513 21856 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:01.625674 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:01.618713 21714 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:01.620294 21659 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:01.619761 21704 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:21:01.634156 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 63643.6us
I20260709 13:21:01.637292 16235 raft_consensus.cc:1240] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:01.639241 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:01.640033 16231 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:21:01.649349 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:01.650132 16239 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:21:01.651882 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:01.635274 21650 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:01.654728 21650 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:01.655136 21650 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:01.661082 15601 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:21:01.666319 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:01.665562 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:01.668533 15669 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
I20260709 13:21:01.668926 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:01.671583 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:01.677017 16234 raft_consensus.cc:2468] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 18.
I20260709 13:21:01.678347 15603 raft_consensus.cc:1240] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:01.632332 21931 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:01.737452 21931 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:01.680866 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:21:01.684518 21928 log.cc:927] Time spent T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.201s	user 0.002s	sys 0.000s
I20260709 13:21:01.739746 21854 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:01.686232 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:01.740329 21854 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:01.634254 21714 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:01.741365 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:01.742523 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:01.743961 15669 raft_consensus.cc:1275] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 6. (term mismatch)
I20260709 13:21:01.744362 15669 pending_rounds.cc:85] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4: Aborting all ops after (but not including) 5
I20260709 13:21:01.744565 15669 pending_rounds.cc:107] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4: Aborting uncommitted NO_OP operation due to leader change: 12.6
I20260709 13:21:01.744786 15669 raft_consensus.cc:2889] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:01.752868 15642 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:01.754585 15642 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:01.693598 15568 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:01.669490 21929 consensus_queue.cc:1059] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.049s
W20260709 13:21:01.806836 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 52334.3us
I20260709 13:21:01.637797 21925 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:01.727635 21856 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:01.807111 15566 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:21:01.810113 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 318895us
I20260709 13:21:01.813422 15669 raft_consensus.cc:1275] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:21:01.817049 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:01.821023 16234 raft_consensus.cc:2393] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 18: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:21:01.821462 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:01.816129 15641 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:21:01.824359 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 130823us
I20260709 13:21:01.824723 15566 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:21:01.814610 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 328011us
I20260709 13:21:01.825747 15641 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:01.766649 21919 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:21:01.831190 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:01.827337 15568 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:01.837111 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:01.849776 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:01.850549 15523 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
I20260709 13:21:01.814790 21765 raft_consensus.cc:2804] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:01.809840 21704 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:01.866385 15528 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 18. Current term is 19. Ops: []
W20260709 13:21:01.867290 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:01.873708 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 49060.3us
I20260709 13:21:01.878741 15566 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:21:01.890693 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 63341.7us
I20260709 13:21:01.823939 21439 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:01.809392 21755 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 13:21:01.824414 21885 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:01.841264 21849 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:21:01.841985 21851 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:01.856352 21779 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:01.872582 21726 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:01.921061 21650 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:01.922178 21439 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:01.926599 21704 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:01.865676 21513 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:01.872972 21848 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:01.929299 21682 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:01.869164 21649 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:01.930665 21822 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:01.921634 21739 raft_consensus.cc:3055] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:01.931115 21765 raft_consensus.cc:697] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:01.866089 21529 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:01.856559 21619 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:01.937085 16203 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:01.855751 21904 consensus_queue.cc:1059] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 13:21:01.872519 21930 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:01.875589 21929 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:01.910446 21856 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:01.936745 21925 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:01.918138 21888 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:21:01.998581 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 145148us
W20260709 13:21:02.007448 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 73628.5us
W20260709 13:21:02.006498 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 179311us
I20260709 13:21:01.998358 21916 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:21:02.012677 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 143078us
W20260709 13:21:02.021530 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 127680us
I20260709 13:21:02.017294 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:02.014766 16204 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:01.997493 21726 raft_consensus.cc:697] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:02.000630 21649 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:02.020426 15641 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:01.997753 21682 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:02.003757 21739 raft_consensus.cc:740] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:02.034066 16204 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:01.998056 21849 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.037808 15641 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:02.040217 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:02.046790 15528 raft_consensus.cc:2393] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 16: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:21:01.998576 21513 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.049366 15641 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:01.999861 21779 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:01.996827 21704 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:01.966090 21885 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:01.983901 21851 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.014524 21947 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:02.019783 21888 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:02.013330 21941 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:02.032444 21649 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:02.041190 21682 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:02.039852 21848 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:02.073796 21942 log.cc:927] Time spent T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:21:02.074683 21885 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:02.075835 21752 consensus_queue.cc:1048] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:02.062093 21947 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:02.078051 21958 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.080468 21658 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:21:02.074594 21936 log.cc:927] Time spent T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:21:02.011303 21529 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.011492 21619 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.047818 21769 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:02.045432 21636 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:02.009958 21934 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.033380 21856 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.033061 21929 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.077019 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:02.072546 21888 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:02.034142 21955 consensus_queue.cc:1048] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.095s
W20260709 13:21:02.188412 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 154430us
I20260709 13:21:02.020242 21930 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.138386 21658 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:02.123395 21957 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:02.132292 21848 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:21:02.150588 21951 log.cc:927] Time spent T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 13:21:02.156688 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:02.161989 21779 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:02.108635 21941 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.167848 21704 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:02.170003 21513 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:02.170655 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:02.171357 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:02.172192 21851 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:02.122116 21765 consensus_queue.cc:237] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.132051 21947 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:02.124208 21739 consensus_queue.cc:260] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.056303 21849 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.139062 21958 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.177599 16231 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 16.
I20260709 13:21:02.123061 21714 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:02.129098 21925 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:21:02.188052 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 138770us
I20260709 13:21:02.126770 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:02.177838 21636 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:02.201510 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:02.220781 15670 raft_consensus.cc:1240] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 13:21:02.205322 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 148535us
I20260709 13:21:02.208755 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:02.210807 15528 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:21:02.222432 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:02.144922 15524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56840 (request call id 2041) took 1464 ms. Trace:
I20260709 13:21:02.197297 21658 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.230005 15524 rpcz_store.cc:276] 0709 13:21:00.680044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.682917 (+  2873us) service_pool.cc:224] Handling call
0709 13:21:02.144676 (+1461759us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b540029c7d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:21:02.144834 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":690081664}
I20260709 13:21:02.127278 21726 consensus_queue.cc:237] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:02.276155 21529 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:02.277910 21941 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:02.282711 21619 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:02.283433 21849 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:02.283872 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:02.285101 21958 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:02.228808 21650 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:21:02.288249 21963 log.cc:927] Time spent T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:21:02.289845 21636 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:02.296470 21930 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:02.297173 15603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 2402) took 1120 ms. Trace:
I20260709 13:21:02.297899 21856 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:02.298444 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:02.298861 21940 consensus_queue.cc:1059] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:21:02.206066 21513 consensus_queue.cc:1059] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.065s
I20260709 13:21:02.224977 21957 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:02.300701 21929 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:02.200625 21848 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.211117 21439 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:02.305256 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:02.315151 21728 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:02.320672 15603 rpcz_store.cc:276] 0709 13:21:01.176305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:01.176519 (+   214us) service_pool.cc:224] Handling call
0709 13:21:01.176854 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:01.735760 (+558906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:02.296926 (+561166us) spinlock_profiling.cc:243] Waited 501 ms on lock 0x7b540043a154. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:21:02.297107 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1152894208}
I20260709 13:21:02.319202 21755 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:02.313452 21955 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.313210 21769 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:02.366750 16231 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 15.
I20260709 13:21:02.218423 21925 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:02.299360 21649 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.325889 15674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 2549) took 1153 ms. Trace:
I20260709 13:21:02.235157 21682 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.386024 15642 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:02.387930 21658 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:02.375315 21957 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.390709 21848 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:02.391480 21957 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:02.392784 21682 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:02.393352 21918 log.cc:927] Time spent T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 1.017s	user 0.000s	sys 0.011s
I20260709 13:21:02.378369 21728 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.398643 21728 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:02.401587 21934 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:02.415652 21769 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:02.377089 21946 raft_consensus.cc:3055] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:02.418889 21946 raft_consensus.cc:740] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:21:02.420413 21946 consensus_queue.cc:260] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:02.422672 21946 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:02.384856 15674 rpcz_store.cc:276] 0709 13:21:01.172707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:01.182227 (+  9520us) service_pool.cc:224] Handling call
0709 13:21:02.325688 (+1143461us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b540046d554. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:21:02.325828 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":876070656}
W20260709 13:21:02.434387 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:02.394215 21925 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:02.434742 15671 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 24.
I20260709 13:21:02.395458 21955 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.442574 15674 raft_consensus.cc:1240] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:21:02.446260 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:21:02.457422 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 71505.1us
I20260709 13:21:02.460638 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:02.460840 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:02.461470 16231 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:21:02.437518 21659 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.435052 21822 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:02.439847 21966 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.445016 21794 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.458782 21769 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:02.484495 15642 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:02.468776 21704 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:02.467054 21888 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:02.505719 15596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53324 (request call id 2395) took 1840 ms. Trace:
I20260709 13:21:02.506904 15596 rpcz_store.cc:276] 0709 13:21:00.664872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.670585 (+  5713us) service_pool.cc:224] Handling call
0709 13:21:00.670879 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:00.786582 (+115703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:02.505467 (+1718885us) spinlock_profiling.cc:243] Waited 82.4 ms on lock 0x7b540043b054. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:21:02.505646 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":189431552}
I20260709 13:21:02.507190 15522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56840 (request call id 2052) took 1064 ms. Trace:
I20260709 13:21:02.509557 15522 rpcz_store.cc:276] 0709 13:21:01.443015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:01.443996 (+   981us) service_pool.cc:224] Handling call
0709 13:21:01.444251 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:01.483535 (+ 39284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:01.483563 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:01.483603 (+    40us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:01.483632 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:02.506947 (+1023315us) spinlock_profiling.cc:243] Waited 39.1 ms on lock 0x7b5400263254. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:21:02.506995 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:02.507136 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89899904}
I20260709 13:21:02.496253 21708 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:02.481186 21659 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.496995 21704 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:02.499929 21439 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:02.505489 21949 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:02.482923 21966 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.482249 21822 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.483563 21794 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:21:02.520711 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 106562us
W20260709 13:21:02.501462 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 53288.6us
I20260709 13:21:02.517005 21704 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:02.529582 21888 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:02.516716 21439 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:02.535178 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:02.537443 15522 raft_consensus.cc:2393] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 18: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:21:02.515106 21708 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.537222 21439 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:02.520803 21949 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:21:02.533034 21561 consensus_queue.cc:1059] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:21:02.599174 21513 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.599221 21888 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:02.623951 21974 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.629482 21955 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:02.635453 21513 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.665396 21911 raft_consensus.cc:3055] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:02.665858 21911 raft_consensus.cc:740] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:21:02.666664 21911 consensus_queue.cc:260] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:02.642267 21649 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:02.677872 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:02.640517 21888 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.661782 21934 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:02.694420 21929 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:02.695099 21929 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.677595 21822 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:02.678922 21854 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:02.667891 21911 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:02.676707 21856 raft_consensus.cc:3055] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:02.642982 21974 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.702554 21794 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:02.705883 21856 raft_consensus.cc:740] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:02.723326 16240 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57244 (request call id 1864) took 1082 ms. Trace:
I20260709 13:21:02.723891 16240 rpcz_store.cc:276] 0709 13:21:01.641128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:01.641227 (+    99us) service_pool.cc:224] Handling call
0709 13:21:01.641554 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:01.663219 (+ 21665us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:01.663252 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:02.547392 (+884140us) spinlock_profiling.cc:243] Waited 921 us on lock 0x7b2c00047828. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce3304a11 00007f5ce3314372 00007f5ce4a46a01 00007f5ce4a0b43a 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2
0709 13:21:02.600167 (+ 52775us) log.cc:844] Serialized 26 byte log entry
0709 13:21:02.667481 (+ 67314us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:21:02.667841 (+   360us) log.cc:844] Serialized 14 byte log entry
0709 13:21:02.668061 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:02.668155 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:02.722988 (+ 54833us) raft_consensus.cc:1685] finished
0709 13:21:02.723100 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:02.723262 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26143,"prepare.queue_time_us":4236,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":18419,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":3004,"spinlock_wait_cycles":2119040,"thread_start_us":60403,"threads_started":1,"wal-append.queue_time_us":71848}
I20260709 13:21:02.726614 21704 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:02.727696 21947 consensus_meta.cc:287] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.512s	user 0.007s	sys 0.003s
I20260709 13:21:02.728140 21947 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.716813 21966 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:02.730624 21708 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:02.736735 21650 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:02.697739 21934 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.737381 21650 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.739107 21934 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:02.739440 21650 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:02.789254 21974 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:02.785252 21529 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:02.791834 21529 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.799952 21529 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:21:02.812251 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:02.724089 21856 consensus_queue.cc:260] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:02.829563 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:02.829983 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:02.830535 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:02.830811 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:02.831377 16240 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:02.830662 16231 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 24.
I20260709 13:21:02.833259 16234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57258 (request call id 1348) took 1009 ms. Trace:
I20260709 13:21:02.833453 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:02.833643 16234 rpcz_store.cc:276] 0709 13:21:01.823854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:01.824025 (+   171us) service_pool.cc:224] Handling call
0709 13:21:02.502046 (+678021us) spinlock_profiling.cc:243] Waited 6.78 ms on lock 0x7b580009ff98. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99e4fc0 00007f5ce99cf929 00007f5ce99cf3d2 00007f5ce2f46bd4
0709 13:21:02.502246 (+   200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:02.502444 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:21:02.542354 (+ 39910us) log.cc:844] Serialized 14 byte log entry
0709 13:21:02.832973 (+290619us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:02.833021 (+    48us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:02.833060 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:02.833118 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:02.833212 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":131493,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"spinlock_wait_cycles":15596928,"thread_start_us":327118,"threads_started":2}
I20260709 13:21:02.835037 16236 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:21:02.807669 21561 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:02.821295 21636 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:02.823202 21940 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:02.841068 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:02.841436 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:02.843463 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:02.845427 16240 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:21:02.840093 21466 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:02.837986 21854 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:02.852066 21466 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.852258 21854 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:02.854555 21854 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:02.855077 21940 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:02.857405 21940 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:02.857625 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:21:02.858170 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:02.840360 21561 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.860237 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:02.860657 21561 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:02.833536 21794 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:02.841948 21636 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:02.862547 21794 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.863091 21466 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:02.833909 21856 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:02.825675 21929 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:02.865653 21636 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:02.834141 21934 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:02.836009 21888 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:02.865154 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:02.875883 16236 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:21:02.890550 16237 raft_consensus.cc:2410] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:21:02.888549 21439 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:02.886044 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:02.883328 21794 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:02.875083 21947 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:02.893247 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:21:02.893910 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 47600.8us
I20260709 13:21:02.898079 16237 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:02.885337 21925 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.879817 21940 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:02.914964 21940 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:02.917346 21940 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:02.896384 21931 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:02.918355 21931 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.919070 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:02.919664 16236 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:21:02.920647 21931 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:02.921489 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:02.922185 16240 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:21:02.928591 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:02.929458 16240 raft_consensus.cc:2393] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 16: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:21:02.930970 15642 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:02.938155 21513 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:21:02.938807 21973 log.cc:927] Time spent T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.445s	user 0.004s	sys 0.007s
I20260709 13:21:02.895267 21934 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.941509 21934 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:02.892558 21636 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:02.942981 21636 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:02.945168 21636 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:02.945524 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:02.946385 15670 raft_consensus.cc:2410] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:21:02.947608 21304 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:02.948067 15642 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:02.949033 21848 raft_consensus.cc:2804] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:21:02.949424 21848 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:02.950124 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:02.952653 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:21:02.953465 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 50054.7us
I20260709 13:21:02.937512 21858 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:02.954293 15493 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:02.956120 15675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49094 (request call id 2536) took 2222 ms. Trace:
I20260709 13:21:02.956233 15642 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:02.957454 21957 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:02.956640 15675 rpcz_store.cc:276] 0709 13:21:00.733618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.959350 (+225732us) service_pool.cc:224] Handling call
0709 13:21:00.959748 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:01.008848 (+ 49100us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:01.008882 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:01.013560 (+  4678us) log.cc:844] Serialized 26 byte log entry
0709 13:21:02.170535 (+1156975us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b180036dc60. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab0d1 00007f5ce49c6a0f
0709 13:21:02.317905 (+147370us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:02.317967 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:02.803343 (+485376us) spinlock_profiling.cc:243] Waited 48.7 ms on lock 0x7b5400186854. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:21:02.803402 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:02.938234 (+134832us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:21:02.955919 (+ 17685us) raft_consensus.cc:1685] finished
0709 13:21:02.955940 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:02.956063 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34652,"raft.queue_time_us":2203,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":22894,"spinlock_wait_cycles":138766720,"thread_start_us":147188,"threads_started":1,"wal-append.queue_time_us":1440676}
I20260709 13:21:02.958675 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:02.961254 16236 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
W20260709 13:21:02.963332 21946 consensus_meta.cc:287] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.508s	user 0.008s	sys 0.001s
I20260709 13:21:02.966965 21794 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.967531 21794 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.970280 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:02.971408 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:02.972199 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:02.974239 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:02.977437 15568 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:02.979209 15568 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:02.981168 21737 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:21:02.981580 21737 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:02.981953 21737 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:02.983084 21466 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:02.984535 21466 raft_consensus.cc:697] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:02.985548 21466 consensus_queue.cc:237] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.990445 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:02.969580 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:02.998390 16240 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 19.
I20260709 13:21:02.999239 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:02.959956 15675 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:21:03.000524 15566 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:21:03.000876 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:21:03.001981 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 47779.3us
I20260709 13:21:03.002718 15493 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:03.002951 15568 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:02.997259 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:03.003643 15642 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:03.004549 15603 raft_consensus.cc:2368] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 16. Current term is 17.
I20260709 13:21:03.007414 16237 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 17.
I20260709 13:21:03.014622 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:03.014987 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:03.015272 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:02.949874 21848 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:21:03.017903 16236 raft_consensus.cc:2393] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 17: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:21:03.017463 15493 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:03.017392 15568 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:02.949467 21769 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:02.969695 21794 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:03.023788 21636 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:03.025776 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:03.034586 15670 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:21:03.027385 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:03.040303 15522 raft_consensus.cc:2410] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 14 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:21:03.041095 15523 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
I20260709 13:21:03.040975 15566 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:21:03.028055 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:03.043164 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:03.044108 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:03.044710 15522 raft_consensus.cc:3060] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:03.047521 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:03.016923 21916 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:03.050168 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:03.019224 21822 raft_consensus.cc:2804] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:03.053731 15568 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:03.070470 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.070853 16235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53976 (request call id 1299) took 1595 ms. Trace:
I20260709 13:21:03.055294 15523 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 9.
I20260709 13:21:03.016744 21856 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:03.071257 16235 rpcz_store.cc:276] 0709 13:21:01.475608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:01.476908 (+  1300us) service_pool.cc:224] Handling call
0709 13:21:01.477158 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:01.638244 (+161086us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:03.070588 (+1432344us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b5400496954. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:21:03.070790 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":367856896}
I20260709 13:21:03.083841 15523 raft_consensus.cc:1275] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:21:03.056828 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:21:03.084388 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 43502.3us
I20260709 13:21:03.020980 21940 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:21:03.055001 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:03.087527 15525 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:21:03.056936 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:03.021510 21854 raft_consensus.cc:2804] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:03.005710 21946 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:21:03.071640 15670 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18.
I20260709 13:21:03.055776 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:03.024652 21769 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:02.996915 21858 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:03.049451 21708 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:03.049695 21794 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:03.045297 21636 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:03.102492 15526 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:21:03.055050 21916 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:03.108294 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:03.120158 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.121081 15524 raft_consensus.cc:2468] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 16.
I20260709 13:21:03.095278 21822 raft_consensus.cc:697] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:03.094527 21649 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:03.098304 21966 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:03.127738 16232 raft_consensus.cc:1275] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:21:03.101452 21854 raft_consensus.cc:697] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:03.128974 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:03.130162 15596 raft_consensus.cc:2393] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 16: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:21:03.110785 21916 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:03.133219 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:03.106220 21636 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:03.135807 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.147965 15522 raft_consensus.cc:2468] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
I20260709 13:21:03.090602 21856 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:03.116281 21946 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:03.163671 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:03.164477 15598 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
W20260709 13:21:03.167660 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 116818us
I20260709 13:21:03.107591 21708 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:03.131423 21966 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:03.175320 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.176149 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:03.194012 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:03.193970 15670 raft_consensus.cc:2393] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 14: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:21:03.196623 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 108535us
I20260709 13:21:03.196841 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:03.167487 21304 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:03.201107 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:03.139273 21649 raft_consensus.cc:697] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:03.198987 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db313a4cd18a47a2b72587d04ed6e747" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:03.200155 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:03.202440 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:03.198181 15568 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:03.137620 21765 consensus_queue.cc:1059] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 13:21:03.257897 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.191051 21966 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:03.279637 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:21:03.279929 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 81897.2us
W20260709 13:21:03.276117 16206 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:03.281294 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:03.282514 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:03.283550 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:03.284051 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:03.191020 21925 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:03.197357 21965 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.289358 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.290604 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:21:03.292587 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:03.296818 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:03.312913 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:03.170002 21946 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:03.319095 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:03.320324 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.321900 15670 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 11.
I20260709 13:21:03.322781 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.323346 15670 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:21:03.324018 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:03.324400 15670 raft_consensus.cc:3060] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:03.334167 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:03.334311 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:03.203994 21439 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:03.336369 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:03.337810 15525 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 9.
W20260709 13:21:03.343091 15672 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49094 (request call id 2540) took 2596 ms (client timeout 1729 ms). Trace:
W20260709 13:21:03.290493 21911 consensus_meta.cc:287] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.585s	user 0.018s	sys 0.000s
I20260709 13:21:03.345472 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:03.349447 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:03.350109 15594 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18.
I20260709 13:21:03.279932 21304 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:03.356029 21915 log.cc:927] Time spent T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 2.174s	user 0.020s	sys 0.040s
I20260709 13:21:03.357338 21304 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:03.349090 15672 rpcz_store.cc:269] 0709 13:21:00.747128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:01.685939 (+938811us) service_pool.cc:224] Handling call
0709 13:21:03.333994 (+1648055us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b5800012398. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6e9c 00007f5ce18e6a5b 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99d10a0 00007f5ce99d0984 00007f5ce2f4b6d4 00007f5ce2f4b662
0709 13:21:03.343016 (+  9022us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34344192}
I20260709 13:21:03.358322 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:03.358812 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.359009 15525 raft_consensus.cc:3060] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:03.363246 15672 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18.
I20260709 13:21:03.292981 21726 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:03.370008 15522 raft_consensus.cc:1275] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:21:03.370442 15522 pending_rounds.cc:85] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 8
I20260709 13:21:03.370668 15522 pending_rounds.cc:107] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:21:03.370903 15522 raft_consensus.cc:2889] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:03.371994 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:03.373299 15668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49094 (request call id 2530) took 3047 ms. Trace:
I20260709 13:21:03.376701 15525 raft_consensus.cc:2468] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:21:03.376933 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:03.377780 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:03.378304 15527 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:03.373736 15668 rpcz_store.cc:276] 0709 13:21:00.319684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.320344 (+   660us) service_pool.cc:224] Handling call
0709 13:21:00.971995 (+651651us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:00.978622 (+  6627us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:00.978651 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:00.992067 (+ 13416us) log.cc:844] Serialized 26 byte log entry
0709 13:21:01.391726 (+399659us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:01.391768 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:01.391848 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:02.393041 (+1001193us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b2400209318. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce17a49c4 00007f5ce17a2acb 00007f5ce195ddc9 00007f5ce195da55 00007f5ce195d09d 00007f5ce458f10e 00007f5ce45a688c 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221
0709 13:21:02.395210 (+  2169us) raft_consensus.cc:1685] finished
0709 13:21:02.395230 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:03.367408 (+972178us) spinlock_profiling.cc:243] Waited 651 ms on lock 0x7b540046da50. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:21:03.367520 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6311,"raft.queue_time_us":6786,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":11840,"spinlock_wait_cycles":1501949952,"thread_start_us":351344,"threads_started":1,"wal-append.queue_time_us":364944}
W20260709 13:21:03.381319 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 178686us
W20260709 13:21:03.348992 21913 log.cc:927] Time spent T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 1.972s	user 0.011s	sys 0.012s
I20260709 13:21:03.389520 15676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 3066) took 2497 ms. Trace:
I20260709 13:21:03.419075 15670 raft_consensus.cc:2468] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
I20260709 13:21:03.420030 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:03.420630 15525 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:03.390194 15676 rpcz_store.cc:276] 0709 13:21:00.891636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.981073 (+ 89437us) service_pool.cc:224] Handling call
0709 13:21:00.981362 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:00.985993 (+  4631us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:00.986022 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:00.986469 (+   447us) log.cc:844] Serialized 26 byte log entry
0709 13:21:01.075158 (+ 88689us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:02.895152 (+1819994us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b1800491b60. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191b455 00007f5ce84033c7 00007f5ce4a159a7 00007f5ce4a250bf 00007f5ce4a25038
0709 13:21:03.047371 (+152219us) log.cc:844] Serialized 14 byte log entry
0709 13:21:03.047576 (+   205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:03.047650 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:03.355729 (+308079us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b2400209318. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce17a49c4 00007f5ce17a2acb 00007f5ce195ddc9 00007f5ce195da55 00007f5ce195d09d 00007f5ce458f10e 00007f5ce45a688c 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221
0709 13:21:03.386315 (+ 30586us) raft_consensus.cc:1685] finished
0709 13:21:03.386338 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:03.389436 (+  3098us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82249,"prepare.queue_time_us":298827,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":5956,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":7143,"spinlock_wait_cycles":40633472,"thread_start_us":240457,"threads_started":2,"wal-append.queue_time_us":169172}
I20260709 13:21:03.420719 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.421810 15670 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
W20260709 13:21:03.383510 21992 log.cc:927] Time spent T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:21:03.428864 21708 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:03.278309 21822 consensus_queue.cc:237] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:03.374629 21849 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.283687 21854 consensus_queue.cc:237] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:03.419137 21988 log.cc:927] Time spent T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
I20260709 13:21:03.441110 15673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49094 (request call id 2537) took 2707 ms. Trace:
I20260709 13:21:03.342518 21965 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:03.385526 21911 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:03.437701 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:03.452193 15598 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:03.386026 21726 raft_consensus.cc:697] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:03.445399 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:03.441566 15673 rpcz_store.cc:276] 0709 13:21:00.733900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.798525 (+ 64625us) service_pool.cc:224] Handling call
0709 13:21:00.798880 (+   355us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:00.812841 (+ 13961us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:00.812876 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:00.813483 (+   607us) log.cc:844] Serialized 52 byte log entry
0709 13:21:01.001557 (+188074us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:01.049949 (+ 48392us) log.cc:844] Serialized 14 byte log entry
0709 13:21:01.152841 (+102892us) log.cc:844] Serialized 14 byte log entry
0709 13:21:01.153062 (+   221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:01.153132 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:01.508810 (+355678us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b180032efc0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce191e522 00007f5ce19207c1 00007f5ce1920779 00007f5ce1920709 00007f5ce19206d1 00007f5ce191f9cd 000000000038f354 000000000038f199 00007f5ce190183a
0709 13:21:03.287028 (+1778218us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b2400209318. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce17a49c4 00007f5ce17a2acb 00007f5ce195ddc9 00007f5ce195da55 00007f5ce195d09d 00007f5ce458f10e 00007f5ce45a688c 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221
0709 13:21:03.440837 (+153809us) raft_consensus.cc:1685] finished
0709 13:21:03.440863 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:03.441038 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":86606,"prepare.queue_time_us":505561,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":7742,"raft.queue_time_us":8027,"raft.run_cpu_time_us":354,"raft.run_wall_time_us":13202,"spinlock_wait_cycles":53364992,"thread_start_us":161104,"threads_started":1,"wal-append.queue_time_us":290353}
I20260709 13:21:03.431605 21769 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:03.437307 21858 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:03.465430 21726 consensus_queue.cc:237] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:03.468403 21990 log.cc:927] Time spent T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 13:21:03.469542 21916 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:03.470768 15601 raft_consensus.cc:2393] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 16: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
W20260709 13:21:03.471760 21995 log.cc:927] Time spent T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:21:03.473605 21916 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:03.474048 21739 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:03.479596 21659 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:03.480635 21989 log.cc:927] Time spent T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 13:21:03.482017 15674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 3078) took 1040 ms. Trace:
I20260709 13:21:03.448516 21737 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:03.482460 15674 rpcz_store.cc:276] 0709 13:21:02.441497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:02.452646 (+ 11149us) service_pool.cc:224] Handling call
0709 13:21:02.452971 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:02.498616 (+ 45645us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:02.498647 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:02.499464 (+   817us) log.cc:844] Serialized 26 byte log entry
0709 13:21:02.995831 (+496367us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b1800491860. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab0d1 00007f5ce49c6a0f
0709 13:21:03.122202 (+126371us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:03.122241 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:03.122304 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:03.481676 (+359372us) raft_consensus.cc:1685] finished
0709 13:21:03.481697 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:03.481947 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44555,"raft.queue_time_us":44376,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":4456,"spinlock_wait_cycles":4414848,"thread_start_us":126177,"threads_started":1,"wal-append.queue_time_us":698516}
I20260709 13:21:03.440963 21849 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:03.488010 21849 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:03.491637 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.493042 15676 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:21:03.493816 15598 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 17.
I20260709 13:21:03.495855 21856 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:03.496477 21737 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 25 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:03.431392 21649 consensus_queue.cc:237] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:03.501557 15669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 3072) took 1821 ms. Trace:
I20260709 13:21:03.503338 15669 rpcz_store.cc:276] 0709 13:21:01.680106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:01.820470 (+140364us) service_pool.cc:224] Handling call
0709 13:21:01.822194 (+  1724us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:01.912887 (+ 90693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:01.912917 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:01.912959 (+    42us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:01.912987 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:03.501286 (+1588299us) spinlock_profiling.cc:243] Waited 90.5 ms on lock 0x7b5400429354. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:21:03.501341 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:03.501487 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208084864}
W20260709 13:21:03.506572 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 162178us
I20260709 13:21:03.504894 21946 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:03.450700 21708 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:03.510690 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.456945 21911 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:03.520776 16205 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:03.515488 21708 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:03.550102 21949 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:21:03.553948 21965 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:03.555807 21957 raft_consensus.cc:3055] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:21:03.556322 21957 raft_consensus.cc:740] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:03.558025 21957 consensus_queue.cc:260] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:03.611433 15420 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58802 (request call id 262) took 31672 ms (client timeout 15000 ms). Trace:
W20260709 13:21:03.620327 15420 rpcz_store.cc:269] 0709 13:20:31.938950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:31.940850 (+  1900us) service_pool.cc:224] Handling call
0709 13:21:03.611099 (+31670249us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:55.625524 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:55.678548 (+ 53024us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:55.678611 (+    63us) write_op.cc:435] Acquired schema lock
0709 13:20:55.678629 (+    18us) tablet.cc:662] Decoding operations
0709 13:20:55.914397 (+235768us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:55.914493 (+    96us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:55.914521 (+    28us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:20:55.915716 (+  1195us) tablet.cc:701] Row locks acquired
0709 13:20:55.915773 (+    57us) write_op.cc:260] PREPARE: finished
0709 13:20:55.915957 (+   184us) write_op.cc:270] Start()
0709 13:20:55.916096 (+   139us) write_op.cc:276] Timestamp: P: 1783603255915890 usec, L: 0
0709 13:20:55.916131 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 13:20:56.002343 (+ 86212us) log.cc:844] Serialized 5090 byte log entry
0709 13:20:56.602669 (+600326us) spinlock_profiling.cc:243] Waited 3.13 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:20:57.046619 (+443950us) op_driver.cc:464] REPLICATION: finished
0709 13:20:57.839125 (+792506us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b5400369554. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a06e28 00007f5ce8441581 00007f5ce8440d58 00007f5ce84459f1 00007f5ce84459a9 00007f5ce8445939 00007f5ce8445901 00007f5ce8444b2d 000000000038f354 000000000038f199
0709 13:20:57.990732 (+151607us) write_op.cc:301] APPLY: starting
0709 13:20:58.003255 (+ 12523us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:58.522995 (+519740us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:58.523039 (+    44us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:58.992181 (+469142us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:59.006217 (+ 14036us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=0,mrs_lookups=0
0709 13:20:59.006283 (+    66us) write_op.cc:312] APPLY: finished
0709 13:20:59.006696 (+   413us) log.cc:844] Serialized 121 byte log entry
0709 13:20:59.347920 (+341224us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:59.348115 (+   195us) write_op.cc:454] Released schema lock
0709 13:20:59.348200 (+    85us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:59.348677 (+   477us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:59.348711 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":680841,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":876050,"apply.run_cpu_time_us":58904,"apply.run_wall_time_us":1381880,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58571,"num_ops":13,"prepare.queue_time_us":206655,"prepare.run_cpu_time_us":34101,"prepare.run_wall_time_us":2213845,"raft.queue_time_us":353769,"raft.run_cpu_time_us":60279,"raft.run_wall_time_us":1004132,"replication_time_us":1064511,"spinlock_wait_cycles":69525632,"thread_start_us":1661954,"threads_started":5,"wal-append.queue_time_us":819363,"wal-append.run_cpu_time_us":17809,"wal-append.run_wall_time_us":2449077}]]}
I20260709 13:21:03.618330 15423 catalog_manager.cc:5697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:03.453727 15525 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:21:03.636962 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:03.639211 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:03.640903 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:03.661141 15523 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
W20260709 13:21:03.661695 15671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57710 (request call id 1893) took 3012 ms (client timeout 1804 ms). Trace:
I20260709 13:21:03.660295 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:21:03.651317 15597 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.014s	sys 0.010s
W20260709 13:21:03.667213 15597 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.014s	sys 0.011s
I20260709 13:21:03.652793 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:03.669623 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.673669 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.667248 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.657903 15641 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:03.662525 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:21:03.662268 15671 rpcz_store.cc:269] 0709 13:21:00.649437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.649612 (+   175us) service_pool.cc:224] Handling call
0709 13:21:03.661196 (+3011584us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b540042bb54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:21:03.661411 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3782045056}
W20260709 13:21:03.660235 15602 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.010s	sys 0.018s
I20260709 13:21:03.675298 15670 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 16.
W20260709 13:21:03.678480 15602 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.010s	sys 0.019s
I20260709 13:21:03.661437 15525 raft_consensus.cc:2393] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 19: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:21:03.676398 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.686416 15669 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 19.
I20260709 13:21:03.687628 15641 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:03.647786 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:03.685068 15523 raft_consensus.cc:1275] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:21:03.694636 15519 raft_consensus.cc:2393] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 15: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:21:03.684638 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:03.637666 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:03.640903 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:03.697008 15520 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:21:03.697594 15524 raft_consensus.cc:2410] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for term 16 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
I20260709 13:21:03.695463 15527 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 17.
W20260709 13:21:03.700270 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 179459us
I20260709 13:21:03.704057 16205 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:21:03.706609 16204 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:03.706882 15677 raft_consensus.cc:1275] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:21:03.704474 16203 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:03.708499 15670 raft_consensus.cc:1240] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:03.709872 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:03.711139 15676 raft_consensus.cc:2393] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 16: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
W20260709 13:21:03.715895 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:03.676389 15673 raft_consensus.cc:2393] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 18: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:21:03.718220 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:03.720012 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:03.733644 15597 raft_consensus.cc:2393] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 25: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:21:03.723587 21304 consensus_queue.cc:1059] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:21:03.734761 21304 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:03.735179 21304 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:03.735201 21369 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:21:03.735805 21304 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:03.724717 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:03.678339 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:03.722501 21334 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:03.738519 21561 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:03.739266 21561 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:03.740165 21334 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:03.738549 15668 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:03.743554 21561 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:03.744809 21561 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.746474 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.747390 15670 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:21:03.748937 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.750705 21439 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:03.751689 21439 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:03.752418 21439 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:03.755882 21439 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:03.758430 21439 raft_consensus.cc:3055] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:21:03.758947 21439 raft_consensus.cc:740] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:03.760226 21439 consensus_queue.cc:260] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:03.760392 21304 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:03.762147 21439 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:03.765178 15668 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 25.
I20260709 13:21:03.745661 21561 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:03.754539 15423 catalog_manager.cc:5697] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:21:03.770336 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 106257us
I20260709 13:21:03.779742 21304 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:03.781698 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.782999 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.784444 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.785218 21658 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:03.786168 21659 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:21:03.786774 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:03.788697 21704 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:03.789896 16231 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 18.
I20260709 13:21:03.791994 21721 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:03.751132 21369 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:21:03.847735 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:03.848240 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:03.850430 21708 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:03.566299 21957 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:03.852660 21658 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:03.858083 21513 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.841100 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.858503 15597 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 15.
I20260709 13:21:03.859906 21911 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:03.859583 21721 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:03.860853 21765 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:03.862483 15670 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
W20260709 13:21:03.893641 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:03.894994 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.874047 21369 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:03.894524 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:03.899785 15642 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:21:03.907981 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 199631us
I20260709 13:21:03.906592 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:21:03.912423 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 200283us
I20260709 13:21:03.901943 21369 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:03.896085 21704 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:21:03.935621 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 231477us
I20260709 13:21:03.915902 21708 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:03.944250 21708 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
W20260709 13:21:03.951033 15672 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.054s	sys 0.045s
W20260709 13:21:03.960566 15569 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.011s	sys 0.003s
I20260709 13:21:03.911130 21513 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:03.960961 15569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.011s	sys 0.003s
I20260709 13:21:03.956182 21708 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:03.964870 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 49552.1us
W20260709 13:21:03.964426 15672 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.055s	sys 0.045s
I20260709 13:21:03.976220 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:03.977118 16232 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:03.978466 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:03.979343 16234 raft_consensus.cc:2393] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 16: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:21:03.979660 21561 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:03.977667 21848 consensus_meta.cc:287] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4: Time spent flushing consensus metadata: real 0.938s	user 0.000s	sys 0.015s
W20260709 13:21:03.988003 21636 consensus_meta.cc:287] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.816s	user 0.006s	sys 0.011s
I20260709 13:21:03.982528 21848 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:03.958628 21704 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:03.989774 16232 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 25.
I20260709 13:21:03.991281 21704 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:03.992738 21658 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:03.995946 15568 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:03.999611 21721 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:03.962842 21993 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.002259 15674 raft_consensus.cc:2393] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 10: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:21:04.002676 21954 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:03.988533 21636 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:03.951658 21765 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:03.966468 15602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 2636) took 1015 ms. Trace:
I20260709 13:21:04.011827 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:03.966332 16203 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:04.014882 15568 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:04.015295 21765 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:04.010758 15602 rpcz_store.cc:276] 0709 13:21:02.951409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:02.951570 (+   161us) service_pool.cc:224] Handling call
0709 13:21:02.951847 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:02.952028 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:21:03.679260 (+727232us) log.cc:844] Serialized 14 byte log entry
0709 13:21:03.936249 (+256989us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:03.942256 (+  6007us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:21:03.952105 (+  9849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:03.959131 (+  7026us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:03.966409 (+  7278us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5510,"prepare.queue_time_us":470954,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":2225,"raft.queue_time_us":24094,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":12359,"thread_start_us":960613,"threads_started":2,"wal-append.queue_time_us":286626}
I20260709 13:21:04.019294 21636 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:04.020246 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:04.017421 15521 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:21:04.041271 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:04.007722 21954 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:04.042204 15676 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 18.
I20260709 13:21:04.042548 21954 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:04.044826 21726 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:21:04.045176 21513 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:04.025586 15566 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:04.045660 15641 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:21:04.046507 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 84992.5us
I20260709 13:21:04.048461 15566 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:21:04.032784 16204 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:04.049389 21659 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:21:04.038554 21966 consensus_meta.cc:287] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.719s	user 0.006s	sys 0.006s
W20260709 13:21:04.001361 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 520569us
I20260709 13:21:04.049484 22013 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:04.059219 21659 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:04.057870 21916 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:04.084053 15566 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:04.077876 21983 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.085268 21983 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.086077 21636 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:04.086622 21636 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.087669 21983 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:04.087888 22012 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:04.088627 21636 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:04.055780 21993 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:04.088781 21855 consensus_queue.cc:1048] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:21:04.084337 21339 consensus_queue.cc:1059] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:04.090816 21339 raft_consensus.cc:3055] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:04.091324 21339 raft_consensus.cc:740] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:04.058652 22005 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.094048 22005 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:04.070019 21513 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:04.098111 22019 raft_consensus.cc:2749] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:04.082377 21916 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:04.068864 21726 raft_consensus.cc:697] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 25 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:04.083132 21659 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:04.107086 21916 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:04.114651 21726 consensus_queue.cc:237] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.098953 22005 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:04.122619 15670 raft_consensus.cc:1240] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:04.125136 15671 raft_consensus.cc:1275] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:21:04.129263 21439 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:04.129484 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:04.129982 21439 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:04.130743 15521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57310 (request call id 2556) took 1032 ms. Trace:
I20260709 13:21:04.131163 15521 rpcz_store.cc:276] 0709 13:21:03.098848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:03.107996 (+  9148us) service_pool.cc:224] Handling call
0709 13:21:04.130570 (+1022574us) spinlock_profiling.cc:243] Waited 908 ms on lock 0x7b5400023d54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:21:04.130690 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2088835328}
I20260709 13:21:04.132119 15491 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:04.138522 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:21:04.141335 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 131687us
I20260709 13:21:04.144644 21954 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.147409 21954 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 18 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:04.147944 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 189641us
I20260709 13:21:04.093469 21339 consensus_queue.cc:260] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.158624 16205 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:04.159148 15675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57710 (request call id 1916) took 1165 ms. Trace:
I20260709 13:21:04.132222 21439 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:04.155138 21911 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:04.067734 21966 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:04.173350 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 66644.6us
I20260709 13:21:04.182631 16203 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:04.184141 16203 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:04.185057 21334 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:04.185971 21334 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:21:04.185709 16203 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:04.187529 16203 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:04.141415 21848 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:04.174530 15675 rpcz_store.cc:276] 0709 13:21:02.993900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:03.015019 (+ 21119us) service_pool.cc:224] Handling call
0709 13:21:04.158053 (+1143034us) spinlock_profiling.cc:243] Waited 36.4 ms on lock 0x7b5400440fa0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4982dc0 00007f5ce4a0ee18 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:21:04.158819 (+   766us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540026b6d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:21:04.158971 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2589254784}
I20260709 13:21:04.193692 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:04.194804 15598 raft_consensus.cc:2393] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 16: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:21:04.195382 21513 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:04.193573 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:04.195897 21513 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:04.196362 21513 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:04.198014 21334 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:04.194859 16203 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:04.200675 16203 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:04.202230 16204 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:04.202227 21334 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:04.207324 21334 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:04.207842 21334 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:04.212626 21513 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.213707 15603 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:04.214735 21993 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:04.215150 21659 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:04.216380 15673 raft_consensus.cc:1240] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:04.216563 15672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49100 (request call id 3081) took 1770 ms. Trace:
I20260709 13:21:04.180809 21911 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:04.216936 15672 rpcz_store.cc:276] 0709 13:21:02.445860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:03.365074 (+919214us) service_pool.cc:224] Handling call
0709 13:21:03.365410 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:03.371771 (+  6361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:03.371800 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:03.372254 (+   454us) log.cc:844] Serialized 26 byte log entry
0709 13:21:04.021008 (+648754us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:21:04.076210 (+ 55202us) log.cc:844] Serialized 14 byte log entry
0709 13:21:04.095853 (+ 19643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:04.095931 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:04.216263 (+120332us) raft_consensus.cc:1685] finished
0709 13:21:04.216287 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:04.216476 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9514,"raft.queue_time_us":46645,"raft.run_cpu_time_us":351,"raft.run_wall_time_us":28118,"thread_start_us":684602,"threads_started":2,"wal-append.queue_time_us":772379}
I20260709 13:21:04.222404 21658 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:04.222910 16204 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:04.222954 21655 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:04.223587 21655 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:04.224030 21655 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:04.225099 16204 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:04.226256 22020 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:04.227460 21339 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:04.228883 22018 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:04.230046 15603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 1746) took 1133 ms. Trace:
I20260709 13:21:04.230469 15603 rpcz_store.cc:276] 0709 13:21:03.096210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:03.096373 (+   163us) service_pool.cc:224] Handling call
0709 13:21:03.096679 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:04.214247 (+1117568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:04.229712 (+ 15465us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540016bf54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:21:04.229975 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2568062208}
I20260709 13:21:04.232074 16204 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:04.232808 21911 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:04.233124 21952 raft_consensus.cc:2804] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:21:04.233068 21655 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.233789 21952 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:04.234261 21952 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:21:04.223856 16203 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:04.246918 16203 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:04.247449 21964 raft_consensus.cc:2804] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:04.248153 21964 raft_consensus.cc:697] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:04.248824 21925 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:04.250610 21966 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:04.250558 21925 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:04.251214 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:04.223191 21658 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:04.251518 21925 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:04.247509 22018 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:21:04.252496 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 93989.5us
I20260709 13:21:04.254137 16205 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:04.254851 16237 raft_consensus.cc:1275] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:21:04.255981 15491 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:04.256918 15491 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:04.257778 21993 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:04.258204 21993 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:04.258584 21993 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:04.258976 16236 raft_consensus.cc:1240] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:04.260504 16237 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:21:04.260730 21925 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.267757 21925 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:04.249195 21964 consensus_queue.cc:237] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.262461 21369 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.262943 21983 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:04.262876 16204 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:21:04.263538 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 44820.3us
I20260709 13:21:04.264201 21649 consensus_queue.cc:1048] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:21:04.258013 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 58908.6us
I20260709 13:21:04.270359 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:04.271289 21916 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:21:04.252415 22018 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.249122 16203 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:04.276849 16232 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 18.
I20260709 13:21:04.262636 21721 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:04.276304 21855 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:04.284572 22019 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:04.277897 22011 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
W20260709 13:21:04.278394 15640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.285082 21708 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:04.277333 22012 consensus_queue.cc:1059] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:04.285969 22019 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:04.286451 22012 raft_consensus.cc:3055] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:04.286844 22012 raft_consensus.cc:740] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:04.287499 22018 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:04.276579 21966 consensus_queue.cc:1059] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 13:21:04.283002 15493 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:04.283598 21369 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:04.284137 21983 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.286432 22019 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:04.214854 21513 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:04.235565 21655 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:04.294955 15568 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:04.284039 15567 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:04.298413 16234 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 19.
I20260709 13:21:04.297701 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:04.299618 15603 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:04.288164 21708 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:04.303282 15568 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:21:04.304219 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 50172.8us
I20260709 13:21:04.306675 16205 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:04.307199 21983 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:04.309607 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 52758.6us
I20260709 13:21:04.302261 21925 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:04.310127 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:04.293593 22011 consensus_queue.cc:1048] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
W20260709 13:21:04.323215 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 74212us
I20260709 13:21:04.315369 22018 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:04.326001 21934 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:04.329864 21916 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:21:04.330598 21911 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:04.332798 21911 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:04.334183 21649 raft_consensus.cc:3055] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:04.334640 21649 raft_consensus.cc:740] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:04.287703 22012 consensus_queue.cc:260] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:04.335472 15493 leader_election.cc:400] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 16. Current term is 17.
I20260709 13:21:04.336674 15567 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:04.368630 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:04.371300 15603 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 16.
I20260709 13:21:04.318517 21855 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:04.370361 15491 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:04.335378 21659 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:04.322907 21966 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:04.336548 21513 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:04.334115 22018 raft_consensus.cc:697] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:04.322861 21946 raft_consensus.cc:2749] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:04.390347 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:04.391258 16234 raft_consensus.cc:2393] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 17: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:21:04.367961 21934 raft_consensus.cc:697] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:04.379841 21911 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:04.397951 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:04.398141 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:04.390029 21954 consensus_queue.cc:1059] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:21:04.400781 15672 raft_consensus.cc:2393] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 18: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:21:04.402355 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:04.402668 15671 raft_consensus.cc:1240] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 15. Current term is 16. Ops: []
W20260709 13:21:04.405886 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.335573 21649 consensus_queue.cc:260] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:04.414391 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 86356.8us
I20260709 13:21:04.418488 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:04.418671 21788 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:21:04.419848 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:04.398761 21855 raft_consensus.cc:697] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:04.420984 21855 consensus_queue.cc:237] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.423323 15642 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:21:04.424456 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:04.403919 22011 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:21:04.427563 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 92355.7us
I20260709 13:21:04.428486 15493 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:04.428879 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:04.429410 22018 consensus_queue.cc:237] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.428530 21708 raft_consensus.cc:2804] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:04.430569 21708 raft_consensus.cc:697] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:04.431432 21708 consensus_queue.cc:237] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:04.434183 15596 raft_consensus.cc:1240] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:04.436122 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:04.435794 21934 consensus_queue.cc:237] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.436697 15674 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:04.425748 21954 raft_consensus.cc:3055] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:04.437282 21954 raft_consensus.cc:740] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:04.438195 21954 consensus_queue.cc:260] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:04.401503 21911 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:04.439453 16203 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:21:04.449867 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 49997.2us
I20260709 13:21:04.447175 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:04.452018 16234 raft_consensus.cc:2393] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 25: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:21:04.406466 21659 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:04.408053 22012 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:04.462255 15642 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:21:04.449848 15567 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 2.713s (SENT) [suppressed 10 similar messages]
I20260709 13:21:04.467657 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:04.469018 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:04.447966 21911 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:04.466689 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:04.476925 21949 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:04.486671 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:04.466735 21636 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:04.502427 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:04.456955 21946 consensus_queue.cc:1059] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:21:04.508430 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:21:04.508669 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 101765us
I20260709 13:21:04.474987 21659 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.501812 21954 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:04.502117 21649 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:04.508562 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:04.500015 21964 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:04.519156 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:21:04.520856 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 70532.3us
I20260709 13:21:04.521220 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:21:04.521425 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 93105.7us
I20260709 13:21:04.504480 21949 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.526660 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:04.527344 16232 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 18.
I20260709 13:21:04.528479 21949 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:04.528818 16232 raft_consensus.cc:1240] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:04.530246 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:04.539985 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:04.499115 21334 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.542290 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:21:04.543905 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.549537 15673 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 21.
I20260709 13:21:04.551463 15524 raft_consensus.cc:2393] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 12: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:21:04.552189 16234 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:21:04.552404 15642 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:04.553960 21304 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:04.555269 21334 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:04.555521 16235 raft_consensus.cc:1240] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:04.558665 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:04.558931 16239 raft_consensus.cc:1275] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:21:04.559667 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:04.560544 21636 raft_consensus.cc:3055] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:04.561039 21822 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:21:04.561489 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:04.562031 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:04.562239 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:04.562845 15602 raft_consensus.cc:2376] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 17. Re-sending same reply.
I20260709 13:21:04.563292 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:04.563877 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:04.564160 21957 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:04.564868 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:04.565843 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:04.567360 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:04.568176 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:04.560973 21636 raft_consensus.cc:740] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:21:04.568540 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:21:04.583076 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 46514.4us
I20260709 13:21:04.582815 21636 consensus_queue.cc:260] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:04.584030 16203 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:04.584329 21636 raft_consensus.cc:3060] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:04.587723 22034 log.cc:927] Time spent T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 13:21:04.589178 22031 log.cc:927] Time spent T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:21:04.591269 21952 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.593640 21952 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:04.594290 16238 raft_consensus.cc:2393] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 10: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:21:04.600191 21513 raft_consensus.cc:2749] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:04.539409 21964 raft_consensus.cc:697] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:04.619319 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:04.627768 21993 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:04.619894 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:04.622012 21369 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:04.623430 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 75056.1us
W20260709 13:21:04.631825 22038 log.cc:927] Time spent T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:21:04.632519 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:04.632709 15598 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:21:04.632843 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:04.633435 15521 raft_consensus.cc:2410] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 14 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:21:04.626405 15674 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:21:04.628608 21964 consensus_queue.cc:237] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.634382 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:04.634955 15521 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 16.
W20260709 13:21:04.635107 22041 log.cc:927] Time spent T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:21:04.637605 15668 raft_consensus.cc:1240] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 13. Current term is 14. Ops: []
W20260709 13:21:04.639902 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 101333us
I20260709 13:21:04.643726 16231 raft_consensus.cc:2393] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 21: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:21:04.645658 15566 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:21:04.646937 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:04.651891 21983 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:04.653558 15601 raft_consensus.cc:2393] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 16: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:21:04.653957 16235 raft_consensus.cc:1240] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:04.654439 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:04.651904 21339 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:04.657040 21339 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:04.657511 21339 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:21:04.657691 22037 log.cc:927] Time spent T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.077s	user 0.001s	sys 0.004s
I20260709 13:21:04.658257 21659 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:04.657317 16236 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:21:04.661096 16236 raft_consensus.cc:606] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:04.661427 16236 raft_consensus.cc:614] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:04.656960 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:04.653680 21914 consensus_queue.cc:1059] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:21:04.665172 21794 raft_consensus.cc:3055] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:21:04.665561 21794 raft_consensus.cc:740] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:04.665935 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:04.666391 21794 consensus_queue.cc:260] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.655915 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:04.621512 22019 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:04.695399 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 48209.6us
I20260709 13:21:04.646322 21993 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:04.634618 21911 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.708639 16233 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 18. Current term is 19. Ops: []
W20260709 13:21:04.713609 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 145274us
I20260709 13:21:04.700815 21822 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:04.718796 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:04.719326 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:04.718230 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:04.721858 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:04.725028 15520 raft_consensus.cc:3060] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:21:04.725446 15522 raft_consensus.cc:1240] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:21:04.657184 21983 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.729938 15491 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:04.733789 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a66a9fb8df432fb0d1d7ba3756235e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:21:04.732465 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 137735us
I20260709 13:21:04.707880 21957 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:04.731050 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:04.738122 15519 raft_consensus.cc:2393] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 18: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:21:04.741345 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:04.744855 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:04.724597 21822 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:04.786716 21822 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:04.784366 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:04.704083 21339 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:04.785290 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 46231.7us
I20260709 13:21:04.794620 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:04.743598 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:04.794404 15674 raft_consensus.cc:1275] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:21:04.796223 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 55163.6us
I20260709 13:21:04.731556 21659 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:04.802147 22044 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:04.835335 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
W20260709 13:21:04.806231 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 91139.4us
I20260709 13:21:04.808073 21983 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:04.813283 22011 consensus_queue.cc:1059] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:04.841377 22044 raft_consensus.cc:697] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:21:04.737933 21304 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.083s
I20260709 13:21:04.842167 21911 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:04.842417 22044 consensus_queue.cc:237] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:04.843510 22019 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:04.832535 21957 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.845383 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:04.821892 15520 raft_consensus.cc:2468] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 10.
I20260709 13:21:04.838503 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:04.847616 15594 raft_consensus.cc:2468] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 19.
I20260709 13:21:04.839648 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:04.848893 15674 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:21:04.840356 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:04.743916 21949 consensus_queue.cc:1048] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.097s
I20260709 13:21:04.840976 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:04.851768 15672 raft_consensus.cc:2393] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 10: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:21:04.851698 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.848214 15528 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 16.
I20260709 13:21:04.846829 16205 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:04.861479 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:04.864804 16203 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:04.866340 16203 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:04.868315 15677 raft_consensus.cc:1275] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:21:04.873689 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:04.869930 16203 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:04.877838 16203 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:04.861858 22011 raft_consensus.cc:3055] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:04.887544 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:04.887972 16203 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:04.875649 22019 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:21:04.867939 21788 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:04.878897 21369 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:21:04.892282 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 45573.8us
I20260709 13:21:04.895210 15523 raft_consensus.cc:1275] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:21:04.884320 21513 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:04.895759 16205 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:04.896821 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:04.898746 15521 raft_consensus.cc:2393] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 14: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:21:04.885891 22011 raft_consensus.cc:740] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:04.900876 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:04.905694 22011 consensus_queue.cc:260] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.907986 21794 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:21:04.911201 22053 log.cc:927] Time spent T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 13:21:04.911365 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:04.912036 15598 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:04.912945 22052 log.cc:927] Time spent T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 13:21:04.891815 21925 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:04.891872 22018 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:04.941968 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:04.898838 21513 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:04.916040 21339 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 21 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:04.929006 21957 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:04.929656 22020 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:21:04.961688 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 73775.3us
I20260709 13:21:04.970543 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:21:04.975883 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 95551.8us
I20260709 13:21:04.960002 21513 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:21:04.983283 22051 log.cc:927] Time spent T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.284s	user 0.000s	sys 0.001s
W20260709 13:21:04.987010 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 72557.4us
W20260709 13:21:04.988780 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.989145 21954 consensus_queue.cc:1048] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:21:04.993594 22061 consensus_queue.cc:1059] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:21:04.994683 21916 raft_consensus.cc:3055] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:04.995059 21916 raft_consensus.cc:740] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:04.995853 21916 consensus_queue.cc:260] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:04.996534 21822 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:04.997633 21916 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:05.002403 15668 raft_consensus.cc:2393] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 19: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:21:05.003396 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:04.989810 15567 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:05.011160 15524 raft_consensus.cc:2468] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 16.
I20260709 13:21:05.012724 16204 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:05.014151 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:05.003316 22062 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.009102 21788 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.018759 22058 consensus_queue.cc:1048] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.009s
I20260709 13:21:05.019061 22055 consensus_queue.cc:1048] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:21:05.019310 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:21:05.007809 22049 log.cc:927] Time spent T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.122s	user 0.000s	sys 0.007s
W20260709 13:21:05.020828 22039 log.cc:927] Time spent T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.320s	user 0.002s	sys 0.000s
I20260709 13:21:05.022369 15598 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 19.
I20260709 13:21:05.023546 22013 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:05.028518 22020 raft_consensus.cc:2804] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:21:05.027767 21788 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:05.030251 22062 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:05.032016 21788 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:05.032547 22062 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:05.029173 22020 raft_consensus.cc:697] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:05.034816 22020 consensus_queue.cc:237] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:05.035954 21822 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:05.036963 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:21:05.039505 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:05.039990 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:05.013113 21946 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.039163 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:05.040889 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:05.044991 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:05.044868 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:05.045794 15519 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:21:05.046406 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:05.044024 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:05.050397 15676 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 25 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:21:05.049791 21946 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:05.051993 22054 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 13:21:05.054415 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 64711us
I20260709 13:21:05.055305 15567 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:05.060953 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:05.060953 15670 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 21.
I20260709 13:21:05.063381 15566 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:21:05.066409 16240 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:21:05.067046 16240 raft_consensus.cc:606] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:05.067381 16240 raft_consensus.cc:614] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:05.068503 22012 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:05.070258 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:05.074684 15673 raft_consensus.cc:1240] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:05.078944 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:05.085762 16204 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:05.087198 22019 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:05.087627 22019 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:05.087999 22019 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:05.085139 21954 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:05.094882 16205 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:05.097760 21954 raft_consensus.cc:697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:05.140468 21954 consensus_queue.cc:237] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:05.143164 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 50060.4us
I20260709 13:21:05.144147 21513 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:05.052503 21946 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:05.148998 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:05.149682 16240 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 19.
I20260709 13:21:05.151858 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:05.152669 15676 raft_consensus.cc:2393] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 14: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
W20260709 13:21:05.154192 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 75894.5us
I20260709 13:21:05.156342 15567 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:05.156427 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:05.157807 15567 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:05.158968 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:05.160303 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:05.161132 15671 raft_consensus.cc:2410] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:21:05.165130 15596 raft_consensus.cc:1240] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:05.165650 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:05.166466 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:05.166484 15594 raft_consensus.cc:1240] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:05.168068 21334 consensus_queue.cc:1059] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:21:05.169358 22066 raft_consensus.cc:3055] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:05.169720 22066 raft_consensus.cc:740] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:21:05.167820 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 70711.3us
I20260709 13:21:05.176201 15603 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:21:05.176596 15493 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:21:05.184638 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 89842.2us
I20260709 13:21:05.185606 16205 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:05.193244 16203 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:05.195722 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:05.195997 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:21:05.196965 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 90303us
I20260709 13:21:05.197908 15493 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:05.203032 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:05.209343 16231 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:05.190237 22044 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.190706 21855 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:05.214809 15641 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:05.100481 21934 consensus_queue.cc:1059] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.101s
I20260709 13:21:05.195122 22013 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:05.186365 21788 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:21:05.200177 21993 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:21:05.243600 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 85904.1us
I20260709 13:21:05.220706 21369 raft_consensus.cc:2804] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:05.231393 21721 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:05.260951 15600 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:05.261291 22064 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:05.261804 22012 raft_consensus.cc:3055] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:05.262240 22012 raft_consensus.cc:740] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:05.263038 22012 consensus_queue.cc:260] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:05.264293 22012 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:05.269701 16231 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 19.
W20260709 13:21:05.255215 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 62058.1us
I20260709 13:21:05.257340 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:05.272910 21946 consensus_queue.cc:1059] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:05.273551 15527 raft_consensus.cc:2393] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 12: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:21:05.228204 21561 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:05.274039 21946 raft_consensus.cc:3055] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:05.274449 21946 raft_consensus.cc:740] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:21:05.274505 21561 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:05.275396 21946 consensus_queue.cc:260] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:05.242226 21957 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:05.276765 21561 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:05.276968 21925 raft_consensus.cc:2749] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:05.278414 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:05.241768 22066 consensus_queue.cc:260] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:05.279767 22066 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:05.280732 16231 raft_consensus.cc:1240] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 16. Current term is 17. Ops: []
W20260709 13:21:05.277421 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:05.233026 21855 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:05.242290 21513 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:21:05.290653 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 81694.8us
W20260709 13:21:05.291356 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 105852us
I20260709 13:21:05.302501 15567 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:05.304979 21914 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:21:05.306088 21369 raft_consensus.cc:697] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:05.307096 21369 consensus_queue.cc:237] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:05.282820 22011 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:05.288168 21304 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.314379 21304 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:05.314659 22019 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:05.316407 21304 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:05.316552 22019 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:05.236207 22044 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:05.289508 21855 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:05.319360 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:05.319927 15522 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:05.281455 21334 consensus_queue.cc:1175] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3ef186ec12554a1d87814af0a0ace9a4" responder_term: 18 status { last_received { term: 18 index: 13 } last_committed_idx: 12 last_received_current_leader { term: 18 index: 13 } }
I20260709 13:21:05.323803 22044 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:05.291719 21966 consensus_queue.cc:1048] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:21:05.276786 21946 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:05.289710 22064 raft_consensus.cc:697] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:05.294832 15641 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:05.331561 15598 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:21:05.332530 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:05.333328 15522 raft_consensus.cc:2468] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 15.
I20260709 13:21:05.285781 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:05.335616 16204 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:05.346344 15600 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:21:05.347450 15603 raft_consensus.cc:1275] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:21:05.348634 22075 raft_consensus.cc:2804] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:05.349345 22075 raft_consensus.cc:697] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
W20260709 13:21:05.349766 22069 log.cc:927] Time spent T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.075s	user 0.003s	sys 0.000s
I20260709 13:21:05.351039 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:05.350525 22075 consensus_queue.cc:237] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:05.352855 15594 raft_consensus.cc:1275] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:21:05.351720 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:05.332870 21966 consensus_queue.cc:1243] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer d3c4f5bd76e0445795bad5f6dd93ff69 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:21:05.358855 22062 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:05.360615 22018 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:05.362914 22018 consensus_queue.cc:1048] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:21:05.363865 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:05.365087 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:21:05.370335 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 75620.9us
I20260709 13:21:05.373656 15641 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:05.385748 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:05.390234 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:05.390234 15641 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:05.392364 15672 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:21:05.351418 22064 consensus_queue.cc:237] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:05.376449 22062 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:21:05.413751 22068 log.cc:927] Time spent T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.125s	user 0.001s	sys 0.001s
I20260709 13:21:05.427070 15603 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:59198
I20260709 13:21:05.427806 15603 raft_consensus.cc:606] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:05.428179 15603 raft_consensus.cc:614] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:05.443233 15522 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:21:05.444607 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 56275.4us
I20260709 13:21:05.446060 15641 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:05.447381 22058 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:05.465338 15598 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:21:05.467473 15669 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:21:05.470201 16238 raft_consensus.cc:1240] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:21:05.471998 15642 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:05.476570 15598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:21:05.479799 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 219389us
I20260709 13:21:05.502360 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:05.504348 16233 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 25 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
W20260709 13:21:05.504730 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 49636.9us
I20260709 13:21:05.506706 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:05.448010 21957 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:05.508981 16240 raft_consensus.cc:2393] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 21: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
W20260709 13:21:05.512101 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 66122.9us
I20260709 13:21:05.512315 16234 raft_consensus.cc:1240] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:05.521725 15641 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:05.497124 22058 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:05.541220 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:05.547297 21993 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:05.547775 21993 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:05.548138 21993 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:05.562975 15425 catalog_manager.cc:5697] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 11 to 17. New cstate: current_term: 17 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:05.573289 16203 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:05.573182 21925 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:21:05.578970 22018 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:05.579543 22018 raft_consensus.cc:697] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:05.580230 21822 consensus_queue.cc:1048] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:21:05.580451 22018 consensus_queue.cc:237] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:05.515924 21794 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.582480 21794 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:05.583624 22058 raft_consensus.cc:697] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:05.584700 21794 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:05.584595 22058 consensus_queue.cc:237] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:05.585726 21304 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:05.591157 21855 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:05.594159 15600 raft_consensus.cc:1240] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:05.598707 21513 consensus_queue.cc:1059] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:05.514516 21952 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.085s
I20260709 13:21:05.599814 21513 raft_consensus.cc:3055] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:05.600203 21513 raft_consensus.cc:740] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:05.601073 21513 consensus_queue.cc:260] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:05.547580 16233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:21:05.603314 16233 raft_consensus.cc:606] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 LEADER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:05.565011 15425 catalog_manager.cc:5697] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 14 to 15, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 15 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:05.622910 15522 raft_consensus.cc:1275] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:21:05.600942 22019 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 13:21:05.624480 22013 consensus_queue.cc:1059] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:05.625607 16236 raft_consensus.cc:2410] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:21:05.626317 21966 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:05.600263 22078 consensus_queue.cc:1059] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 13:21:05.630501 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:05.634192 16235 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:21:05.641601 15425 catalog_manager.cc:5697] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 10 to 11, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 11 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:05.655043 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:05.655472 15493 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:05.656682 15493 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:21:05.658329 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 158717us
I20260709 13:21:05.655740 16233 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:21:05.661828 15493 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:05.618597 21855 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:05.664079 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 48450.2us
I20260709 13:21:05.669837 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:05.675102 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:05.662909 15425 catalog_manager.cc:5697] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:05.675834 15675 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:21:05.642302 22078 raft_consensus.cc:3055] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:05.679997 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:05.664978 21911 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:05.683043 15519 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:05.665673 21636 consensus_queue.cc:1059] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:21:05.657691 21334 consensus_queue.cc:1048] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:21:05.686481 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:05.730717 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:05.663723 22013 consensus_queue.cc:1243] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer dd1a14c3173d49dc9b55bb116e3783c9 log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:21:05.693818 15568 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:05.694478 21649 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:05.665194 21983 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:05.707026 15522 raft_consensus.cc:1275] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:21:05.707041 15524 raft_consensus.cc:1240] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:05.707744 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:05.708978 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:05.729218 21721 consensus_queue.cc:1059] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:05.731006 15674 raft_consensus.cc:2393] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 18: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:21:05.687779 15672 raft_consensus.cc:1275] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:21:05.734432 15425 catalog_manager.cc:5697] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 14 to 16, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 16 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:05.663220 22044 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:05.746969 15601 raft_consensus.cc:2410] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:21:05.744580 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:05.748914 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:05.749256 16233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:21:05.749604 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:05.752041 16233 raft_consensus.cc:606] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:05.752313 15522 raft_consensus.cc:2393] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 16: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:21:05.752475 16233 raft_consensus.cc:614] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:05.753717 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:05.754560 15526 raft_consensus.cc:2393] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 15: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:21:05.758561 15425 catalog_manager.cc:5697] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 12 to 13, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 13 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:05.759475 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:05.760732 15526 pending_rounds.cc:85] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 6
I20260709 13:21:05.761046 15526 pending_rounds.cc:107] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:21:05.760409 15425 catalog_manager.cc:5697] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 18 to 22, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 22 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:05.761338 15526 raft_consensus.cc:2889] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:05.761689 15664 tablet_service.cc:1558] Processing DeleteTablet for tablet bd36c2cfd99c45cea9e48096d54bed1a with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 5 (current committed config index is 6)) from {username='slave'} at 127.0.0.1:49088
I20260709 13:21:05.762228 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:05.762521 15660 tablet_service.cc:1558] Processing DeleteTablet for tablet 0af5649725244ae599dfc6ed7aeac5f1 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 5 (current committed config index is 7)) from {username='slave'} at 127.0.0.1:49088
I20260709 13:21:05.734042 21513 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:05.718899 22078 raft_consensus.cc:740] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:21:05.772688 15493 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:05.773763 15425 catalog_manager.cc:5697] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:21:05.782719 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 261076us
I20260709 13:21:05.743232 22081 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:05.747651 21966 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:05.760874 22019 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.787334 15425 catalog_manager.cc:5697] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 8 to 10, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 10 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:21:05.791712 15423 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49144 (request call id 352) took 59804 ms (client timeout 15000 ms). Trace:
I20260709 13:21:05.769483 21561 raft_consensus.cc:3055] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:05.798436 15525 raft_consensus.cc:1275] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:21:05.777091 21957 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:05.810672 15425 catalog_manager.cc:5697] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:21:05.820797 22089 log.cc:927] Time spent T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 13:21:05.825861 15519 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:21:05.836169 15425 catalog_manager.cc:5697] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 9 to 13, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 13 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:05.795470 21966 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:05.858385 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 85799.8us
I20260709 13:21:05.859228 15493 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:05.860064 22044 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:05.860512 15493 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:05.860669 21993 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:05.861346 22044 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:05.800448 21561 raft_consensus.cc:740] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
W20260709 13:21:05.920358 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 148645us
I20260709 13:21:05.873629 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:21:05.803335 22081 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:05.890760 15519 raft_consensus.cc:1275] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:21:05.861845 15493 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:21:05.908047 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 124489us
W20260709 13:21:05.924700 15492 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:05.925139 15641 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:05.824026 22082 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:21:05.874783 22091 log.cc:927] Time spent T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 13:21:05.931619 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 189584us
I20260709 13:21:05.844013 21966 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:05.889575 21794 raft_consensus.cc:3055] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:05.934562 15671 raft_consensus.cc:1275] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:21:05.883268 21721 raft_consensus.cc:3055] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:21:05.796892 15423 rpcz_store.cc:269] 0709 13:20:05.977816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:05.978068 (+   252us) service_pool.cc:224] Handling call
0709 13:21:05.782199 (+59804131us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:04.244980 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:04.245662 (+   682us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:04.245717 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:21:04.245740 (+    23us) tablet.cc:662] Decoding operations
0709 13:21:04.246259 (+   519us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:04.246360 (+   101us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:04.246388 (+    28us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:21:04.246848 (+   460us) tablet.cc:701] Row locks acquired
0709 13:21:04.246892 (+    44us) write_op.cc:260] PREPARE: finished
0709 13:21:04.247051 (+   159us) write_op.cc:270] Start()
0709 13:21:04.247214 (+   163us) write_op.cc:276] Timestamp: P: 1783603264246988 usec, L: 0
0709 13:21:04.247250 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 13:21:04.248049 (+   799us) log.cc:844] Serialized 913 byte log entry
0709 13:21:04.590040 (+341991us) op_driver.cc:464] REPLICATION: finished
0709 13:21:04.712395 (+122355us) write_op.cc:301] APPLY: starting
0709 13:21:04.712485 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:04.805468 (+ 92983us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:04.805599 (+   131us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:04.998538 (+192939us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:04.998871 (+   333us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 13:21:04.998938 (+    67us) write_op.cc:312] APPLY: finished
0709 13:21:04.999255 (+   317us) log.cc:844] Serialized 33 byte log entry
0709 13:21:04.999465 (+   210us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:04.999567 (+   102us) write_op.cc:454] Released schema lock
0709 13:21:04.999633 (+    66us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:05.000116 (+   483us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:05.000147 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":145795,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":122024,"apply.run_cpu_time_us":7084,"apply.run_wall_time_us":287706,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12503,"num_ops":2,"prepare.queue_time_us":375001,"prepare.run_cpu_time_us":16041,"prepare.run_wall_time_us":71487,"raft.queue_time_us":89306,"raft.run_cpu_time_us":6114,"raft.run_wall_time_us":175695,"replication_time_us":295705,"thread_start_us":297702,"threads_started":4,"wal-append.queue_time_us":130188}]]}
I20260709 13:21:05.808419 22019 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:05.929713 15425 catalog_manager.cc:5697] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 14 to 16, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 16 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:05.942322 15528 raft_consensus.cc:2368] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 19. Current term is 20.
I20260709 13:21:05.949172 15672 raft_consensus.cc:3060] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:05.952836 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 91062.1us
I20260709 13:21:05.953668 15493 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:05.931396 21911 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:05.930130 21304 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
W20260709 13:21:05.977226 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 52171.7us
I20260709 13:21:05.978191 15641 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:21:05.970230 15567 leader_election.cc:336] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Timed out: RequestConsensusVote RPC to 127.14.23.65:42481 timed out after 1.542s (SENT)
I20260709 13:21:05.979204 15567 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:05.943653 21794 raft_consensus.cc:740] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:05.944542 21721 raft_consensus.cc:740] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
W20260709 13:21:06.005290 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 68105.1us
I20260709 13:21:06.009085 22013 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.009707 22013 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:06.011861 22013 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:21:06.015735 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 68589.5us
W20260709 13:21:06.018337 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 64745.3us
I20260709 13:21:06.019693 15493 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:06.021166 15493 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:06.028795 21911 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:21:05.888698 22078 consensus_queue.cc:260] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:06.006505 22082 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:06.004549 15425 catalog_manager.cc:5697] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 13 to 16, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 16 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:06.041980 15524 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:06.033779 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:21:06.017735 21659 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:21:06.055135 22097 log.cc:927] Time spent T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:21:06.083201 15425 catalog_manager.cc:5697] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 13 to 14, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 14 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:06.092887 15425 catalog_manager.cc:5697] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 4 to 6, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 6 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:06.004458 21304 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:06.116571 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 95481.3us
I20260709 13:21:06.117568 15493 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:06.118968 15493 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:21:06.120844 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 177279us
W20260709 13:21:06.105857 22087 log.cc:927] Time spent T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:21:06.056581 22094 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:21:06.089612 22012 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:21:06.125208 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 102411us
I20260709 13:21:06.125612 15493 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:06.128798 15425 catalog_manager.cc:5697] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:06.132771 21983 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:06.143554 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 185979us
I20260709 13:21:05.981784 21561 consensus_queue.cc:260] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
W20260709 13:21:06.152124 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 174049us
I20260709 13:21:06.153815 15641 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:06.122756 22044 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:06.156637 15641 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:21:06.199656 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 47660.2us
W20260709 13:21:06.154208 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 147025us
W20260709 13:21:06.202991 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 47445.1us
I20260709 13:21:06.141577 22082 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:06.218547 22101 log.cc:927] Time spent T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.174s	user 0.000s	sys 0.003s
I20260709 13:21:06.230454 21993 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 20 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:06.234023 22078 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:06.238320 21561 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:06.146116 21659 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:06.194942 15425 catalog_manager.cc:5697] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:21:06.250056 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 94498.6us
I20260709 13:21:06.249686 15493 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:06.255623 15493 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:06.256847 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:06.257606 15671 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 18.
I20260709 13:21:06.264566 15674 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:21:06.114141 21794 consensus_queue.cc:260] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:06.245687 21911 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:06.261612 15425 catalog_manager.cc:5697] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:06.268673 21957 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:06.153591 21788 consensus_queue.cc:1048] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.102s
I20260709 13:21:06.122128 21721 consensus_queue.cc:260] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:06.286303 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 59382us
I20260709 13:21:06.286899 15425 catalog_manager.cc:5697] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:06.288512 15425 catalog_manager.cc:5697] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:06.263216 21946 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:06.279834 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:21:06.291200 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 160842us
I20260709 13:21:06.292438 15600 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 18.
I20260709 13:21:06.295645 15566 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:21:06.299890 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 322342us
I20260709 13:21:06.302500 15600 raft_consensus.cc:1275] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:21:06.308923 15425 catalog_manager.cc:5697] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:06.252425 21983 consensus_queue.cc:1048] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:21:06.252632 21659 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:06.276361 21957 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:06.313184 22114 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:06.299940 22018 consensus_queue.cc:1048] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:21:06.261281 21993 consensus_queue.cc:1059] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:21:06.315389 21946 raft_consensus.cc:697] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
W20260709 13:21:06.322710 22103 log.cc:927] Time spent T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:21:06.328342 15672 raft_consensus.cc:1275] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:21:06.333061 15425 catalog_manager.cc:5697] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 12 to 13, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 13 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:06.299427 22019 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:06.310560 21916 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.327548 21957 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:06.327759 22114 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:06.353144 16237 raft_consensus.cc:1275] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:21:06.356151 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 90027.7us
I20260709 13:21:06.352205 15425 catalog_manager.cc:5697] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 7 to 9, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 9 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:06.359584 15425 catalog_manager.cc:5697] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 11 to 16, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 16 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:21:06.358352 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 203301us
I20260709 13:21:06.365466 15425 catalog_manager.cc:5697] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 10 to 11, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 11 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:06.371474 15425 catalog_manager.cc:5697] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:06.373795 15567 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:06.378371 15425 catalog_manager.cc:5697] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:06.356644 22114 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:06.376217 22108 log.cc:927] Time spent T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 13:21:06.383220 15425 catalog_manager.cc:5697] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:06.340366 21954 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 13:21:06.386504 22058 consensus_queue.cc:1048] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:06.386924 22117 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:06.387145 22107 log.cc:927] Time spent T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
I20260709 13:21:06.387159 22119 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:21:06.388374 22105 log.cc:927] Time spent T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:21:06.367653 22082 raft_consensus.cc:2749] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:21:06.392807 15491 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:06.377063 21949 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:06.354346 21916 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:06.396149 15425 catalog_manager.cc:5697] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 11 to 15, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 15 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:21:06.401576 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:06.407389 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:06.401672 15600 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:21:06.397895 15493 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:06.403590 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:06.403728 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:06.409889 15520 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 17.
I20260709 13:21:06.411398 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:06.412091 15525 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 15.
I20260709 13:21:06.377733 21855 consensus_queue.cc:1048] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:21:06.379740 22061 consensus_queue.cc:1059] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
W20260709 13:21:06.421509 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 53975.2us
I20260709 13:21:06.404739 21946 consensus_queue.cc:237] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:06.440632 15603 raft_consensus.cc:1275] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:21:06.424026 22129 consensus_queue.cc:1048] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.021s
I20260709 13:21:06.441864 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:06.443285 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:06.446349 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:06.447311 15676 raft_consensus.cc:2410] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:21:06.423260 21911 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:06.445570 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:06.453128 15493 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:06.455919 15674 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:21:06.458374 21983 consensus_queue.cc:1048] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:21:06.423574 21966 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:06.468839 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 51179.8us
I20260709 13:21:06.470650 15493 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:21:06.475509 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 108818us
I20260709 13:21:06.476419 15641 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:06.450567 21911 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:06.483361 22132 consensus_queue.cc:1048] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:06.484000 22131 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:06.484822 21334 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:06.485101 15641 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:06.485893 22131 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:06.490942 15641 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:06.492142 22134 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:06.492583 22134 raft_consensus.cc:493] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:06.492956 22134 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:06.504208 15641 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:21:06.477408 21513 consensus_meta.cc:287] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9: Time spent flushing consensus metadata: real 0.694s	user 0.010s	sys 0.001s
W20260709 13:21:06.478752 22100 log.cc:927] Time spent T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
I20260709 13:21:06.474089 21659 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:06.476400 22122 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:06.513775 22124 log.cc:927] Time spent T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:21:06.512075 22131 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:06.519920 22117 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:06.520603 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 47772us
I20260709 13:21:06.477049 22109 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:06.502684 22134 raft_consensus.cc:515] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:06.481050 21911 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:06.533011 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 61404.3us
I20260709 13:21:06.537353 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:06.537905 16238 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:21:06.540156 15567 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:06.541085 15568 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:06.550195 22012 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:06.551795 22081 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:06.539494 21916 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:06.558156 22077 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.558722 22077 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:06.562126 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:06.536696 21946 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.564596 16205 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:21:06.570287 22128 log.cc:927] Time spent T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:21:06.544082 21949 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:06.552089 22134 leader_election.cc:290] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:06.585038 21946 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:06.581733 22054 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.587431 21949 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:06.635378 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:06.638764 15675 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:06.586525 21513 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:21:06.634302 21949 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:06.645869 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 116589us
I20260709 13:21:06.655458 21721 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:06.652351 22074 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:06.666687 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:06.672459 16234 raft_consensus.cc:1240] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 14. Current term is 15. Ops: [14.11-14.11]
W20260709 13:21:06.683347 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 96923us
I20260709 13:21:06.695927 15528 raft_consensus.cc:1275] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:21:06.690989 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:06.692876 15601 raft_consensus.cc:1240] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:06.706313 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:06.710517 22013 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:06.710901 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:06.711508 15524 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:06.711936 22013 consensus_queue.cc:1059] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:06.715262 15522 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:21:06.717406 15675 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:21:06.718418 21946 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:06.707098 15523 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 19.
I20260709 13:21:06.686776 21993 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:06.636008 22054 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:06.730167 21334 raft_consensus.cc:493] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:06.731480 21334 raft_consensus.cc:515] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:06.732744 22111 raft_consensus.cc:993] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9: : Instructing follower 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:21:06.733989 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:06.700379 21788 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:06.671437 22145 consensus_queue.cc:1048] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.085s
I20260709 13:21:06.701814 22011 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:06.741182 16236 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:21:06.742519 21334 leader_election.cc:290] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:06.743105 21334 raft_consensus.cc:1081] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 LEADER]: Signalling peer 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:21:06.743162 15410 catalog_manager.cc:5028] TS 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): tablet bd36c2cfd99c45cea9e48096d54bed1a (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:21:06.744084 15410 catalog_manager.cc:5028] TS 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): tablet 0af5649725244ae599dfc6ed7aeac5f1 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:21:06.744174 22011 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:06.746417 22011 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:06.750725 21914 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.751286 21914 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:06.753448 21914 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:06.755259 22121 raft_consensus.cc:3055] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:06.755700 22121 raft_consensus.cc:740] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:21:06.758545 22077 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:06.733481 22078 consensus_queue.cc:1059] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:21:06.760156 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:06.760787 15669 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:21:06.768723 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:06.769539 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:06.771770 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:06.772347 16238 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:06.772466 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:06.773484 15641 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:06.775372 22054 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:06.779134 21911 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:06.726025 21966 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:06.773375 21788 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:06.767773 15566 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:21:06.788121 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:06.772199 15524 raft_consensus.cc:2468] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 16.
I20260709 13:21:06.790838 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:06.792363 15641 leader_election.cc:304] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:06.794289 16234 raft_consensus.cc:2410] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:21:06.795687 15566 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:21:06.797797 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:06.798521 15603 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:21:06.798902 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:06.799603 15672 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:21:06.797703 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:06.802139 15523 raft_consensus.cc:2368] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 19. Current term is 20.
I20260709 13:21:06.802709 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4344539a12c94917991c2f39d3ff5858" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:06.805305 16238 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:21:06.808216 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:06.725064 22117 consensus_queue.cc:1048] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 13:21:06.812059 22062 raft_consensus.cc:2804] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:06.809998 21649 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:06.813601 15675 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:57710
I20260709 13:21:06.824831 15675 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:06.825264 15675 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:06.801615 21957 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:06.815979 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:06.815724 15519 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
I20260709 13:21:06.818117 15566 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:21:06.832211 15672 raft_consensus.cc:1240] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:06.834707 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:06.838541 15520 raft_consensus.cc:1240] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:21:06.845034 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:06.798699 22058 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:06.815671 22054 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:06.854861 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:06.856835 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:06.866318 16239 raft_consensus.cc:2393] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 19: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:21:06.862555 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:06.867534 16236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57244 (request call id 1927) took 1215 ms. Trace:
I20260709 13:21:06.825973 22075 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.868194 16236 rpcz_store.cc:276] 0709 13:21:05.630027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:05.630227 (+   200us) service_pool.cc:224] Handling call
0709 13:21:06.845614 (+1215387us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b540072ea54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:21:06.845710 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2025189632}
I20260709 13:21:06.871017 15677 raft_consensus.cc:1240] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:06.882460 16240 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:21:06.839501 22062 raft_consensus.cc:697] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:06.850214 22119 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.861501 21304 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:06.862546 21649 raft_consensus.cc:697] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:06.862879 21914 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.902313 22148 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:06.902922 22139 consensus_queue.cc:1059] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:21:06.903633 22092 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:06.904431 22146 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:06.904930 22082 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:06.861689 21721 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:06.865252 22058 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:06.867429 22054 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:06.890286 22097 log.cc:927] Time spent T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:21:06.927518 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 77581.6us
I20260709 13:21:06.919278 21788 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:06.934963 15675 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:06.943856 21788 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:21:06.939338 22054 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:06.932996 22146 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:06.921360 22148 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:06.885483 22075 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:06.936869 22058 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:06.950940 22114 consensus_meta.cc:287] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.555s	user 0.006s	sys 0.005s
I20260709 13:21:06.935138 21914 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:06.956666 22061 raft_consensus.cc:493] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:06.974377 22062 consensus_queue.cc:237] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:06.935400 22092 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:06.941802 22082 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:06.975632 21649 consensus_queue.cc:237] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:06.967902 22114 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:06.977589 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:06.978082 21949 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:06.922185 22119 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:06.981982 21561 consensus_meta.cc:287] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.741s	user 0.011s	sys 0.001s
I20260709 13:21:06.956171 22148 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:06.965242 22139 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.976832 21957 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:06.939698 21721 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:07.001714 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 126833us
I20260709 13:21:07.012255 15675 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:07.014442 15528 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 20.
I20260709 13:21:07.015305 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:07.013342 15642 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:21:07.016841 15520 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:21:07.024132 21914 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:07.015928 15523 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 17.
I20260709 13:21:07.026599 22082 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:07.027837 21721 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:07.014163 22139 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:07.029958 22121 consensus_queue.cc:260] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:07.030896 22139 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:07.031419 22121 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:07.031868 22092 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:07.026096 21949 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:07.033293 22075 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:07.034322 16237 raft_consensus.cc:2393] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 12: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:21:07.037971 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:07.038638 15672 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:21:07.047356 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:07.046741 22092 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:07.085484 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:07.086236 16235 raft_consensus.cc:3055] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:07.087013 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:07.087881 16236 raft_consensus.cc:2393] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 15: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:21:07.086571 16235 raft_consensus.cc:740] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:07.089320 15601 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 13:21:06.990393 22061 raft_consensus.cc:515] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:07.089929 15601 pending_rounds.cc:85] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting all ops after (but not including) 14
I20260709 13:21:07.091269 15601 pending_rounds.cc:107] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 13:21:07.091581 15601 raft_consensus.cc:2889] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:07.096920 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:07.097342 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:07.090575 16235 consensus_queue.cc:260] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:07.098685 16235 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:07.099237 15642 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:07.048545 22092 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:07.106770 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dd26481a1b4442b702cb5976b1fb41" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:07.108574 15672 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:21:07.110468 22114 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:07.110091 22081 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:07.113982 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:07.120029 15601 raft_consensus.cc:2368] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 15. Current term is 16.
I20260709 13:21:07.112432 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:07.120116 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:07.120592 15520 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:07.120690 15599 raft_consensus.cc:3060] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:07.126682 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:07.111496 21957 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:07.116483 22092 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:21:07.156840 22155 log.cc:927] Time spent T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:21:07.131256 22054 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:07.168694 22054 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:07.174533 15520 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 19.
I20260709 13:21:07.179185 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:07.179781 22058 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:07.184350 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:07.190446 15491 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:07.191195 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:07.191951 15520 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 19.
I20260709 13:21:07.188593 16204 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:07.195806 15599 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
I20260709 13:21:07.197526 22019 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:21:07.223448 22066 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:07.223958 22066 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:07.224367 22066 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:07.224737 22129 raft_consensus.cc:3055] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:07.225116 22129 raft_consensus.cc:740] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:07.233424 15566 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:21:07.305080 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 71723.9us
W20260709 13:21:07.307958 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:07.308144 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:07.308804 15521 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 19.
I20260709 13:21:07.308894 15672 raft_consensus.cc:3060] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:07.275041 15528 raft_consensus.cc:1240] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:21:07.223850 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:07.342908 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:07.342617 15675 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:07.345889 15672 raft_consensus.cc:2468] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 19.
I20260709 13:21:07.231897 21794 raft_consensus.cc:3060] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:07.241044 22163 log.cc:927] Time spent T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 13:21:07.241864 22169 log.cc:927] Time spent T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:21:07.223515 16204 leader_election.cc:304] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:07.253345 15641 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:07.347653 22019 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:07.232152 22066 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:07.238301 22058 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:07.323385 22054 raft_consensus.cc:2804] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:07.351533 15566 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:21:07.404126 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:21:07.404739 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 53285.8us
I20260709 13:21:07.230188 22148 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:07.240869 15567 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:21:07.238974 22167 log.cc:927] Time spent T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:21:07.246232 22061 leader_election.cc:290] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:07.377317 15675 raft_consensus.cc:2468] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:21:07.380605 21911 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:07.385076 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:21:07.421309 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 180518us
I20260709 13:21:07.369530 16235 raft_consensus.cc:2468] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
I20260709 13:21:07.396952 15568 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:21:07.398247 15569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:07.423259 15567 leader_election.cc:400] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 19. Current term is 20.
W20260709 13:21:07.423183 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 169945us
I20260709 13:21:07.424614 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:07.241976 22146 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:07.303627 22081 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
W20260709 13:21:07.401561 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 178126us
I20260709 13:21:07.400274 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:07.431109 15672 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18.
I20260709 13:21:07.402643 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:07.236572 22119 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:21:07.247597 22168 log.cc:927] Time spent T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:21:07.415016 15671 raft_consensus.cc:1240] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:07.418092 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:07.420105 15566 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
W20260709 13:21:07.420982 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 315374us
I20260709 13:21:07.448426 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:07.383697 22092 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:07.395629 22120 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:07.406113 22110 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.225894 22129 consensus_queue.cc:260] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:07.412642 22111 raft_consensus.cc:2749] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:07.414884 22114 raft_consensus.cc:2804] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:07.439462 16237 raft_consensus.cc:2393] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 13: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:21:07.438639 15601 raft_consensus.cc:1240] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:07.450448 15673 raft_consensus.cc:1240] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:07.450892 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:07.451431 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:07.459364 15641 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:07.418886 22117 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:07.466971 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:07.421720 22054 raft_consensus.cc:697] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:07.442543 22109 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:07.498929 15668 raft_consensus.cc:2368] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 18. Current term is 19.
I20260709 13:21:07.499150 22110 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:07.499805 22129 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:07.426381 21914 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:07.500908 22164 raft_consensus.cc:2804] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:07.413157 22019 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:07.501439 22164 raft_consensus.cc:697] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:07.502985 22019 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:07.502597 22164 consensus_queue.cc:237] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:07.425298 21334 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.475776 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:21:07.468415 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 48410.9us
I20260709 13:21:07.511693 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:07.511922 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:07.510732 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:07.438771 22013 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:07.522003 15674 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 12.
I20260709 13:21:07.512935 16235 raft_consensus.cc:3060] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:07.439028 21946 raft_consensus.cc:493] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:07.527278 16240 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 19.
W20260709 13:21:07.529026 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 69735us
W20260709 13:21:07.479415 22170 log.cc:927] Time spent T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.277s	user 0.000s	sys 0.001s
I20260709 13:21:07.504465 22117 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:07.509079 21334 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:07.495406 22066 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:07.440125 22152 consensus_queue.cc:1059] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.087s
I20260709 13:21:07.543339 16235 raft_consensus.cc:2468] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:21:07.458488 22081 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
W20260709 13:21:07.549101 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 130076us
I20260709 13:21:07.526412 22061 raft_consensus.cc:493] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:07.522397 22114 raft_consensus.cc:697] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:07.480123 21854 consensus_queue.cc:1059] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.117s
W20260709 13:21:07.550838 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 83977.5us
I20260709 13:21:07.534063 21708 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:07.551709 15642 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:07.558568 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:07.558806 15642 leader_election.cc:400] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 15. Current term is 16.
I20260709 13:21:07.559170 15642 leader_election.cc:403] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:21:07.542582 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 112096us
I20260709 13:21:07.555884 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:07.544191 22117 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:07.560542 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f25e63ed82fa4e008109b74fe8df71ba" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:07.559478 16203 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:07.542800 22013 raft_consensus.cc:697] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:07.562862 15602 raft_consensus.cc:2393] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 13: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:21:07.514479 22120 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:07.534359 21946 raft_consensus.cc:515] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:07.565850 16204 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:21:07.555883 22142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.001s	sys 0.010s
I20260709 13:21:07.557109 21949 raft_consensus.cc:3055] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:07.560017 21304 raft_consensus.cc:2749] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:07.586223 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:07.560907 21822 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 13:21:07.561357 22146 consensus_queue.cc:1059] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:21:07.606962 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:07.605546 15524 raft_consensus.cc:1275] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:21:07.582456 21949 raft_consensus.cc:740] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:21:07.585837 22142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.002s	sys 0.010s
W20260709 13:21:07.611948 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 52548.2us
I20260709 13:21:07.560199 22061 raft_consensus.cc:515] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:07.577867 22094 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 15. Current term is 16.
I20260709 13:21:07.617715 15568 leader_election.cc:304] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:07.626171 22146 consensus_queue.cc:1243] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer 3ef186ec12554a1d87814af0a0ace9a4 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:21:07.636741 22172 log.cc:927] Time spent T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:21:07.642855 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:07.643836 16236 raft_consensus.cc:2393] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 19: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:21:07.647025 21946 leader_election.cc:290] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:07.648515 21334 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:21:07.650457 22175 log.cc:927] Time spent T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:21:07.578796 22114 consensus_queue.cc:237] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:07.653888 15596 raft_consensus.cc:1275] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:21:07.654323 15596 pending_rounds.cc:85] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting all ops after (but not including) 10
I20260709 13:21:07.654551 15596 pending_rounds.cc:107] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:21:07.654789 15596 raft_consensus.cc:2889] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:07.655628 22081 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:07.563160 22054 consensus_queue.cc:237] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:07.603042 22180 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:07.618909 22135 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:07.664793 15671 raft_consensus.cc:1240] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 16. Current term is 17. Ops: []
W20260709 13:21:07.666405 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 85745us
I20260709 13:21:07.667042 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:07.618530 22121 consensus_queue.cc:1059] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:21:07.667701 15602 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 19.
I20260709 13:21:07.631338 22148 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:07.676594 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:07.679827 16236 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:07.682421 16205 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:07.661571 22180 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:07.661899 22011 raft_consensus.cc:2749] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:07.699263 16236 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
W20260709 13:21:07.699816 15566 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:07.684345 22092 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:07.734450 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 175766us
I20260709 13:21:07.707842 22109 raft_consensus.cc:3055] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:07.694000 22180 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:07.736533 15642 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:21:07.739427 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:07.725718 22148 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:07.743769 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 121628us
I20260709 13:21:07.700930 21708 raft_consensus.cc:697] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:07.747115 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:07.748024 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:07.662412 22013 consensus_queue.cc:237] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:07.749529 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:07.750831 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:07.750608 16203 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:07.738122 22079 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:07.685159 21949 consensus_queue.cc:260] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:07.740015 22109 raft_consensus.cc:740] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:21:07.737709 22110 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:21:07.763885 22184 log.cc:927] Time spent T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:21:07.767272 15675 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:21:07.758061 22092 raft_consensus.cc:2804] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:07.772289 22092 raft_consensus.cc:697] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:07.773186 22092 consensus_queue.cc:237] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:07.781409 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:07.782101 16234 raft_consensus.cc:2368] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13.
I20260709 13:21:07.757625 22077 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:21:07.777325 22117 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:07.752686 22148 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:07.770949 22109 consensus_queue.cc:260] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:07.791673 22148 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:07.791800 22180 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:07.793716 15527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57300 (request call id 2581) took 1004 ms. Trace:
I20260709 13:21:07.793876 22180 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:07.794549 15527 rpcz_store.cc:276] 0709 13:21:06.789105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:06.789239 (+   134us) service_pool.cc:224] Handling call
0709 13:21:06.789534 (+   295us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:06.810379 (+ 20845us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:06.814083 (+  3704us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:06.814668 (+   585us) log.cc:844] Serialized 52 byte log entry
0709 13:21:07.203840 (+389172us) spinlock_profiling.cc:243] Waited 1.95 ms on lock 0x7b18003cdba0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce1900e07 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191dc02 00007f5ce45a520b 00007f5ce45aad23 00007f5ce45ab0d1 00007f5ce49c6a0f
0709 13:21:07.416947 (+213107us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:07.432722 (+ 15775us) log.cc:844] Serialized 14 byte log entry
0709 13:21:07.439932 (+  7210us) log.cc:844] Serialized 14 byte log entry
0709 13:21:07.440154 (+   222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:07.440259 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:07.529531 (+ 89272us) spinlock_profiling.cc:243] Waited 43.5 ms on lock 0x7b18003cdba0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce191e522 00007f5ce19207c1 00007f5ce1920779 00007f5ce1920709 00007f5ce19206d1 00007f5ce191f9cd 000000000038f354 000000000038f199 00007f5ce190183a
0709 13:21:07.791495 (+261964us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b18003cdba0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce191e3f3 00007f5ce19207c1 00007f5ce1920779 00007f5ce1920709 00007f5ce19206d1 00007f5ce191f9cd 000000000038f354 000000000038f199 00007f5ce190183a
0709 13:21:07.793508 (+  2013us) raft_consensus.cc:1685] finished
0709 13:21:07.793529 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:07.793660 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3103,"prepare.queue_time_us":15590,"prepare.run_wall_time_us":8506,"raft.queue_time_us":5098,"raft.run_cpu_time_us":431,"raft.run_wall_time_us":59698,"spinlock_wait_cycles":202549504,"thread_start_us":212911,"threads_started":1,"wal-append.queue_time_us":661338}
I20260709 13:21:07.796861 15642 leader_election.cc:400] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13.
I20260709 13:21:07.791944 22109 raft_consensus.cc:3060] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:07.797309 15642 leader_election.cc:403] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:21:07.798058 22092 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:07.784293 22061 leader_election.cc:290] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:07.802484 15671 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:07.815174 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:07.814960 21708 consensus_queue.cc:237] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:07.815442 22079 raft_consensus.cc:493] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:07.816365 15676 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:21:07.820443 15493 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:07.823487 22121 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:07.817286 22079 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:07.829290 22187 consensus_queue.cc:1048] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:21:07.834787 15671 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18.
I20260709 13:21:07.838699 16203 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:07.840886 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:07.847318 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:07.846905 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:07.846791 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:07.847937 15523 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 20.
I20260709 13:21:07.845132 22074 raft_consensus.cc:2764] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:21:07.859460 22012 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 13:21:07.866489 22117 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:07.869098 15641 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:07.879104 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:07.894433 15676 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 20.
I20260709 13:21:07.932713 16237 raft_consensus.cc:2368] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 17. Current term is 18.
W20260709 13:21:07.933077 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 43916.2us
I20260709 13:21:07.896643 16203 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:07.935477 15642 leader_election.cc:400] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 17. Current term is 18.
I20260709 13:21:07.935859 15642 leader_election.cc:403] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:21:07.937445 21822 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 17. Current term is 18.
I20260709 13:21:07.938807 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:07.947983 22079 raft_consensus.cc:515] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:07.951372 22092 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 12. Current term is 13.
I20260709 13:21:07.853207 22148 consensus_queue.cc:1059] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.099s
I20260709 13:21:07.957039 15520 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
W20260709 13:21:07.952697 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 56169us
W20260709 13:21:07.985764 15493 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:07.957762 21946 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:07.990123 21946 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:07.992321 21946 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:07.960312 22074 raft_consensus.cc:2804] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:07.993470 22074 raft_consensus.cc:697] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:07.994413 22074 consensus_queue.cc:237] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:07.958046 22142 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:08.002853 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f8365fc4e2f4decb3d554a43e7d4d52" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:07.952018 22117 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:08.041218 15491 leader_election.cc:400] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 election: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 18. Current term is 19.
I20260709 13:21:08.041648 15491 leader_election.cc:403] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:21:08.041550 22117 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:08.076292 15491 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:08.040056 22142 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:08.094933 22158 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:08.097589 22158 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:08.099831 22158 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:08.100127 22144 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 18. Current term is 19.
I20260709 13:21:08.100996 22144 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:08.101537 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:08.070075 22148 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:08.075994 21855 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:08.080629 22117 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:08.120685 15525 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 18.
I20260709 13:21:08.122104 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:08.098073 22079 leader_election.cc:290] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:08.202613 21949 raft_consensus.cc:3060] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:08.124751 21855 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:08.204252 22012 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.210731 15596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34102 (request call id 1810) took 1169 ms. Trace:
I20260709 13:21:08.232277 15596 rpcz_store.cc:276] 0709 13:21:07.041657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:07.041779 (+   122us) service_pool.cc:224] Handling call
0709 13:21:07.085069 (+ 43290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:07.655185 (+570116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:08.207316 (+552131us) spinlock_profiling.cc:243] Waited 568 ms on lock 0x7b540043ba54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0be01 00007f5ce4a0a83e 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:21:08.210640 (+  3324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1307373440}
I20260709 13:21:08.142686 22148 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:08.276677 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 200458us
I20260709 13:21:08.235960 22012 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:08.283126 15491 leader_election.cc:304] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:08.290313 21914 raft_consensus.cc:3055] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:08.290719 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:08.292567 15526 raft_consensus.cc:3060] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:08.287214 22144 raft_consensus.cc:2749] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:08.302594 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:08.303282 15528 raft_consensus.cc:3060] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:08.307256 15567 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:08.311865 15526 raft_consensus.cc:2468] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18.
I20260709 13:21:08.314502 22188 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:08.292268 21914 raft_consensus.cc:740] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:08.297313 21855 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:08.317698 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:08.296257 22142 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:08.323612 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:08.324484 15522 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 19.
I20260709 13:21:08.329861 21914 consensus_queue.cc:260] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:08.331248 21914 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:08.334805 22012 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:21:08.337136 22109 consensus_meta.cc:287] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69: Time spent flushing consensus metadata: real 0.539s	user 0.001s	sys 0.009s
I20260709 13:21:08.343251 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:08.344852 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:08.344059 15519 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:21:08.362349 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:08.362972 22148 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:08.363004 15602 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 18.
I20260709 13:21:08.354939 15567 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:08.377898 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:08.380918 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:08.376430 15567 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:21:08.387259 22195 log.cc:927] Time spent T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:21:08.389488 22120 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:08.385519 22064 consensus_queue.cc:1048] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:21:08.383932 22114 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:08.395943 22114 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:08.396355 22114 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:08.406020 15602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9328896aa5a4fdc8fa7c908efdae695" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:08.406754 15602 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:08.410884 15493 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:08.413008 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:08.413431 15528 raft_consensus.cc:2468] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 20.
I20260709 13:21:08.435135 22199 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:08.435734 22199 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:08.398093 22148 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:08.442946 22148 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:08.443445 22148 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:08.436121 22199 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:08.361711 22117 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:21:08.505198 22144 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:08.509403 22144 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:08.511498 22144 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:08.504937 22109 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:08.513816 22109 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:08.515887 22109 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:08.519079 22198 log.cc:927] Time spent T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.257s	user 0.007s	sys 0.000s
I20260709 13:21:08.523880 16238 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:21:08.538389 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:08.544588 16236 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:21:08.533834 16204 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:08.577783 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:21:08.578541 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 44812us
I20260709 13:21:08.579269 16204 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:08.580470 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:08.581976 16204 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:08.580448 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:08.584177 22064 raft_consensus.cc:2804] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:08.585377 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:08.581017 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:08.588944 21964 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:08.591835 15522 raft_consensus.cc:1275] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:21:08.593751 15568 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:08.594965 15568 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:08.619071 15521 raft_consensus.cc:1275] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:21:08.619566 15521 pending_rounds.cc:85] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting all ops after (but not including) 10
I20260709 13:21:08.619808 15521 pending_rounds.cc:107] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:21:08.623924 15521 raft_consensus.cc:2889] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:08.578658 22109 raft_consensus.cc:493] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
W20260709 13:21:08.632843 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 51425.7us
I20260709 13:21:08.646546 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:08.650210 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:08.651019 15596 raft_consensus.cc:2393] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 20: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:21:08.651503 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:08.598994 22064 raft_consensus.cc:697] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:08.660137 22064 consensus_queue.cc:237] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:08.621181 22131 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:08.606467 22149 raft_consensus.cc:493] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:21:08.677093 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 54056.1us
I20260709 13:21:08.670717 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:08.645617 21964 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:08.654209 22164 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.655234 22145 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:08.610028 22148 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:08.696600 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:08.673627 21854 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:08.666905 22188 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:21:08.703521 22187 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:08.716454 21949 consensus_queue.cc:1048] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:08.679965 22131 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:08.725700 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 68072.3us
I20260709 13:21:08.698551 21964 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:08.719185 15600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53324 (request call id 2441) took 1260 ms. Trace:
W20260709 13:21:08.739221 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 86841.6us
I20260709 13:21:08.679370 15602 raft_consensus.cc:2468] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 20.
I20260709 13:21:08.735531 15601 raft_consensus.cc:1240] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:08.739573 15668 raft_consensus.cc:1275] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:21:08.731359 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:08.741321 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:08.736306 15600 rpcz_store.cc:276] 0709 13:21:07.458315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:07.475517 (+ 17202us) service_pool.cc:224] Handling call
0709 13:21:08.718961 (+1243444us) spinlock_profiling.cc:243] Waited 992 ms on lock 0x7b540043ce54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:21:08.719093 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2280901504}
I20260709 13:21:08.746354 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:08.748194 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:08.749471 15603 raft_consensus.cc:1240] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:21:08.746969 15668 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 16.
I20260709 13:21:08.680831 21855 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:21:08.680303 22109 raft_consensus.cc:515] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:08.758574 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 114806us
I20260709 13:21:08.760721 15602 raft_consensus.cc:1275] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:21:08.682199 22149 raft_consensus.cc:515] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:08.769698 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:08.741254 21964 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:08.773950 15668 raft_consensus.cc:1275] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:21:08.708135 22164 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:08.752591 21914 raft_consensus.cc:2764] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:21:08.712112 21854 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:08.794413 22149 leader_election.cc:290] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:21:08.800307 22204 log.cc:927] Time spent T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:21:08.807399 22149 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:08.809180 22131 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:08.811048 22109 leader_election.cc:290] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:08.812885 21854 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:08.819980 22117 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 20 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:08.774816 22148 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:08.774674 22144 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:08.824930 22114 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:08.825472 22144 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:08.827627 22144 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:08.740604 22187 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:21:08.837179 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 225184us
W20260709 13:21:08.834833 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 78770.7us
I20260709 13:21:08.840853 16205 leader_election.cc:304] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:08.842993 15568 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:08.867712 15568 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:08.840528 22145 raft_consensus.cc:697] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:08.870769 22011 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:08.875250 22188 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:08.851801 22019 raft_consensus.cc:2804] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:08.887982 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:08.883306 15568 leader_election.cc:304] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:08.907944 15493 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:08.912031 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:08.912611 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:08.875660 22164 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:08.916108 15669 raft_consensus.cc:1275] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:21:08.905524 22019 raft_consensus.cc:697] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:08.917706 22201 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:08.929514 21964 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:08.939846 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 56633.2us
I20260709 13:21:08.940752 15568 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:08.930243 22215 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:08.943369 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 99207.5us
I20260709 13:21:08.947587 16239 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:21:08.948189 16239 raft_consensus.cc:606] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:08.948508 16239 raft_consensus.cc:614] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:08.937671 22164 consensus_queue.cc:1048] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
W20260709 13:21:09.014496 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 54721.6us
W20260709 13:21:09.017040 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 56278.6us
I20260709 13:21:08.950243 22220 raft_consensus.cc:2749] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:08.934705 22145 consensus_queue.cc:237] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:09.022955 15520 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:21:08.960842 22218 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:09.025521 16240 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:21:09.028138 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:09.028614 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:09.032467 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:08.975334 22114 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:09.051265 15601 raft_consensus.cc:1275] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:21:09.020293 22019 consensus_queue.cc:237] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:09.053746 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:09.059782 21964 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:09.046310 22208 consensus_queue.cc:1048] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:21:09.070732 22114 consensus_queue.cc:1048] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:09.073223 21334 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:21:09.074056 22131 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:09.077883 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:09.079231 15527 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:09.053296 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:09.081964 16232 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 19.
I20260709 13:21:08.950085 22199 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:08.950662 22215 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:21:09.048208 22187 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:09.019846 22181 consensus_queue.cc:1059] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.068s
W20260709 13:21:09.099691 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 188506us
I20260709 13:21:09.108351 15527 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 20.
I20260709 13:21:09.118955 15599 raft_consensus.cc:2393] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 20: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:21:09.118378 15493 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:09.110219 16204 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:09.076826 22148 consensus_queue.cc:1048] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:21:09.123029 16232 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "974a060267d24f00888c0942dfbca77e"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
W20260709 13:21:09.122593 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 52647.6us
I20260709 13:21:09.124284 22121 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:21:09.124810 15493 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:09.126056 16232 raft_consensus.cc:606] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:21:09.126029 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 52428.5us
I20260709 13:21:09.130523 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:09.135038 15523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:09.136484 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:09.137095 15525 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:09.138957 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340e065cf2744fde9d65230a8ea91e14" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:09.140059 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4da1202d21704a61a455cc91fa7c4022" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:09.142303 15676 raft_consensus.cc:3060] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:09.147444 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:09.148079 15521 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 20.
I20260709 13:21:09.159901 15676 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 17.
I20260709 13:21:09.131889 21946 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:09.134289 22209 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:09.164855 15566 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:21:09.146131 22225 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:09.170670 21649 raft_consensus.cc:2804] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:09.171279 21649 raft_consensus.cc:697] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:09.172171 21649 consensus_queue.cc:237] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:09.173197 22225 raft_consensus.cc:697] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:09.142350 22117 consensus_queue.cc:1048] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 13:21:09.174157 22225 consensus_queue.cc:237] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:09.164928 22226 log.cc:927] Time spent T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:21:09.184027 15525 raft_consensus.cc:2468] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 17.
I20260709 13:21:09.162851 22144 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:09.189599 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 79486.4us
I20260709 13:21:09.194589 16235 raft_consensus.cc:1240] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:09.198284 22215 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
W20260709 13:21:09.202601 22227 log.cc:927] Time spent T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:21:09.167466 22209 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:09.222334 22209 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:09.223122 22199 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:09.205139 22144 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:09.236940 22144 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:09.244230 22237 log.cc:927] Time spent T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:21:09.255726 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:09.259527 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9a4c0b115544577a855e9c7383d9a48" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:09.258311 22131 consensus_queue.cc:1059] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.006s
I20260709 13:21:09.271015 22191 raft_consensus.cc:3055] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:09.271602 22191 raft_consensus.cc:740] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:09.272523 22191 consensus_queue.cc:260] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:09.293401 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:09.296406 15493 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:09.297818 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83022a40fade44a2988f3de3b6b36aec" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:09.299551 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:09.300859 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:09.301708 16231 raft_consensus.cc:1275] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:21:09.301885 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:09.302817 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:09.303601 15599 raft_consensus.cc:2410] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 21 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 13:21:09.304710 15520 raft_consensus.cc:1240] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:09.306007 15674 raft_consensus.cc:1275] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:21:09.306571 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:09.306228 16238 raft_consensus.cc:2410] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 21 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 13:21:09.313661 15525 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 20.
I20260709 13:21:09.316596 15527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:09.324206 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:09.324844 15641 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:09.278470 22191 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:09.312393 22186 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.001s	sys 0.028s
I20260709 13:21:09.351198 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:09.300531 22213 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:09.322232 22199 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:09.327323 22131 consensus_queue.cc:1243] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer dd1a14c3173d49dc9b55bb116e3783c9 log is divergent from this leader: its last log entry 20.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:21:09.357993 15493 leader_election.cc:304] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:21:09.353312 22186 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.001s	sys 0.029s
I20260709 13:21:09.365890 22201 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:09.353789 22117 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:09.358875 21946 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:09.394165 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:09.363129 22208 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:09.395929 15493 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:09.395426 15568 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:09.400874 15567 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:09.403234 15567 leader_election.cc:304] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:09.403584 22200 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:09.391520 22199 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:09.406111 22199 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:09.406651 15567 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:09.407321 22217 raft_consensus.cc:2749] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:09.407831 21708 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.408408 22217 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:09.408319 21708 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:09.418172 15493 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:09.393616 22246 raft_consensus.cc:2749] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:21:09.398347 15595 raft_consensus.cc:2393] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 19: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:21:09.421752 16234 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 19.
I20260709 13:21:09.425112 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:09.426653 15528 raft_consensus.cc:1275] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:21:09.426008 15525 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 20.
I20260709 13:21:09.508818 15567 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:21:09.546864 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 91807us
I20260709 13:21:09.417138 22209 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.364003 22213 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:09.422643 22121 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:09.422621 22114 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:09.422726 22208 raft_consensus.cc:697] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:09.429060 21649 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:09.429287 22134 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.614300 22208 consensus_queue.cc:237] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:21:09.567787 22215 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:09.633605 22236 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.636981 21708 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:09.637427 22134 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:09.638885 22111 consensus_queue.cc:1048] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:09.668324 22199 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:09.670307 15594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53310 (request call id 2678) took 1021 ms. Trace:
I20260709 13:21:09.670769 15594 rpcz_store.cc:276] 0709 13:21:08.648896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:08.649745 (+   849us) service_pool.cc:224] Handling call
0709 13:21:09.670096 (+1020351us) spinlock_profiling.cc:243] Waited 927 ms on lock 0x7b540032b9d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:21:09.670248 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2133029120}
I20260709 13:21:09.673051 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:09.681556 16232 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:21:09.682219 16232 raft_consensus.cc:606] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 25 FOLLOWER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:09.682571 16232 raft_consensus.cc:614] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:09.670614 22236 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:09.685700 15594 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:59198
I20260709 13:21:09.686395 15594 raft_consensus.cc:606] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:09.688119 22079 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.688761 22079 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:09.685712 22236 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:09.698314 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:09.702354 15603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:09.703362 16233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:21:09.465452 22188 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.131s
I20260709 13:21:09.703962 16233 raft_consensus.cc:606] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:09.419724 21946 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:09.419878 22117 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:09.656996 22213 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:09.592552 22114 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:09.589511 22209 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:09.644042 22215 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:09.664278 22134 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:09.705562 15568 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:09.717867 15493 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:09.732647 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:21:09.722169 22114 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:09.737234 22215 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:09.714403 22219 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
W20260709 13:21:09.755335 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 428115us
I20260709 13:21:09.756274 15668 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:21:09.756342 15641 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:09.744652 22218 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:09.763589 15641 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:09.748924 22199 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:09.804281 15676 raft_consensus.cc:3055] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:09.805351 15676 raft_consensus.cc:740] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:09.804533 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:21:09.810297 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 484163us
I20260709 13:21:09.813624 16232 raft_consensus.cc:1217] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 LEADER]: Deduplicated request from leader. Original: 17.14->[19.15-19.15]   Dedup: 19.15->[]
I20260709 13:21:09.814198 16232 raft_consensus.cc:1240] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:21:09.815922 16233 raft_consensus.cc:1240] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:09.760915 22134 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:09.819710 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:09.821996 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:09.840613 15528 raft_consensus.cc:1275] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:21:09.842624 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:09.824473 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:09.826150 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:09.828042 22120 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:09.830139 22252 log.cc:927] Time spent T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:21:09.858431 22117 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:09.860044 21946 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:09.861584 22209 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:09.802093 22216 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:21:09.832226 22250 log.cc:927] Time spent T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:21:09.811097 22200 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:09.806131 15676 consensus_queue.cc:260] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:09.834522 22254 log.cc:927] Time spent T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:21:09.849059 22120 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:09.874984 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:09.895180 15528 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 16.
I20260709 13:21:09.859366 22215 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:21:09.891013 15676 raft_consensus.cc:3060] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:09.890595 15642 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:09.764353 22187 consensus_queue.cc:1059] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.106s
I20260709 13:21:09.911780 15676 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:21:09.813133 22211 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.083s
I20260709 13:21:09.917740 15641 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:09.889217 22213 consensus_queue.cc:1048] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:21:09.920938 22257 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:09.921424 22257 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:09.921757 22257 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:09.930568 15669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:09.931329 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:09.909370 22079 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:09.936470 22120 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:09.894616 22200 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:09.951633 22258 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:09.955013 22215 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 21 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481)
I20260709 13:21:09.957068 22200 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:09.957186 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:09.960742 15596 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 17.
I20260709 13:21:09.962221 16233 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 13.
I20260709 13:21:09.967955 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:09.968672 15675 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:21:09.969199 15669 raft_consensus.cc:3055] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:09.969570 15669 raft_consensus.cc:740] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:09.970167 16203 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:09.971226 22211 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:09.971589 22211 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:09.971916 22211 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:09.976310 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:10.002931 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:10.003587 15493 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:10.003005 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:21:10.020454 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 79252.5us
I20260709 13:21:09.990052 15669 consensus_queue.cc:260] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:10.029199 15669 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:10.036190 15528 raft_consensus.cc:1275] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:21:10.038285 15493 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:10.069301 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:10.070250 15528 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:21:10.072821 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:10.073500 15521 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:10.075381 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:10.079499 15642 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:10.081053 15671 raft_consensus.cc:1275] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:21:10.083868 15673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:10.039494 21946 raft_consensus.cc:493] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:10.088439 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:10.089607 15597 raft_consensus.cc:1275] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:21:10.094098 15669 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 13:21:10.098567 15669 pending_rounds.cc:85] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4: Aborting all ops after (but not including) 14
I20260709 13:21:10.098826 15669 pending_rounds.cc:107] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:21:10.099064 15669 raft_consensus.cc:2889] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:10.104862 21946 raft_consensus.cc:515] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:10.105741 22266 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:10.107311 21946 leader_election.cc:290] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:10.108932 22264 raft_consensus.cc:993] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9: : Instructing follower 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:21:10.109454 22264 raft_consensus.cc:1081] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Signalling peer 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:21:10.112915 22120 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:10.113545 22120 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:10.116887 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:10.066912 22246 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:10.119392 15673 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:57710
I20260709 13:21:10.040246 22229 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:10.148469 15673 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:10.149066 15673 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:10.078116 22199 raft_consensus.cc:697] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:21:10.135941 15521 raft_consensus.cc:2468] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 21.
I20260709 13:21:10.116237 22121 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:10.150543 22229 raft_consensus.cc:493] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:10.152118 22120 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:10.169399 16204 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:10.184412 22199 consensus_queue.cc:237] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:10.177888 22111 consensus_queue.cc:1048] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 13:21:10.178673 22229 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:10.188788 15673 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:10.192363 22211 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:10.113754 22114 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:10.158035 22264 consensus_queue.cc:1175] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" responder_term: 12 status { last_received { term: 12 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 12 index: 10 } }
I20260709 13:21:10.204289 16232 consensus_queue.cc:237] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:10.206178 22064 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:10.208830 22064 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:10.209209 22064 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:10.212607 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:10.178483 22121 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:10.221544 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:10.223379 15525 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:10.241410 15673 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:10.243111 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 163703us
I20260709 13:21:10.252720 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:10.260391 15491 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:10.261260 22267 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:10.266569 22180 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:21:10.269129 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:10.269783 16237 raft_consensus.cc:3055] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:10.270112 16237 raft_consensus.cc:740] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:10.270889 16237 consensus_queue.cc:260] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:10.271678 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:21:10.273563 22074 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:10.274294 22074 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:10.276598 22074 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:10.279081 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:10.280579 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:10.272068 16237 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:10.305984 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:10.324568 15525 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 19.
I20260709 13:21:10.333362 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:21:10.362387 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 55746.6us
I20260709 13:21:10.367970 15642 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:10.301126 22208 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:10.379743 22240 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:10.382898 22114 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 21 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:10.366627 22216 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 13:21:10.375061 22211 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:10.389973 22232 consensus_queue.cc:1048] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:10.332635 22257 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:10.392381 15672 raft_consensus.cc:2368] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 17. Current term is 18.
I20260709 13:21:10.393435 22257 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:10.405797 22217 consensus_queue.cc:1048] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:10.408367 22121 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:10.409523 22188 raft_consensus.cc:993] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:21:10.410179 22188 raft_consensus.cc:1081] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Signalling peer 3ef186ec12554a1d87814af0a0ace9a4 to start an election
W20260709 13:21:10.422689 22270 log.cc:927] Time spent T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.240s	user 0.003s	sys 0.000s
I20260709 13:21:10.446089 22266 raft_consensus.cc:993] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9: : Instructing follower d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:21:10.446753 22266 raft_consensus.cc:1081] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 LEADER]: Signalling peer d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:21:10.463215 15567 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:10.481217 15676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
W20260709 13:21:10.482164 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 312851us
I20260709 13:21:10.482031 15676 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:10.482450 15641 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:10.483870 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:10.484057 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:10.489542 16203 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:10.491688 15676 raft_consensus.cc:2468] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:21:10.499179 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:10.499867 15524 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:10.506115 15675 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:49100
I20260709 13:21:10.506709 15677 raft_consensus.cc:1240] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:10.506755 15675 raft_consensus.cc:493] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:10.507450 15675 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:10.517290 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:10.526734 15522 raft_consensus.cc:1240] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:10.527657 16203 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:10.532447 22232 raft_consensus.cc:493] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:10.533069 22232 raft_consensus.cc:515] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:10.534042 22158 consensus_queue.cc:1059] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:10.500556 22257 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:10.534979 22257 raft_consensus.cc:697] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:10.535881 22257 consensus_queue.cc:237] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:10.511027 22229 raft_consensus.cc:515] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:10.544771 22229 leader_election.cc:290] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:10.545534 15526 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:10.547937 15675 raft_consensus.cc:515] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:10.574430 15675 leader_election.cc:290] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:10.551607 16237 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 19.
I20260709 13:21:10.509584 21964 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:10.514472 22217 raft_consensus.cc:2804] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:10.594194 16238 raft_consensus.cc:1240] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:10.595359 15524 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 21.
W20260709 13:21:10.604144 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 76572.7us
I20260709 13:21:10.605142 16203 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:10.552734 22110 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:21:10.612643 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 57891us
W20260709 13:21:10.613584 15425 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49140 (request call id 364) took 16501 ms (client timeout 15000 ms). Trace:
W20260709 13:21:10.601791 22259 log.cc:927] Time spent T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 13:21:10.609614 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 241697us
I20260709 13:21:10.613471 15526 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 18.
I20260709 13:21:10.629172 15524 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:21:10.626211 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 73276.5us
I20260709 13:21:10.607264 22201 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:10.644605 22201 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:10.645371 22286 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:10.646824 22201 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:10.650907 15521 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "974a060267d24f00888c0942dfbca77e"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:56840
I20260709 13:21:10.651530 15521 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:10.651918 15521 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:10.665709 22158 raft_consensus.cc:3055] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:10.666226 22158 raft_consensus.cc:740] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:21:10.666733 22275 log.cc:927] Time spent T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:21:10.667058 22158 consensus_queue.cc:260] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:10.634109 22094 consensus_queue.cc:1059] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:21:10.668283 22158 raft_consensus.cc:3060] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:10.670745 22217 raft_consensus.cc:697] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:10.671684 22217 consensus_queue.cc:237] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:21:10.674034 22232 leader_election.cc:290] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:21:10.674465 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 54184.8us
I20260709 13:21:10.624143 22110 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:10.677270 22110 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:10.678061 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:10.678660 15567 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
W20260709 13:21:10.680058 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:10.626128 21964 raft_consensus.cc:2804] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:10.680464 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:10.687423 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:10.661230 15521 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:10.678684 15595 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
I20260709 13:21:10.694305 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:10.698745 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:10.698807 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:10.706301 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:21:10.618920 15425 rpcz_store.cc:269] 0709 13:20:54.098452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:54.102104 (+  3652us) service_pool.cc:224] Handling call
0709 13:21:10.599929 (+16497825us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:06.644065 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:06.644619 (+   554us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:06.644657 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:21:06.644675 (+    18us) tablet.cc:662] Decoding operations
0709 13:21:06.646681 (+  2006us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:06.646778 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:06.646806 (+    28us) tablet.cc:685] Acquiring locks for 29 operations
0709 13:21:06.783033 (+136227us) tablet.cc:701] Row locks acquired
0709 13:21:06.783058 (+    25us) write_op.cc:260] PREPARE: finished
0709 13:21:06.783203 (+   145us) write_op.cc:270] Start()
0709 13:21:06.783323 (+   120us) write_op.cc:276] Timestamp: P: 1783603266783154 usec, L: 0
0709 13:21:06.783345 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:21:06.807450 (+ 24105us) log.cc:844] Serialized 11277 byte log entry
0709 13:21:07.569099 (+761649us) op_driver.cc:464] REPLICATION: finished
0709 13:21:07.745963 (+176864us) write_op.cc:301] APPLY: starting
0709 13:21:07.746080 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:07.940022 (+193942us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:07.940121 (+    99us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:08.815279 (+875158us) spinlock_profiling.cc:243] Waited 2.88 ms on lock 0x7b18002fe240. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce474a0ec 00007f5ce474caf7 00007f5ce474dfb0 00007f5ce84621bb 00007f5ce84a95d4 00007f5ce83558da 00007f5ce83583cb 00007f5ce8357fd3 00007f5ce8456e34
0709 13:21:08.876912 (+ 61633us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:08.877283 (+   371us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=58,key_file_lookups=58,delta_file_lookups=0,mrs_lookups=0
0709 13:21:08.877330 (+    47us) write_op.cc:312] APPLY: finished
0709 13:21:08.877833 (+   503us) log.cc:844] Serialized 252 byte log entry
0709 13:21:09.016649 (+138816us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:09.056117 (+ 39468us) write_op.cc:454] Released schema lock
0709 13:21:09.056195 (+    78us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:09.056686 (+   491us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:09.056721 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":715455,"reported_tablets":40,"child_traces":[["op",{"apply.queue_time_us":173330,"apply.run_cpu_time_us":59331,"apply.run_wall_time_us":1310738,"cfile_cache_hit":118,"cfile_cache_hit_bytes":125579,"num_ops":29,"prepare.queue_time_us":57694,"prepare.run_cpu_time_us":65102,"prepare.run_wall_time_us":321635,"raft.queue_time_us":371609,"raft.run_cpu_time_us":18217,"raft.run_wall_time_us":201825,"replication_time_us":766439,"spinlock_wait_cycles":6617984,"thread_start_us":665934,"threads_started":5,"wal-append.queue_time_us":336984,"wal-append.run_cpu_time_us":54767,"wal-append.run_wall_time_us":2657698}]]}
I20260709 13:21:10.714035 22094 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:10.714424 22276 raft_consensus.cc:2804] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:10.714986 22276 raft_consensus.cc:697] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:10.714687 22094 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:10.715863 22276 consensus_queue.cc:237] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:10.716758 22209 raft_consensus.cc:3055] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:21:10.717159 22209 raft_consensus.cc:740] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:10.717957 22209 consensus_queue.cc:260] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:21:10.719110 22209 raft_consensus.cc:3060] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:10.720475 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:10.722072 15596 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:21:10.687686 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 65070.6us
I20260709 13:21:10.760941 15521 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:10.768715 15642 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:10.744571 21964 raft_consensus.cc:697] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:10.751134 22201 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:10.688395 22064 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:10.783308 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 118208us
I20260709 13:21:10.774725 22235 raft_consensus.cc:493] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:10.800662 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d461474623405ab8ee3ca7fdff1722" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:10.809041 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:10.812885 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e67cef93bbdb4ff88a2ee2ac558ba090" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:10.823149 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:10.823976 15670 raft_consensus.cc:2368] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 17. Current term is 18.
W20260709 13:21:10.851205 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 52396.5us
I20260709 13:21:10.866626 16234 raft_consensus.cc:1240] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:21:10.868793 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 51533.6us
I20260709 13:21:10.810616 22187 raft_consensus.cc:2764] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:21:10.799214 22183 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.074s
W20260709 13:21:10.913097 22284 log.cc:927] Time spent T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:21:10.918752 15600 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 17.
I20260709 13:21:10.953078 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:21:10.914906 22252 log.cc:927] Time spent T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:21:10.817412 22201 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:10.852389 22235 raft_consensus.cc:515] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:10.969534 22262 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:10.900058 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:10.997013 15601 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:11.054641 22064 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:10.970100 21964 consensus_queue.cc:237] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:11.071609 22201 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:11.047474 22299 log.cc:927] Time spent T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 13:21:11.101706 22298 log.cc:927] Time spent T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:21:11.104297 22235 leader_election.cc:290] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:11.111197 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 258831us
I20260709 13:21:11.119513 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:11.124636 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5243f722506441269fac66aa989ce36d" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:11.125321 15600 raft_consensus.cc:3060] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:11.126577 15597 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:21:11.129982 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 314320us
W20260709 13:21:11.134886 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 265139us
I20260709 13:21:11.138099 15600 raft_consensus.cc:2468] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 18.
I20260709 13:21:11.141266 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:11.141952 15597 raft_consensus.cc:3055] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:11.142242 15597 raft_consensus.cc:740] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:21:11.142372 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 322710us
I20260709 13:21:11.144841 15493 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:11.167256 15600 raft_consensus.cc:1240] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 LEADER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:11.232338 15674 raft_consensus.cc:1240] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:11.150625 15597 consensus_queue.cc:260] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:11.246754 15597 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:11.257319 15601 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 18.
I20260709 13:21:11.260452 16203 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:11.260706 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:11.261287 16233 raft_consensus.cc:3055] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:11.261602 16233 raft_consensus.cc:740] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:11.259825 22013 raft_consensus.cc:493] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:11.260027 22217 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:11.263114 22217 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:11.263034 22013 raft_consensus.cc:515] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:11.278625 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 133866us
W20260709 13:21:11.282050 15567 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:11.262912 16233 consensus_queue.cc:260] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:11.292016 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:21:11.335269 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 55860.2us
W20260709 13:21:11.304710 22266 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.015s	sys 0.021s
W20260709 13:21:11.354597 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 63968.7us
I20260709 13:21:11.355329 15493 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:11.358557 22246 raft_consensus.cc:2764] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
W20260709 13:21:11.354206 22277 log.cc:927] Time spent T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
W20260709 13:21:11.354625 22266 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.015s	sys 0.021s
I20260709 13:21:11.368340 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:11.371846 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:11.372697 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:11.292814 22201 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:21:11.326265 22199 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 13:21:11.374790 15525 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:21:11.346787 22218 consensus_queue.cc:1059] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:21:11.382345 15641 leader_election.cc:304] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:11.386664 22187 raft_consensus.cc:2804] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:11.387212 22187 raft_consensus.cc:697] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:11.379294 22148 raft_consensus.cc:3055] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:11.388000 22148 raft_consensus.cc:740] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:11.388049 22187 consensus_queue.cc:237] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:11.389168 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:11.389781 16234 raft_consensus.cc:3060] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:21:11.388876 22148 consensus_queue.cc:260] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:11.392694 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15e5bd79113c48c5a0b0f265bdd1d7b2" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:11.378449 22311 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:21:11.404327 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 270423us
I20260709 13:21:11.405866 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d5c4641304e03a8a3fd4782b043e5" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:11.406495 16232 raft_consensus.cc:3060] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:11.333834 22307 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:11.400393 22145 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:11.399732 22199 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
W20260709 13:21:11.439239 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 178831us
I20260709 13:21:11.444288 16232 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
I20260709 13:21:11.448637 16234 raft_consensus.cc:2468] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 21.
I20260709 13:21:11.438118 22148 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:11.457050 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa44270fe124591b1e8c82c82b89efe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:11.460085 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:11.437503 22307 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:11.466790 22307 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:21:11.467941 22158 consensus_meta.cc:287] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f: Time spent flushing consensus metadata: real 0.799s	user 0.008s	sys 0.004s
I20260709 13:21:11.473945 21708 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:11.437521 22077 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:21:11.474644 21708 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:11.442287 22145 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:11.482255 22145 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:11.489445 22217 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:11.442643 22199 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:11.491937 22199 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:11.497874 22199 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:11.498389 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:11.498431 22199 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:11.502776 16233 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:11.503002 22199 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:11.502651 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:11.508522 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:11.510099 16236 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:21:11.511971 15597 raft_consensus.cc:2468] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
I20260709 13:21:11.477689 21708 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:11.528499 16240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:11.530076 22300 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:11.531257 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:11.554226 16234 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 14.
I20260709 13:21:11.532382 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:11.533692 16233 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 18.
I20260709 13:21:11.535738 15672 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:21:11.538362 22229 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:21:11.555555 15493 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:11.562436 15493 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:11.518864 22199 raft_consensus.cc:493] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:11.561858 22246 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:11.564921 22201 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:21:11.572022 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 147346us
I20260709 13:21:11.567323 22121 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:11.576767 16204 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:11.563730 21964 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:21:11.563953 22300 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:11.574604 22201 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:11.586197 22121 raft_consensus.cc:697] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:21:11.578361 22074 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.581094 22119 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:11.593696 22074 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:11.588320 22201 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:11.575091 22246 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:11.603901 22074 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:11.604468 22246 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:11.587395 22312 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:11.606144 22312 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:11.606959 16233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:11.590359 22121 consensus_queue.cc:237] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:11.610139 22300 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:11.572921 22199 raft_consensus.cc:515] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:11.612725 22199 leader_election.cc:290] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:11.616484 16232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:11.617624 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:11.623899 15672 raft_consensus.cc:1275] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 13:21:11.624248 15672 pending_rounds.cc:85] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting all ops after (but not including) 11
I20260709 13:21:11.624423 15672 pending_rounds.cc:107] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 13:21:11.624590 15672 raft_consensus.cc:2889] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:21:11.629490 22319 log.cc:927] Time spent T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 13:21:11.626909 22309 log.cc:927] Time spent T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.185s	user 0.003s	sys 0.000s
I20260709 13:21:11.643172 22094 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:11.646766 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:11.648343 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:11.648952 15521 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:21:11.651991 22094 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:11.652603 22094 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:11.654736 22094 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:11.626214 22199 raft_consensus.cc:493] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:11.656885 22199 raft_consensus.cc:515] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:11.660394 22158 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:11.669430 15668 raft_consensus.cc:1275] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:21:11.670337 22201 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:11.671180 22132 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:11.671775 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:11.677246 15520 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 20.
I20260709 13:21:11.673480 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:11.684687 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:11.684963 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:11.685237 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:11.685882 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:11.688059 22312 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:21:11.691748 15642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:11.694638 15525 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:21:11.695863 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:11.696529 15521 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:21:11.711351 15641 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:11.713546 15520 raft_consensus.cc:1240] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 LEADER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:11.722386 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
W20260709 13:21:11.722406 16205 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:11.728255 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:11.729096 15597 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 12.
I20260709 13:21:11.730485 15524 raft_consensus.cc:1240] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:11.732772 15491 leader_election.cc:400] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 17. Current term is 18.
I20260709 13:21:11.740319 15491 leader_election.cc:304] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:21:11.737783 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 48532us
I20260709 13:21:11.716765 22199 leader_election.cc:290] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:11.755759 15491 leader_election.cc:304] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:11.758028 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:11.767457 15491 leader_election.cc:400] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 17 election: Vote denied by peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779) with higher term. Message: Invalid argument: T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 17. Current term is 18.
I20260709 13:21:11.770260 15601 raft_consensus.cc:1240] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:11.782029 15567 leader_election.cc:304] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:11.787076 15567 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:11.770794 22229 raft_consensus.cc:2749] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:11.771158 22300 raft_consensus.cc:2749] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:21:11.789651 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:11.793524 15600 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 19.
I20260709 13:21:11.788025 15601 raft_consensus.cc:1275] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 13:21:11.792626 15596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b7542687ad43acb128a39bb1f40829" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:11.795200 15601 pending_rounds.cc:85] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting all ops after (but not including) 11
I20260709 13:21:11.795534 15601 pending_rounds.cc:107] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:21:11.795830 15601 raft_consensus.cc:2889] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:21:11.798842 22326 log.cc:927] Time spent T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:21:11.802917 22061 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:21:11.794086 22220 raft_consensus.cc:2749] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:11.810549 22324 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:11.811192 22324 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:11.826277 15567 leader_election.cc:304] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:11.827000 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:11.838660 15599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:11.839286 15599 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 19.
W20260709 13:21:11.841225 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 70137.6us
I20260709 13:21:11.843217 16204 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:21:11.870078 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 43927.6us
I20260709 13:21:11.876533 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:11.881786 15567 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:11.882761 21855 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:11.884280 21855 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:21:11.884673 21855 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:11.885079 21855 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:21:11.897945 22308 log.cc:927] Time spent T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.492s	user 0.001s	sys 0.003s
W20260709 13:21:11.927140 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 57899.9us
I20260709 13:21:11.924818 15493 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:21:11.934796 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 158409us
W20260709 13:21:11.935534 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 54916.2us
I20260709 13:21:11.933868 15642 leader_election.cc:304] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:11.938447 16237 raft_consensus.cc:1275] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:21:11.879987 22148 raft_consensus.cc:2749] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:11.880970 22229 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.881494 22064 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:11.936403 16205 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:11.955713 15642 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:11.929419 22013 leader_election.cc:290] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:12.006537 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 81604.4us
I20260709 13:21:12.007408 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
W20260709 13:21:12.007472 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 71179.1us
I20260709 13:21:11.881065 22011 consensus_queue.cc:1048] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 13:21:12.008648 21708 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:21:12.016419 22332 log.cc:927] Time spent T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:21:12.018101 22317 raft_consensus.cc:2749] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:21:12.019219 22302 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:12.019693 22302 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:12.020027 22302 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:12.007562 22200 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:21:12.005704 22229 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:12.025097 22229 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:12.022027 22336 log.cc:927] Time spent T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:21:12.027441 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 70191.5us
I20260709 13:21:11.957974 22201 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:12.039192 22208 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:12.034926 16205 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:12.006642 22121 raft_consensus.cc:493] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:12.040947 21855 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:12.055186 16205 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:21:12.061043 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 105388us
I20260709 13:21:12.044911 22303 consensus_queue.cc:1048] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.013s
I20260709 13:21:12.063339 22291 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:12.049014 22121 raft_consensus.cc:515] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:12.073434 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:12.073824 22114 raft_consensus.cc:493] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.062641 22318 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:12.071867 21855 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:12.083283 22294 consensus_queue.cc:1048] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:21:12.078509 22114 raft_consensus.cc:515] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:12.085670 22324 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:12.086886 22114 leader_election.cc:290] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:12.095748 22329 log.cc:927] Time spent T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.298s	user 0.002s	sys 0.000s
I20260709 13:21:12.125618 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:12.132805 15594 raft_consensus.cc:2955] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Committing config change with OpId 12.11: config changed from index 5 to 11, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:12.077760 22318 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:12.141566 22318 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:12.068461 22266 raft_consensus.cc:2955] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Committing config change with OpId 12.11: config changed from index 5 to 11, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:12.159962 22340 raft_consensus.cc:2804] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:21:12.173425 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 95507.4us
I20260709 13:21:12.179731 22340 raft_consensus.cc:697] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:12.186858 22340 consensus_queue.cc:237] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:12.198426 21708 raft_consensus.cc:493] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:12.199047 21708 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:12.201478 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b73eb6a0096455b87f6109b059cf0ee" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:12.202611 22276 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:12.203154 22276 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:12.213660 15568 leader_election.cc:304] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:12.210734 15672 raft_consensus.cc:2955] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Committing config change with OpId 12.11: config changed from index 5 to 11, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:21:12.219288 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 44148.3us
I20260709 13:21:12.220188 15642 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:12.228945 16231 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:21:12.230093 15594 raft_consensus.cc:1275] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:21:12.237012 22328 log.cc:927] Time spent T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.442s	user 0.001s	sys 0.001s
I20260709 13:21:12.240031 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:12.242738 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:12.243986 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2afa8e112a744f218227526d861b98c3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:12.245952 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50726e0d396948b79be43aece292acb5" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:12.248173 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:12.248687 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:12.253646 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:12.254312 21708 raft_consensus.cc:515] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:12.254804 22276 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:12.255834 15668 raft_consensus.cc:1240] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:12.258912 22315 consensus_queue.cc:1059] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:21:12.260262 22315 raft_consensus.cc:3055] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:21:12.234084 22210 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.265131 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:12.266747 21708 leader_election.cc:290] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 21 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:12.271243 22121 leader_election.cc:290] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:12.255208 22081 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:12.275430 22210 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:12.276641 22149 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:12.277226 22149 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:12.279244 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:12.279878 15519 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 19.
I20260709 13:21:12.284166 16204 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:12.293161 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:12.300539 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:12.277657 22210 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:12.260594 22315 raft_consensus.cc:740] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:12.228912 22302 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:12.322942 22149 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:12.323707 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:12.324033 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef08c91ca56042df89e729f13df83ed9" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:12.316573 22054 raft_consensus.cc:2749] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:12.303653 22303 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:12.327540 22303 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:12.312287 22180 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:12.329430 22180 raft_consensus.cc:493] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:12.333039 22081 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:12.333614 22081 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:12.334180 15568 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:12.335358 22276 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:12.338022 22180 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:12.341138 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:12.344431 22266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:45438 (request call id 3) took 2276 ms. Trace:
I20260709 13:21:12.358208 22318 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:12.364459 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:12.365356 15491 leader_election.cc:304] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:12.366832 15491 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:12.370021 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:12.344926 22266 rpcz_store.cc:276] 0709 13:21:10.067605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.067850 (+   245us) service_pool.cc:224] Handling call
0709 13:21:10.415551 (+347701us) spinlock_profiling.cc:243] Waited 29.8 ms on lock 0x7b5400850fa0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce497fa07 00007f5ce4a06749 00007f5ce4a0776a 00007f5ce4a0691b 00007f5ce4a1203c 00007f5ce4a116b3 00007f5ce4a099fa 00007f5ce99d14c9 00007f5ce2f4dc54 00007f5ce2f4dbe2
0709 13:21:10.417654 (+  2103us) log.cc:844] Serialized 494 byte log entry
0709 13:21:11.056866 (+639212us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b5400850fa0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4984142 00007f5ce4964eaa 00007f5ce496bd53 00007f5ce496bcd9 00007f5ce496bc69 00007f5ce496bc31 00007f5ce496adfd 000000000038f354 000000000038f199 00007f5ce191e475
0709 13:21:11.487503 (+430637us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b5400850fa0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce498bb26 00007f5ce498549a 00007f5ce4984714 00007f5ce4964eaa 00007f5ce496bd53 00007f5ce496bcd9 00007f5ce496bc69 00007f5ce496bc31 00007f5ce496adfd 000000000038f354
0709 13:21:11.669696 (+182193us) spinlock_profiling.cc:243] Waited 8.7 ms on lock 0x7b180038e180. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce191e3f3 00007f5ce19207c1 00007f5ce1920779 00007f5ce1920709 00007f5ce19206d1 00007f5ce191f9cd 000000000038f354 000000000038f199 00007f5ce190183a
0709 13:21:12.344331 (+674635us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75885,"raft.queue_time_us":96279,"raft.run_cpu_time_us":51049,"raft.run_wall_time_us":1263669,"spinlock_wait_cycles":167812736,"thread_start_us":99017,"threads_started":2,"wal-append.queue_time_us":300859}
I20260709 13:21:12.369253 21708 consensus_queue.cc:1048] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:21:12.371662 22199 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:12.369383 22246 raft_consensus.cc:2749] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:12.375677 22180 raft_consensus.cc:515] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:12.376784 15491 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:12.377854 22246 raft_consensus.cc:2749] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:12.379156 15421 catalog_manager.cc:5697] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 12 to 15, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 15 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:12.382315 15412 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 582343936346420b87ecf2eebd00dc25 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 13)
I20260709 13:21:12.380993 15421 catalog_manager.cc:5697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 15 to 19, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 19 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:12.329757 22303 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:12.402057 22302 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 20 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:12.402858 22315 consensus_queue.cc:260] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:12.404273 22315 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:12.406643 22180 leader_election.cc:290] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:12.407294 15597 raft_consensus.cc:1240] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:21:12.409842 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:12.410979 22120 consensus_queue.cc:1059] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:12.412099 22210 raft_consensus.cc:3055] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:21:12.412525 22210 raft_consensus.cc:740] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:12.413403 22210 consensus_queue.cc:260] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:12.431226 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:12.432559 15519 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:12.394872 15421 catalog_manager.cc:5697] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 11 to 12, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 12 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:12.442365 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
W20260709 13:21:12.443719 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 72931.3us
I20260709 13:21:12.451550 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:12.453542 16204 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:12.457469 15520 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 19. Current term is 20. Ops: []
W20260709 13:21:12.461241 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 453892us
I20260709 13:21:12.462424 15493 leader_election.cc:304] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:12.465561 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e80bd01b0c4006852da32ca0eeb0ae" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:12.467137 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:12.462901 21708 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.006s
I20260709 13:21:12.472961 22257 consensus_queue.cc:1059] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:12.454486 22318 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:12.474190 22257 raft_consensus.cc:3055] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:12.474812 22257 raft_consensus.cc:740] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:12.475672 22257 consensus_queue.cc:260] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:12.474267 22291 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:12.491583 16235 raft_consensus.cc:1240] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:21:12.460179 22210 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:12.500258 22110 consensus_queue.cc:1059] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:12.501142 22262 consensus_queue.cc:1059] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:12.503250 16234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6670c20b1954b36a4920b44d1a8209d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:12.527086 16234 raft_consensus.cc:2393] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 20: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:21:12.464627 22246 raft_consensus.cc:2749] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:12.506364 22081 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:12.509372 16238 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:21:12.510262 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:12.513684 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a5f6755312b4dc09781ecf8a9468c17" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:12.523322 15519 raft_consensus.cc:2468] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 20.
I20260709 13:21:12.525666 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:12.535957 15493 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:12.530261 15421 catalog_manager.cc:5697] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:12.540629 16204 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:12.533227 22257 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:12.532001 22121 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.543359 22246 raft_consensus.cc:2749] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:12.581454 22081 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:12.582670 15594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:12.582412 15421 catalog_manager.cc:5697] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 7 to 18. New cstate: current_term: 18 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:12.583987 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:12.584146 15421 catalog_manager.cc:5697] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 8 to 18. New cstate: current_term: 18 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:12.555474 22110 raft_consensus.cc:2804] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:12.592177 16231 raft_consensus.cc:2468] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 21.
I20260709 13:21:12.600884 15421 catalog_manager.cc:5697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 11 to 19, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 19 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:12.602732 15421 catalog_manager.cc:5697] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:12.608398 15421 catalog_manager.cc:5697] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 7 to 11, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 11 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:12.613843 22340 raft_consensus.cc:3055] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:12.618716 22291 raft_consensus.cc:3055] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:12.572300 22121 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:21:12.622484 22164 raft_consensus.cc:493] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:12.628791 15642 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:12.600135 22110 raft_consensus.cc:697] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:12.631317 22340 raft_consensus.cc:740] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:12.643714 15421 catalog_manager.cc:5697] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:12.632576 22291 raft_consensus.cc:740] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:12.649717 15642 leader_election.cc:304] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:12.653275 22094 raft_consensus.cc:2804] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:12.653849 22094 raft_consensus.cc:697] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:12.654712 22094 consensus_queue.cc:237] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:12.655236 15421 catalog_manager.cc:5697] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:12.656576 22110 consensus_queue.cc:237] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:12.657670 15421 catalog_manager.cc:5697] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 14 to 18, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 18 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:12.661007 22121 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:12.659741 15421 catalog_manager.cc:5697] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:12.639178 22164 raft_consensus.cc:515] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:12.664580 22164 leader_election.cc:290] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:12.666123 22291 consensus_queue.cc:260] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:12.666582 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:12.667549 22291 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:12.671310 22340 consensus_queue.cc:260] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:12.672667 22340 raft_consensus.cc:3060] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:12.645149 22081 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:12.677624 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:21:12.699784 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 153594us
I20260709 13:21:12.700644 15493 leader_election.cc:304] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:12.707630 22246 raft_consensus.cc:2749] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:12.708844 15493 leader_election.cc:304] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:12.723295 15526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:12.729637 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:12.734918 15522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5315557f994ec689bb9a79498bb2d5" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:12.736367 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:12.723538 22331 raft_consensus.cc:2749] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:12.718658 22199 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 13:21:12.769161 15521 raft_consensus.cc:1240] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Rejecting Update request from peer 3ef186ec12554a1d87814af0a0ace9a4 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:12.771884 15519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c48036212bb4277acf36aac79d8b35e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:12.779542 15597 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:21:12.779949 15520 raft_consensus.cc:1275] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:21:12.789776 16231 raft_consensus.cc:1240] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Rejecting Update request from peer d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:12.792567 15601 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:21:12.793388 15567 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:12.794549 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:12.799830 22342 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:21:12.803861 15493 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:12.804641 15567 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:12.807772 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:12.813180 22331 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:12.828086 15668 raft_consensus.cc:1240] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Rejecting Update request from peer dd1a14c3173d49dc9b55bb116e3783c9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:12.814929 22149 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:12.823342 22324 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:12.850492 15567 leader_election.cc:304] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:12.834419 22114 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:21:12.847326 22081 consensus_queue.cc:1048] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:21:12.886559 22351 consensus_queue.cc:1059] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 13:21:12.845858 22110 consensus_queue.cc:1059] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:21:12.888348 22110 raft_consensus.cc:3055] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:12.888725 22110 raft_consensus.cc:740] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:12.890074 22110 consensus_queue.cc:260] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:12.891577 22110 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:21:12.895367 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 44956.5us
I20260709 13:21:12.898733 15567 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:12.901866 22212 raft_consensus.cc:2749] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:12.904088 22012 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:21:12.938396 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 103436us
W20260709 13:21:12.938962 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 120898us
W20260709 13:21:12.946970 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 104129us
I20260709 13:21:12.948966 16231 raft_consensus.cc:1275] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:21:12.952605 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 155595us
W20260709 13:21:12.953047 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 136283us
I20260709 13:21:12.957511 22303 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.974298 22303 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:12.970239 22217 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.980362 22217 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:12.981995 22331 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:12.964979 22342 consensus_queue.cc:1048] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:21:12.995669 22121 raft_consensus.cc:3055] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:12.996161 22121 raft_consensus.cc:740] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:21:12.998601 15493 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:12.996960 22121 consensus_queue.cc:260] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:13.014647 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 48746.8us
I20260709 13:21:13.017841 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:13.017871 15524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:13.027570 22347 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:13.037277 22121 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:13.080700 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:13.081029 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:13.081398 16239 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:21:13.082387 15568 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:13.085840 15568 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:13.090459 22054 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:13.090947 22054 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:13.102097 22303 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:13.107372 15528 raft_consensus.cc:3060] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:13.083467 22217 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:13.096336 22276 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:13.122722 22054 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:13.196771 22111 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:13.197350 22111 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:13.201375 22358 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:13.202010 22358 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:13.204422 22358 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:13.208340 15520 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:21:13.210103 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:13.214340 15528 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:21:13.222674 22111 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:13.225147 22318 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:13.225627 15525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:13.226537 15525 raft_consensus.cc:2393] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 15: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:21:13.227325 16235 raft_consensus.cc:2393] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 14: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:21:13.230712 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:13.231371 15672 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 15.
I20260709 13:21:13.234335 16203 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:13.235534 22111 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:13.235999 22111 raft_consensus.cc:493] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:13.236189 22342 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:21:13.236922 22366 log.cc:927] Time spent T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 13:21:13.256055 16231 raft_consensus.cc:1275] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:21:13.250969 22111 raft_consensus.cc:3060] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:13.270310 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:13.262459 22342 consensus_queue.cc:1048] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.010s
I20260709 13:21:13.285395 15668 raft_consensus.cc:1275] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 16. (term mismatch)
I20260709 13:21:13.285822 15668 pending_rounds.cc:85] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4: Aborting all ops after (but not including) 15
I20260709 13:21:13.286075 15668 pending_rounds.cc:107] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4: Aborting uncommitted NO_OP operation due to leader change: 19.16
I20260709 13:21:13.286293 15668 raft_consensus.cc:2889] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:13.313719 22318 consensus_queue.cc:1048] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.010s
I20260709 13:21:13.375726 22111 raft_consensus.cc:515] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:13.329759 22054 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:13.398680 15601 raft_consensus.cc:2368] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 14. Current term is 15.
I20260709 13:21:13.401736 15493 leader_election.cc:400] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 14. Current term is 15.
I20260709 13:21:13.402092 15493 leader_election.cc:403] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:13.403126 22121 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:21:13.418932 22054 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:13.420420 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:13.420933 16238 raft_consensus.cc:3060] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:13.422259 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dbabc834ad54c16ae25f79064932f42" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:13.423903 22121 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593) with higher term. Message: Invalid argument: T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for earlier term 14. Current term is 15.
I20260709 13:21:13.432397 15520 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:21:13.433990 16238 raft_consensus.cc:2468] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:21:13.435266 15568 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:13.440579 22054 raft_consensus.cc:2804] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:13.441532 22054 raft_consensus.cc:697] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:13.442772 22054 consensus_queue.cc:237] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:13.452551 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:13.452901 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7562b5f342134e39a5e6934d5ca16e46" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:13.453441 15520 raft_consensus.cc:2393] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 15: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:21:13.458056 22111 leader_election.cc:290] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:13.466472 15668 raft_consensus.cc:2468] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 15.
I20260709 13:21:13.467571 16203 leader_election.cc:304] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:13.473785 22291 raft_consensus.cc:2804] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:13.474332 22291 raft_consensus.cc:697] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:13.475142 22291 consensus_queue.cc:237] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:13.563648 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:13.680452 22121 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:13.687609 22121 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:13.689740 22121 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:13.690636 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:13.691131 15595 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:59198
I20260709 13:21:13.691330 16238 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:21:13.691728 15595 raft_consensus.cc:606] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:13.690634 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:13.693085 15601 raft_consensus.cc:2393] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 11: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:21:13.694128 15493 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:13.694957 22121 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:21:13.695403 22121 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:13.695832 22121 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:13.715642 15424 catalog_manager.cc:5697] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 10 to 16, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 16 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:13.719941 15424 catalog_manager.cc:5697] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 12 to 14, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 14 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:13.722710 15421 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58802 (request call id 263) took 18197 ms (client timeout 15000 ms). Trace:
W20260709 13:21:13.723934 15412 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 582343936346420b87ecf2eebd00dc25 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica dd1a14c3173d49dc9b55bb116e3783c9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:13.724004 16238 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:21:13.725569 16238 raft_consensus.cc:606] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:13.730166 16238 raft_consensus.cc:614] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:13.724037 15424 catalog_manager.cc:5697] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:13.731520 22121 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:13.734259 15424 catalog_manager.cc:5697] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 8 to 12, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 12 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
W20260709 13:21:13.724397 15421 rpcz_store.cc:269] 0709 13:20:55.524991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:55.525199 (+   208us) service_pool.cc:224] Handling call
0709 13:21:13.722634 (+18197435us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:12.935060 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:12.973223 (+ 38163us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:12.973263 (+    40us) write_op.cc:435] Acquired schema lock
0709 13:21:12.973281 (+    18us) tablet.cc:662] Decoding operations
0709 13:21:12.974498 (+  1217us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:12.974577 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:12.974604 (+    27us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:21:12.978874 (+  4270us) tablet.cc:701] Row locks acquired
0709 13:21:12.978895 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:21:12.979046 (+   151us) write_op.cc:270] Start()
0709 13:21:12.979161 (+   115us) write_op.cc:276] Timestamp: P: 1783603272978998 usec, L: 0
0709 13:21:12.979181 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:21:13.046178 (+ 66997us) log.cc:844] Serialized 5094 byte log entry
0709 13:21:13.172419 (+126241us) spinlock_profiling.cc:243] Waited 33.6 ms on lock 0x7b5400369554. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a06e28 00007f5ce8441581 00007f5ce8440d58 00007f5ce84459f1 00007f5ce84459a9 00007f5ce8445939 00007f5ce8445901 00007f5ce8444b2d 000000000038f354 000000000038f199
0709 13:21:13.384140 (+211721us) op_driver.cc:464] REPLICATION: finished
0709 13:21:13.559839 (+175699us) write_op.cc:301] APPLY: starting
0709 13:21:13.559931 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:13.586727 (+ 26796us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:13.586757 (+    30us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:13.605875 (+ 19118us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:13.634611 (+ 28736us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=0,mrs_lookups=0
0709 13:21:13.634659 (+    48us) write_op.cc:312] APPLY: finished
0709 13:21:13.634986 (+   327us) log.cc:844] Serialized 121 byte log entry
0709 13:21:13.635168 (+   182us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:13.635306 (+   138us) write_op.cc:454] Released schema lock
0709 13:21:13.635357 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:13.644327 (+  8970us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:13.644363 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6022,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":175462,"apply.run_cpu_time_us":15662,"apply.run_wall_time_us":78604,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58571,"num_ops":13,"prepare.queue_time_us":265752,"prepare.run_cpu_time_us":10164,"prepare.run_wall_time_us":238381,"raft.queue_time_us":108935,"raft.run_cpu_time_us":41255,"raft.run_wall_time_us":230628,"replication_time_us":365106,"spinlock_wait_cycles":77301120,"thread_start_us":484308,"threads_started":4,"wal-append.queue_time_us":63455}]]}
I20260709 13:21:13.735718 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:13.736088 15601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:59198
I20260709 13:21:13.742411 15601 raft_consensus.cc:606] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:13.742784 15601 raft_consensus.cc:614] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:13.737308 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:13.737167 15424 catalog_manager.cc:5697] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 10 to 15, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 15 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:13.743896 15595 raft_consensus.cc:2393] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 11: Already voted for candidate 5498ef3e89a941e39c25f68cf64e6b0f in this term.
I20260709 13:21:13.750576 15595 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:59198
I20260709 13:21:13.752053 15595 raft_consensus.cc:606] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:13.752470 15595 raft_consensus.cc:614] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:13.755496 15601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:59198
I20260709 13:21:13.756083 15601 raft_consensus.cc:606] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 25 LEADER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:13.760366 15601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ccb13c2addbf41569d0c2c23750bd4c9"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:59198
I20260709 13:21:13.760926 15601 raft_consensus.cc:606] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:13.761247 15601 raft_consensus.cc:614] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:13.762303 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:21:13.762451 22358 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:13.763067 22358 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:13.764089 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:21:13.733666 22121 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:13.768446 15672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:13.769054 15672 raft_consensus.cc:1813] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Handling vote request from an unknown peer d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:13.769515 15672 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 in current term 13: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:21:13.771170 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:13.772598 16231 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 11.
I20260709 13:21:13.772782 22358 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:13.776633 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:13.778283 16231 raft_consensus.cc:1813] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Handling vote request from an unknown peer d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:13.778920 16231 raft_consensus.cc:2410] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3c4f5bd76e0445795bad5f6dd93ff69 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:21:13.777434 15493 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:13.780074 22358 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:13.790066 15493 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:13.794656 22121 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:21:13.795303 22121 raft_consensus.cc:697] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:21:13.796257 22121 consensus_queue.cc:237] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:13.818393 22358 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:13.855836 22054 raft_consensus.cc:993] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:21:13.856686 22276 raft_consensus.cc:1081] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Signalling peer 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:21:13.860064 15675 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:49100
I20260709 13:21:13.860519 15675 raft_consensus.cc:493] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:13.860807 15675 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:13.865247 15425 catalog_manager.cc:5697] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:13.875562 15675 raft_consensus.cc:515] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:13.877674 15675 leader_election.cc:290] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:13.882303 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:13.882948 15601 raft_consensus.cc:3055] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:13.883239 15601 raft_consensus.cc:740] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:13.883975 15601 consensus_queue.cc:260] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:13.885036 15601 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:13.887216 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:13.887820 15528 raft_consensus.cc:3060] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:13.889835 16238 raft_consensus.cc:1275] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:21:13.896919 22235 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:13.901836 15528 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
I20260709 13:21:13.905211 15601 raft_consensus.cc:2468] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
I20260709 13:21:13.904525 15425 catalog_manager.cc:5697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 19 to 21, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 21 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:13.903401 15641 leader_election.cc:304] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:13.906348 22209 raft_consensus.cc:2804] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:13.906858 22209 raft_consensus.cc:697] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:13.906864 15425 catalog_manager.cc:5697] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:13.907750 22209 consensus_queue.cc:237] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:13.910151 15425 catalog_manager.cc:5697] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 15 to 17, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 17 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:13.911962 15425 catalog_manager.cc:5697] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:13.920503 15425 catalog_manager.cc:5697] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 13 to 19, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 19 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:13.922111 15425 catalog_manager.cc:5697] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 12 to 17, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 17 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:13.923885 15425 catalog_manager.cc:5697] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 15 to 20, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 20 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:13.948714 15675 raft_consensus.cc:1275] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:21:14.042379 15425 catalog_manager.cc:5697] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:14.042994 22120 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.046140 22120 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:14.062626 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:14.062968 22315 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:14.063998 15521 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 20.
I20260709 13:21:14.066314 15425 catalog_manager.cc:5697] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 16 to 18, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 18 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:14.067790 15528 raft_consensus.cc:1275] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:21:14.074441 16204 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:14.078610 22318 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:14.079185 22318 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:14.079535 22318 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:14.086822 22217 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:14.087838 22217 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:14.090785 22217 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:14.097522 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:14.099941 15528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:14.101054 15528 raft_consensus.cc:2393] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 20: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:21:14.079667 22054 consensus_queue.cc:1048] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:21:14.113538 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:14.114578 16237 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 20.
I20260709 13:21:14.118326 15521 raft_consensus.cc:1275] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:21:14.118477 22369 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:14.119469 22120 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:14.119395 22369 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:14.124822 15664 tablet_service.cc:1558] Processing DeleteTablet for tablet 0af5649725244ae599dfc6ed7aeac5f1 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 5 (current committed config index is 7)) from {username='slave'} at 127.0.0.1:49088
I20260709 13:21:14.161316 22120 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.164455 22120 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:14.069538 15425 catalog_manager.cc:5697] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:14.175125 15568 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:14.176270 22217 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:14.177024 22217 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:14.177433 22217 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:14.179538 22369 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:14.186578 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 91346.7us
I20260709 13:21:14.191148 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:21:14.192490 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 93590.6us
I20260709 13:21:14.172966 15425 catalog_manager.cc:5697] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 13 to 17, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 17 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:21:14.177357 15424 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49144 (request call id 354) took 36007 ms (client timeout 15000 ms). Trace:
I20260709 13:21:14.193941 16237 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 13.
I20260709 13:21:14.198011 15568 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:14.203840 22276 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:14.204327 22276 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:14.204759 22276 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:14.206743 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:14.212774 22217 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:14.216879 22217 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:14.195017 15424 rpcz_store.cc:269] 0709 13:20:38.169315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.195330 (+ 26015us) service_pool.cc:224] Handling call
0709 13:21:13.814528 (+35619198us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:13.814557 (+    29us) write_op.cc:454] Released schema lock
0709 13:21:14.177093 (+362536us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:13.774586 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:13.775110 (+   524us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:13.775140 (+    30us) write_op.cc:435] Acquired schema lock
0709 13:21:13.775164 (+    24us) tablet.cc:662] Decoding operations
0709 13:21:13.775791 (+   627us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:13.775866 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:13.775893 (+    27us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:21:13.776370 (+   477us) tablet.cc:701] Row locks acquired
0709 13:21:13.776384 (+    14us) write_op.cc:260] PREPARE: finished
0709 13:21:13.776520 (+   136us) write_op.cc:270] Start()
0709 13:21:13.776622 (+   102us) write_op.cc:276] Timestamp: P: 1783603273776475 usec, L: 0
0709 13:21:13.776639 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:21:13.777251 (+   612us) log.cc:844] Serialized 2065 byte log entry
0709 13:21:13.779491 (+  2240us) op_driver.cc:464] REPLICATION: finished
0709 13:21:13.779933 (+   442us) write_op.cc:301] APPLY: starting
0709 13:21:13.780007 (+    74us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:13.782738 (+  2731us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:13.782875 (+   137us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:13.786145 (+  3270us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:13.786318 (+   173us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=0,mrs_lookups=0
0709 13:21:13.786358 (+    40us) write_op.cc:312] APPLY: finished
0709 13:21:13.786570 (+   212us) log.cc:844] Serialized 57 byte log entry
0709 13:21:13.787323 (+   753us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:13.787428 (+   105us) write_op.cc:454] Released schema lock
0709 13:21:13.787467 (+    39us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":12659,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":246,"apply.run_cpu_time_us":7223,"apply.run_wall_time_us":7808,"cfile_cache_hit":20,"cfile_cache_hit_bytes":20940,"num_ops":5,"prepare.queue_time_us":23605,"prepare.run_cpu_time_us":2969,"prepare.run_wall_time_us":13365,"replication_time_us":2713,"thread_start_us":63538,"threads_started":1}]]}
I20260709 13:21:14.226457 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:14.229426 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:14.230434 15521 raft_consensus.cc:2393] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 20: Already voted for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in this term.
I20260709 13:21:14.237604 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:14.238693 15668 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 13: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:21:14.242373 15425 catalog_manager.cc:5697] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 22 to 25. New cstate: current_term: 25 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:14.243757 22180 consensus_queue.cc:1048] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:14.245126 15425 catalog_manager.cc:5697] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 10 to 11, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 11 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:14.256894 22394 raft_consensus.cc:993] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:21:14.223474 22120 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:14.268802 22218 raft_consensus.cc:1081] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 25 LEADER]: Signalling peer 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:21:14.259438 15595 raft_consensus.cc:1275] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:21:14.259667 15410 catalog_manager.cc:5028] TS 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): tablet 0af5649725244ae599dfc6ed7aeac5f1 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:21:14.263311 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:14.273826 15668 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:21:14.275312 16203 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:14.276224 22120 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:14.276636 22120 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:14.276996 22120 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:14.280339 22390 consensus_queue.cc:1048] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:14.294982 16238 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 20.
I20260709 13:21:14.285707 22276 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:14.301249 15568 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:14.291340 22318 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:14.310726 15597 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:21:14.321262 15425 catalog_manager.cc:5697] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 13 to 15, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 15 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:14.323498 15425 catalog_manager.cc:5697] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:14.325734 15425 catalog_manager.cc:5697] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 16 to 19, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 19 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:14.329084 15425 catalog_manager.cc:5697] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:14.333597 22318 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:14.334784 15520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
I20260709 13:21:14.307562 22220 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:14.335601 15520 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:14.337039 15425 catalog_manager.cc:5697] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:14.349678 15425 catalog_manager.cc:5697] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 6 to 8, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 8 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:14.357559 22220 raft_consensus.cc:697] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:14.364691 22220 consensus_queue.cc:237] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:14.359596 22276 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:14.361646 15520 raft_consensus.cc:2468] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 21.
I20260709 13:21:14.370553 16204 leader_election.cc:304] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:14.393600 15425 catalog_manager.cc:5697] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 8 to 10, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 10 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:14.398790 15425 catalog_manager.cc:5697] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:14.400542 15425 catalog_manager.cc:5697] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 12 to 13, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 13 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:14.407327 22120 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:14.407023 15425 catalog_manager.cc:5697] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 11 to 14, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 14 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:21:14.409343 15425 catalog_manager.cc:5697] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 18 to 19, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 19 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:14.411127 15425 catalog_manager.cc:5697] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 13 to 16, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 16 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:21:14.409624 22120 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:21:14.415544 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 103220us
I20260709 13:21:14.412920 15425 catalog_manager.cc:5697] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:14.420408 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:14.429039 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:14.429971 15674 raft_consensus.cc:2393] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f in current term 13: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:21:14.415755 22180 raft_consensus.cc:2804] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:14.431092 15425 catalog_manager.cc:5697] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 9 to 13, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 13 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
W20260709 13:21:14.448719 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 141217us
I20260709 13:21:14.454334 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:14.456192 15595 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:14.461558 15677 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:49100
I20260709 13:21:14.462365 15677 raft_consensus.cc:493] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 25 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:14.462742 15677 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:14.469383 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:14.470104 15674 raft_consensus.cc:3060] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:14.451829 22180 raft_consensus.cc:697] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:14.485960 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:14.486881 16239 raft_consensus.cc:2368] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 13. Current term is 14.
I20260709 13:21:14.489457 22180 consensus_queue.cc:237] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:14.502697 15528 raft_consensus.cc:1275] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:21:14.505162 21708 consensus_queue.cc:1048] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:21:14.506196 15595 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:21:14.507520 16205 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:14.508391 22315 raft_consensus.cc:2804] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:14.508925 22315 raft_consensus.cc:697] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:14.509887 22315 consensus_queue.cc:237] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:14.498456 15425 catalog_manager.cc:5697] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:14.513659 15568 leader_election.cc:400] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 13. Current term is 14.
I20260709 13:21:14.498905 15674 raft_consensus.cc:2468] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:21:14.514034 15568 leader_election.cc:403] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:21:14.516237 22217 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) with higher term. Message: Invalid argument: T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 13. Current term is 14.
I20260709 13:21:14.519676 15677 raft_consensus.cc:515] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 26 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:14.524551 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 73262us
I20260709 13:21:14.513242 15425 catalog_manager.cc:5697] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 10 to 14, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 14 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:14.530745 22199 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.531359 22199 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:14.533841 22199 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:14.534801 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:14.536913 22409 raft_consensus.cc:493] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:14.537606 22409 raft_consensus.cc:515] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:14.538084 15677 leader_election.cc:290] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 26 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:14.539842 22409 leader_election.cc:290] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:14.540346 16237 raft_consensus.cc:1275] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:21:14.542053 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:14.542625 15597 raft_consensus.cc:3055] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:21:14.542943 15597 raft_consensus.cc:740] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:14.543718 15597 consensus_queue.cc:260] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:14.544812 15521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" is_pre_election: true
I20260709 13:21:14.547304 15521 raft_consensus.cc:2368] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 20. Current term is 21.
I20260709 13:21:14.547844 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:14.548516 16237 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 20.
I20260709 13:21:14.551155 22121 consensus_queue.cc:1048] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:14.544889 15597 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:14.556408 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:14.558804 15567 leader_election.cc:400] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 pre-election: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 20. Current term is 21.
I20260709 13:21:14.559172 15567 leader_election.cc:403] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:14.559911 22054 raft_consensus.cc:3060] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:21:14.557094 16237 raft_consensus.cc:3060] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:14.571130 22054 raft_consensus.cc:2749] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481) with higher term. Message: Invalid argument: T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 20. Current term is 21.
I20260709 13:21:14.578739 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb5853a05664246a26d3fc7ecf92777" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:14.585788 15595 raft_consensus.cc:1275] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:21:14.587301 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:14.589715 22359 consensus_queue.cc:1048] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:21:14.592978 22121 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:14.593477 22121 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:14.595495 22121 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:14.593639 15425 catalog_manager.cc:5697] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 12 to 14, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 14 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:21:14.599664 15425 catalog_manager.cc:5697] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:14.604527 15425 catalog_manager.cc:5697] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 14 to 16, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 16 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:14.606828 22121 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:14.619797 15597 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 26.
I20260709 13:21:14.627677 15425 catalog_manager.cc:5697] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 13 to 14, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 14 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:21:14.654424 15425 catalog_manager.cc:5697] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 15 to 16, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 16 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:21:14.659732 16237 raft_consensus.cc:2468] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 26.
I20260709 13:21:14.676213 15642 leader_election.cc:304] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:21:14.677289 22342 raft_consensus.cc:2804] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:21:14.677785 22342 raft_consensus.cc:697] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 26 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:14.680047 22342 consensus_queue.cc:237] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:14.715982 22121 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 21 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:14.748378 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 84300.2us
I20260709 13:21:14.760597 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:14.766435 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:14.767798 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:14.768440 16235 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:14.791854 16235 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 21.
I20260709 13:21:14.804615 15493 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:14.807255 22420 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:14.811141 22420 raft_consensus.cc:697] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:21:14.812016 22420 consensus_queue.cc:237] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:14.829957 16235 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:14.831867 22054 consensus_queue.cc:1059] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:14.832644 15521 raft_consensus.cc:1240] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 LEADER]: Rejecting Update request from peer 5498ef3e89a941e39c25f68cf64e6b0f for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:14.835965 22414 raft_consensus.cc:3055] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:21:14.836385 22414 raft_consensus.cc:740] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:14.836493 22220 consensus_queue.cc:1059] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:14.837188 22220 consensus_queue.cc:1243] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Peer d3c4f5bd76e0445795bad5f6dd93ff69 log is divergent from this leader: its last log entry 21.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:21:14.840425 22414 consensus_queue.cc:260] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:14.841842 22414 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:14.903340 15521 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:21:14.905264 22406 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:21:14.918641 15677 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:21:14.927591 22406 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:14.925591 15601 raft_consensus.cc:1275] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:21:14.932190 22406 consensus_queue.cc:1048] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:15.050024 15601 raft_consensus.cc:1275] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:21:15.062548 22318 consensus_queue.cc:1048] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:15.094798 15595 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 26 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:21:15.099833 21708 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:21:15.185392 16235 raft_consensus.cc:1275] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 26 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:21:15.187463 22209 consensus_queue.cc:1048] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 13:21:15.266520 15412 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 582343936346420b87ecf2eebd00dc25 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 5 but the committed config has opid_index of 11
I20260709 13:21:15.363500 16235 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:21:15.371042 22199 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:15.379829 15601 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 15. (term mismatch)
I20260709 13:21:15.380347 15601 pending_rounds.cc:85] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting all ops after (but not including) 14
I20260709 13:21:15.380573 15601 pending_rounds.cc:107] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f: Aborting uncommitted NO_OP operation due to leader change: 20.15
I20260709 13:21:15.380877 15601 raft_consensus.cc:2889] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:15.392046 22199 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:15.427217 15420 catalog_manager.cc:5697] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 16 to 19, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 19 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:15.435662 15425 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49140 (request call id 365) took 4079 ms. Trace:
I20260709 13:21:15.438551 15425 rpcz_store.cc:276] 0709 13:21:11.356672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:11.359002 (+  2330us) service_pool.cc:224] Handling call
0709 13:21:15.435560 (+4076558us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:14.754061 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:14.754690 (+   629us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:14.754741 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:21:14.754760 (+    19us) tablet.cc:662] Decoding operations
0709 13:21:14.805762 (+ 51002us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:14.805873 (+   111us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:14.805902 (+    29us) tablet.cc:685] Acquiring locks for 35 operations
0709 13:21:14.808298 (+  2396us) tablet.cc:701] Row locks acquired
0709 13:21:14.808352 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:21:14.808517 (+   165us) write_op.cc:270] Start()
0709 13:21:14.808659 (+   142us) write_op.cc:276] Timestamp: P: 1783603274808454 usec, L: 0
0709 13:21:14.808695 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 13:21:14.809489 (+   794us) log.cc:844] Serialized 13510 byte log entry
0709 13:21:14.861665 (+ 52176us) op_driver.cc:464] REPLICATION: finished
0709 13:21:14.901889 (+ 40224us) write_op.cc:301] APPLY: starting
0709 13:21:14.902040 (+   151us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:15.289300 (+387260us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:15.289444 (+   144us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:15.342122 (+ 52678us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:15.342602 (+   480us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=70,key_file_lookups=70,delta_file_lookups=0,mrs_lookups=0
0709 13:21:15.342664 (+    62us) write_op.cc:312] APPLY: finished
0709 13:21:15.345590 (+  2926us) log.cc:844] Serialized 300 byte log entry
0709 13:21:15.345807 (+   217us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:15.346104 (+   297us) write_op.cc:454] Released schema lock
0709 13:21:15.346194 (+    90us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:15.346665 (+   471us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:15.346703 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":104791,"reported_tablets":44,"spinlock_wait_cycles":1920,"child_traces":[["op",{"apply.queue_time_us":39951,"apply.run_cpu_time_us":44809,"apply.run_wall_time_us":444760,"cfile_cache_hit":142,"cfile_cache_hit_bytes":150707,"num_ops":35,"prepare.queue_time_us":3161,"prepare.run_cpu_time_us":8199,"prepare.run_wall_time_us":96303,"raft.queue_time_us":25082,"raft.run_cpu_time_us":16992,"raft.run_wall_time_us":22241,"replication_time_us":52778,"thread_start_us":95971,"threads_started":4,"wal-append.queue_time_us":23181}]]}
I20260709 13:21:15.441900 15420 catalog_manager.cc:5697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 19 to 20, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 20 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:15.470072 15420 catalog_manager.cc:5697] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 13 to 14, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 14 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:15.483433 15420 catalog_manager.cc:5697] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 18 to 20, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 20 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:15.497507 15420 catalog_manager.cc:5697] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 13 to 16, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 16 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:15.501013 15420 catalog_manager.cc:5697] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 15 to 18, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 18 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:15.521723 15420 catalog_manager.cc:5697] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 12 to 14, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 14 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:21:15.809161 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 70284.5us
I20260709 13:21:15.820372 22440 raft_consensus.cc:493] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:15.888768 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 78906.8us
I20260709 13:21:15.877020 22440 raft_consensus.cc:515] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } attrs { replace: true } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:15.978477 22440 leader_election.cc:290] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:16.010883 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:16.012181 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:21:16.043726 15491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 51240.5us
I20260709 13:21:16.052309 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582343936346420b87ecf2eebd00dc25" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:16.112195 15493 leader_election.cc:304] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:16.113032 22420 raft_consensus.cc:2749] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:16.226564 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:16.385344 15425 catalog_manager.cc:5697] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 19 to 21, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 21 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:16.389607 15425 catalog_manager.cc:5697] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 16 to 17, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 17 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:16.391168 15425 catalog_manager.cc:5697] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:16.392462 15425 catalog_manager.cc:5697] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 19 to 21, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 21 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:16.393754 15425 catalog_manager.cc:5697] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 25 to 26, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 26 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:21:16.482905 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 49000.9us
I20260709 13:21:16.395105 15425 catalog_manager.cc:5697] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 17 to 18, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 18 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:16.494546 15425 catalog_manager.cc:5697] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:16.498103 15601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c0b73c09a0849518f4bc6e3886a348e"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:59198
I20260709 13:21:16.505076 15601 raft_consensus.cc:606] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:16.505476 15601 raft_consensus.cc:614] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:16.518246 15425 catalog_manager.cc:5697] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 13 to 14, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 14 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:16.530694 15425 catalog_manager.cc:5697] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 10 to 11, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 11 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:16.533048 15601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:59198
I20260709 13:21:16.533438 15425 catalog_manager.cc:5697] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: config changed from index 5 to 11, term changed from 12 to 14, VOTER d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) evicted. New cstate: current_term: 14 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:16.535187 15601 raft_consensus.cc:606] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
W20260709 13:21:16.536128 15420 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49132 (request call id 343) took 11815 ms (client timeout 15000 ms). Trace:
I20260709 13:21:16.539228 15425 catalog_manager.cc:5697] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 16 to 21, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 21 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:16.540947 15425 catalog_manager.cc:5697] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:21:16.542176 15425 catalog_manager.cc:5697] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 13 to 14, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 14 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } }
I20260709 13:21:16.550112 16235 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b8ed23a60014da39806e750cee98fa6"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:21:16.550786 16235 raft_consensus.cc:606] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:16.551095 16235 raft_consensus.cc:614] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:16.562207 15601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab68091b31d248a6ac99ca234a252f77"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:59198
I20260709 13:21:16.562930 15601 raft_consensus.cc:606] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 26 FOLLOWER]: Received request to transfer leadership to TS 3ef186ec12554a1d87814af0a0ace9a4
I20260709 13:21:16.563340 15601 raft_consensus.cc:614] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:16.572984 15601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:59198
W20260709 13:21:16.538516 15420 rpcz_store.cc:269] 0709 13:21:04.743484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:04.794869 (+ 51385us) service_pool.cc:224] Handling call
0709 13:21:16.533814 (+11738945us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:15.751474 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:15.752055 (+   581us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:15.752097 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:21:15.752117 (+    20us) tablet.cc:662] Decoding operations
0709 13:21:15.904516 (+152399us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:15.907017 (+  2501us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:15.910165 (+  3148us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:21:15.986159 (+ 75994us) tablet.cc:701] Row locks acquired
0709 13:21:15.986181 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:21:15.986322 (+   141us) write_op.cc:270] Start()
0709 13:21:15.986438 (+   116us) write_op.cc:276] Timestamp: P: 1783603275986274 usec, L: 0
0709 13:21:15.986462 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:21:15.987135 (+   673us) log.cc:844] Serialized 2815 byte log entry
0709 13:21:16.043070 (+ 55935us) op_driver.cc:464] REPLICATION: finished
0709 13:21:16.167196 (+124126us) write_op.cc:301] APPLY: starting
0709 13:21:16.167306 (+   110us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:16.202783 (+ 35477us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:16.202879 (+    96us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:16.292725 (+ 89846us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:16.301634 (+  8909us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=0,mrs_lookups=0
0709 13:21:16.301687 (+    53us) write_op.cc:312] APPLY: finished
0709 13:21:16.306030 (+  4343us) log.cc:844] Serialized 73 byte log entry
0709 13:21:16.306914 (+   884us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:16.307033 (+   119us) write_op.cc:454] Released schema lock
0709 13:21:16.307082 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:16.325043 (+ 17961us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:16.325073 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":12618,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":123872,"apply.run_cpu_time_us":10641,"apply.run_wall_time_us":157798,"cfile_cache_hit":30,"cfile_cache_hit_bytes":33443,"num_ops":7,"prepare.queue_time_us":185035,"prepare.run_cpu_time_us":5038,"prepare.run_wall_time_us":235945,"replication_time_us":56457,"thread_start_us":281877,"threads_started":2}]]}
I20260709 13:21:16.573702 15601 raft_consensus.cc:606] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:16.600144 15597 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d481434f60dc4276831163b535fb8159"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:59198
I20260709 13:21:16.600862 15597 raft_consensus.cc:606] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:16.606626 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:21:16.608742 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:21:16.616493 22464 raft_consensus.cc:993] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:21:16.617161 22464 raft_consensus.cc:1081] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Signalling peer 3ef186ec12554a1d87814af0a0ace9a4 to start an election
I20260709 13:21:16.621455 15670 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78"
dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
 from {username='slave'} at 127.0.0.1:49100
I20260709 13:21:16.622140 15670 raft_consensus.cc:493] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:16.622516 15670 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:16.631071 15670 raft_consensus.cc:515] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:16.640903 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:16.641525 15601 raft_consensus.cc:3055] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:16.641822 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9f476ac2ad44cd8124fde4f1aadc78" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:16.641852 15601 raft_consensus.cc:740] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:16.642576 16235 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:16.643038 15601 consensus_queue.cc:260] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:16.644169 15601 raft_consensus.cc:3060] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:16.652369 15670 leader_election.cc:290] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:16.656734 16235 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 16.
I20260709 13:21:16.658100 15642 leader_election.cc:304] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:16.658957 22468 raft_consensus.cc:2804] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:16.659475 22468 raft_consensus.cc:697] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:16.660341 22468 consensus_queue.cc:237] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:16.681155 15601 raft_consensus.cc:2468] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 16.
I20260709 13:21:16.804895 22235 raft_consensus.cc:993] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:21:16.805430 22235 raft_consensus.cc:1081] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Signalling peer d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:21:16.808775 15521 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d481434f60dc4276831163b535fb8159"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:57310
I20260709 13:21:16.809355 15521 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:16.809695 15521 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:16.817083 15521 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:16.835506 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:16.836089 15670 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:16.848197 15670 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 16.
I20260709 13:21:16.849489 15491 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:16.850411 22447 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:16.851114 22447 raft_consensus.cc:697] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:21:16.851982 22447 consensus_queue.cc:237] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:16.853987 15425 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49140 (request call id 366) took 1336 ms. Trace:
I20260709 13:21:16.862052 15521 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:16.854379 15425 rpcz_store.cc:276] 0709 13:21:15.517644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:15.517836 (+   192us) service_pool.cc:224] Handling call
0709 13:21:15.534928 (+ 17092us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b5400002490. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e7640 00007f5cea7beb76 00007f5cea8c8c3e 00007f5ce8d892a4 00007f5ce8d89232 00007f5ce8d89161 00007f5ce8d890dc 00007f5ce8d88392 00007f5ce2d4648c 00007f5ce2d458c6
0709 13:21:16.853883 (+1318955us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:16.626738 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:16.631032 (+  4294us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:16.631072 (+    40us) write_op.cc:435] Acquired schema lock
0709 13:21:16.631088 (+    16us) tablet.cc:662] Decoding operations
0709 13:21:16.638209 (+  7121us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:16.638294 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:16.638323 (+    29us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:21:16.639326 (+  1003us) tablet.cc:701] Row locks acquired
0709 13:21:16.639341 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:21:16.639485 (+   144us) write_op.cc:270] Start()
0709 13:21:16.639598 (+   113us) write_op.cc:276] Timestamp: P: 1783603276639436 usec, L: 0
0709 13:21:16.639620 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:21:16.640417 (+   797us) log.cc:844] Serialized 5101 byte log entry
0709 13:21:16.656924 (+ 16507us) op_driver.cc:464] REPLICATION: finished
0709 13:21:16.657651 (+   727us) write_op.cc:301] APPLY: starting
0709 13:21:16.657729 (+    78us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:16.673599 (+ 15870us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:16.690667 (+ 17068us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:16.735676 (+ 45009us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:16.735905 (+   229us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=0,mrs_lookups=0
0709 13:21:16.735949 (+    44us) write_op.cc:312] APPLY: finished
0709 13:21:16.736269 (+   320us) log.cc:844] Serialized 121 byte log entry
0709 13:21:16.736451 (+   182us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:16.736596 (+   145us) write_op.cc:454] Released schema lock
0709 13:21:16.736650 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:16.762029 (+ 25379us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:16.762061 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":64938,"reported_tablets":19,"spinlock_wait_cycles":29379456,"child_traces":[["op",{"apply.queue_time_us":503,"apply.run_cpu_time_us":16897,"apply.run_wall_time_us":79324,"cfile_cache_hit":52,"cfile_cache_hit_bytes":54444,"num_ops":13,"prepare.queue_time_us":78800,"prepare.run_cpu_time_us":4438,"prepare.run_wall_time_us":27542,"replication_time_us":17146,"thread_start_us":78419,"threads_started":1}]]}
I20260709 13:21:16.876799 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:16.877415 15597 raft_consensus.cc:3055] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:16.877703 15597 raft_consensus.cc:740] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:16.879501 15597 consensus_queue.cc:260] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:16.880991 15597 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:16.891845 15418 catalog_manager.cc:5697] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 16 to 18, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 18 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:16.905154 15597 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 16.
I20260709 13:21:16.901707 15418 catalog_manager.cc:5697] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 13 to 18, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 18 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:16.908061 15418 catalog_manager.cc:5697] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 8 to 12, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 12 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:16.914840 15418 catalog_manager.cc:5697] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 13 to 17, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 17 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:16.919791 15418 catalog_manager.cc:5697] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 14 to 20, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 20 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:16.926496 15418 catalog_manager.cc:5697] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:16.930711 15418 catalog_manager.cc:5697] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 18 to 20, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 20 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:16.951622 15514 tablet_service.cc:1558] Processing DeleteTablet for tablet 582343936346420b87ecf2eebd00dc25 with delete_type TABLET_DATA_TOMBSTONED (TS d3c4f5bd76e0445795bad5f6dd93ff69 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:40446
I20260709 13:21:16.958474 22476 tablet_replica.cc:333] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:16.959784 22476 raft_consensus.cc:2243] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:16.960850 22476 raft_consensus.cc:2272] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:16.964525 22476 ts_tablet_manager.cc:1916] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:21:16.985172 22476 ts_tablet_manager.cc:1929] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 12.10
I20260709 13:21:16.985777 22476 log.cc:1199] T 582343936346420b87ecf2eebd00dc25 P d3c4f5bd76e0445795bad5f6dd93ff69: Deleting WAL directory at /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603055563165-14429-0/minicluster-data/ts-0-root/wals/582343936346420b87ecf2eebd00dc25
I20260709 13:21:16.989179 15411 catalog_manager.cc:5028] TS d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): tablet 582343936346420b87ecf2eebd00dc25 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:21:17.003417 15420 catalog_manager.cc:5697] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 14 to 15, leader changed from 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 15 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:17.007454 15420 catalog_manager.cc:5697] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:17.021708 22464 raft_consensus.cc:993] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:21:17.029863 15418 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58802 (request call id 264) took 4607 ms. Trace:
I20260709 13:21:17.030347 22464 raft_consensus.cc:1081] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 LEADER]: Signalling peer d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:21:17.030993 15420 catalog_manager.cc:5697] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 20 to 21, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 21 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:17.043594 15521 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:57310
I20260709 13:21:17.044251 15521 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:17.038271 15418 rpcz_store.cc:276] 0709 13:21:12.422819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:12.423087 (+   268us) service_pool.cc:224] Handling call
0709 13:21:12.615637 (+192550us) spinlock_profiling.cc:243] Waited 9.92 ms on lock 0x7b5400002490. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e70b2 00007f5cea860a89 00007f5cea7f8da6 00007f5cea7ca088 00007f5cea8c837e 00007f5ce8d892a4 00007f5ce8d89232 00007f5ce8d89161 00007f5ce8d890dc
0709 13:21:17.029773 (+4414136us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:16.939876 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:16.940434 (+   558us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:16.940466 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:21:16.940482 (+    16us) tablet.cc:662] Decoding operations
0709 13:21:16.941248 (+   766us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:16.941316 (+    68us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:16.941344 (+    28us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:21:16.941884 (+   540us) tablet.cc:701] Row locks acquired
0709 13:21:16.941903 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:21:16.942080 (+   177us) write_op.cc:270] Start()
0709 13:21:16.942194 (+   114us) write_op.cc:276] Timestamp: P: 1783603276942039 usec, L: 0
0709 13:21:16.942223 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:21:16.942885 (+   662us) log.cc:844] Serialized 2814 byte log entry
0709 13:21:16.944876 (+  1991us) op_driver.cc:464] REPLICATION: finished
0709 13:21:16.945175 (+   299us) write_op.cc:301] APPLY: starting
0709 13:21:16.945254 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:16.948683 (+  3429us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:16.948786 (+   103us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:16.952778 (+  3992us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:16.952980 (+   202us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=0,mrs_lookups=0
0709 13:21:16.953029 (+    49us) write_op.cc:312] APPLY: finished
0709 13:21:16.953307 (+   278us) log.cc:844] Serialized 73 byte log entry
0709 13:21:16.953482 (+   175us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:16.953597 (+   115us) write_op.cc:454] Released schema lock
0709 13:21:16.953638 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:16.954013 (+   375us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:16.954041 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":32990,"reported_tablets":44,"spinlock_wait_cycles":22817024,"child_traces":[["op",{"apply.queue_time_us":95,"apply.run_cpu_time_us":8756,"apply.run_wall_time_us":8756,"cfile_cache_hit":28,"cfile_cache_hit_bytes":29316,"num_ops":7,"prepare.queue_time_us":778,"prepare.run_cpu_time_us":3281,"prepare.run_wall_time_us":3281,"replication_time_us":2493}]]}
I20260709 13:21:17.044602 15521 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:17.073223 15521 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:17.086807 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:17.087443 15601 raft_consensus.cc:3055] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:17.087746 15601 raft_consensus.cc:740] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:17.088574 15601 consensus_queue.cc:260] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:17.089655 15601 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:17.101091 15597 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:21:17.103338 22468 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:21:17.125044 15601 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 18.
I20260709 13:21:17.126379 15493 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:17.127291 22420 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:17.127817 22420 raft_consensus.cc:697] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:21:17.128697 22420 consensus_queue.cc:237] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:17.129315 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:17.129940 16235 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:17.146749 16235 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 18.
I20260709 13:21:17.148820 15521 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 18 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:17.152202 15420 catalog_manager.cc:5697] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 15 to 16, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 16 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:17.160010 16235 raft_consensus.cc:1275] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:21:17.162124 22468 consensus_queue.cc:1048] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
W20260709 13:21:17.205813 15422 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49132 (request call id 341) took 53321 ms (client timeout 15000 ms). Trace:
W20260709 13:21:17.211019 15422 rpcz_store.cc:269] 0709 13:20:23.891541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:23.923973 (+ 32432us) service_pool.cc:224] Handling call
0709 13:21:17.205587 (+53281614us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":318745,"reported_tablets":46}
I20260709 13:21:17.219252 15514 tablet_service.cc:1558] Processing DeleteTablet for tablet 582343936346420b87ecf2eebd00dc25 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 5 (current committed config index is 11)) from {username='slave'} at 127.0.0.1:40446
W20260709 13:21:17.220371 22477 log.cc:927] Time spent T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:21:17.230510 15411 catalog_manager.cc:5028] TS d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): tablet 582343936346420b87ecf2eebd00dc25 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:21:17.236662 15425 catalog_manager.cc:5697] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 15 to 16, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 16 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
W20260709 13:21:17.280606 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 86233.4us
I20260709 13:21:17.299276 15670 raft_consensus.cc:1275] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:21:17.298252 15422 catalog_manager.cc:5697] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 17 to 18, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 18 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:17.306535 22460 consensus_queue.cc:1048] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:17.350391 15601 raft_consensus.cc:1275] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:21:17.356622 15423 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49144 (request call id 356) took 5010 ms. Trace:
I20260709 13:21:17.357158 15423 rpcz_store.cc:276] 0709 13:21:12.346097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:12.350080 (+  3983us) service_pool.cc:224] Handling call
0709 13:21:17.356445 (+5006365us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2108,"reported_tablets":46}
I20260709 13:21:17.369954 22420 consensus_queue.cc:1048] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:21:17.460825 15426 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49132 (request call id 342) took 32712 ms (client timeout 15000 ms). Trace:
W20260709 13:21:17.461299 15426 rpcz_store.cc:269] 0709 13:20:44.748622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:44.833735 (+ 85113us) service_pool.cc:224] Handling call
0709 13:21:17.460620 (+32626885us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5015,"reported_tablets":46}
I20260709 13:21:17.471349 15514 tablet_service.cc:1558] Processing DeleteTablet for tablet 582343936346420b87ecf2eebd00dc25 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 5 (current committed config index is 11)) from {username='slave'} at 127.0.0.1:40446
I20260709 13:21:17.475063 15411 catalog_manager.cc:5028] TS d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): tablet 582343936346420b87ecf2eebd00dc25 (table test-workload [id=8efcedb1864d41909abf9d2535dfa264]) successfully deleted
I20260709 13:21:17.520480 15601 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:21:17.522236 22447 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:21:17.542686 15419 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49144 (request call id 355) took 21447 ms (client timeout 15000 ms). Trace:
W20260709 13:21:17.543215 15419 rpcz_store.cc:269] 0709 13:20:56.095184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:56.142256 (+ 47072us) service_pool.cc:224] Handling call
0709 13:21:17.542535 (+21400279us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7296,"reported_tablets":46}
I20260709 13:21:17.586913 16237 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:21:17.599893 22447 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:17.609889 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:21:18.421293 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 48878.4us
I20260709 13:21:18.848325 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:18.969592 15601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6139553a47b64d798a29413921f18396"
dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:59198
I20260709 13:21:18.970252 15601 raft_consensus.cc:606] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:18.990912 16237 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:21:18.991554 16237 raft_consensus.cc:606] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:19.002792 16237 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e"
dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
mode: GRACEFUL
new_leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:52820
I20260709 13:21:19.007046 16237 raft_consensus.cc:606] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Received request to transfer leadership to TS d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:19.008637 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:21:19.010442 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:21:19.321362 22490 raft_consensus.cc:993] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9: : Instructing follower d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:21:19.322026 22493 raft_consensus.cc:1081] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Signalling peer d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:21:19.332016 15528 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:56840
I20260709 13:21:19.336208 15528 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:19.336710 15528 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:21:19.349609 22464 raft_consensus.cc:993] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f: : Instructing follower d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
W20260709 13:21:19.358544 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 91984.3us
I20260709 13:21:19.366717 22461 raft_consensus.cc:1081] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 LEADER]: Signalling peer d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:21:19.376226 15521 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6139553a47b64d798a29413921f18396"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:57310
I20260709 13:21:19.376806 15521 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:19.377126 15521 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:19.382967 15528 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:19.389854 15521 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:19.395741 15528 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:19.434546 15521 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:19.430881 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:19.436055 15597 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:21:19.450699 15597 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:21:19.489440 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:19.490191 16237 raft_consensus.cc:3055] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:21:19.490541 16237 raft_consensus.cc:740] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:19.491429 16237 consensus_queue.cc:260] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:19.492887 16237 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:21:19.516310 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 84470.4us
I20260709 13:21:19.520687 15597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: true dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:19.521311 15597 raft_consensus.cc:3055] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:19.521642 15597 raft_consensus.cc:740] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:19.522413 15597 consensus_queue.cc:260] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:19.523702 15597 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:19.526019 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:19.526633 16235 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:19.536334 15597 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 19.
I20260709 13:21:19.544008 22490 raft_consensus.cc:993] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: : Instructing follower d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:21:19.544570 22490 raft_consensus.cc:1081] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Signalling peer d3c4f5bd76e0445795bad5f6dd93ff69 to start an election
I20260709 13:21:19.546483 15528 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e"
dest_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69"
 from {username='slave'} at 127.0.0.1:56840
I20260709 13:21:19.547053 15528 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:19.547384 15528 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:19.558712 15528 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:19.564049 16235 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 19.
I20260709 13:21:19.576270 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:19.576889 15668 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:19.597541 15668 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:21:19.599012 15491 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:19.599896 22492 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:19.600409 22492 raft_consensus.cc:697] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:21:19.601281 22492 consensus_queue.cc:237] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:19.611820 16237 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 9.
I20260709 13:21:19.612850 15528 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 13 election: Requested vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:19.648687 15493 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:19.656944 22460 raft_consensus.cc:2804] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:21:19.657522 22460 raft_consensus.cc:697] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:21:19.658442 22460 consensus_queue.cc:237] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:19.670987 15493 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 
I20260709 13:21:19.680006 22460 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:19.680992 22460 raft_consensus.cc:697] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 LEADER]: Becoming Leader. State: Replica: d3c4f5bd76e0445795bad5f6dd93ff69, State: Running, Role: LEADER
I20260709 13:21:19.686833 22460 consensus_queue.cc:237] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:19.684458 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:19.688879 16239 raft_consensus.cc:3055] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:19.689242 16239 raft_consensus.cc:740] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:19.690438 16239 consensus_queue.cc:260] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:19.691672 16239 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:19.698967 16239 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c4f5bd76e0445795bad5f6dd93ff69 in term 13.
I20260709 13:21:19.710573 15419 catalog_manager.cc:5697] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 12 to 13, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 13 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:19.787312 15493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 90672.8us
W20260709 13:21:19.815790 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 75304.8us
I20260709 13:21:19.883957 15419 catalog_manager.cc:5697] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 8 to 9, leader changed from dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 9 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:19.885614 15419 catalog_manager.cc:5697] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 reported cstate change: term changed from 18 to 19, leader changed from 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66) to d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65). New cstate: current_term: 19 leader_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:20.011415 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:20.075081 16239 raft_consensus.cc:1275] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:21:20.083536 22501 consensus_queue.cc:1048] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:20.135087 15601 raft_consensus.cc:1275] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:21:20.148036 15668 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:21:20.154050 22460 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:20.161249 22460 consensus_queue.cc:1048] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:20.169447 16237 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:21:20.185519 15601 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:21:20.226749 16237 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Refusing update from remote peer d3c4f5bd76e0445795bad5f6dd93ff69: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:21:20.195246 22460 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.019s
I20260709 13:21:20.248577 22492 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 13:21:20.249636 14429 tablet_server.cc:179] TabletServer@127.14.23.65:0 shutting down...
I20260709 13:21:20.238993 22511 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
W20260709 13:21:20.299880 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 72254.2us
W20260709 13:21:20.322182 15567 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 6 similar messages]
W20260709 13:21:20.374470 15567 consensus_peers.cc:597] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.410511 15567 consensus_peers.cc:597] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.411247 15567 consensus_peers.cc:597] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.411778 15567 consensus_peers.cc:597] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.421253 15567 consensus_peers.cc:597] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.427649 15567 consensus_peers.cc:597] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.445170 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 58314.9us
W20260709 13:21:20.500725 16204 consensus_peers.cc:597] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.503720 16204 consensus_peers.cc:597] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.504721 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 47069.5us
W20260709 13:21:20.515779 16204 consensus_peers.cc:597] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.516661 16204 consensus_peers.cc:597] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.538492 16204 consensus_peers.cc:597] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.576913 15567 consensus_peers.cc:597] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.605597 15567 consensus_peers.cc:597] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.610682 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 79483.8us
W20260709 13:21:20.628495 15567 consensus_peers.cc:597] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.659772 15567 consensus_peers.cc:597] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.680220 15641 consensus_peers.cc:597] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.681007 15641 consensus_peers.cc:597] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.681671 15641 consensus_peers.cc:597] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.682318 15641 consensus_peers.cc:597] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.687295 16204 consensus_peers.cc:597] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.689253 15641 consensus_peers.cc:597] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.691627 15641 consensus_peers.cc:597] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.693367 15641 consensus_peers.cc:597] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.700067 15641 consensus_peers.cc:597] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.716131 16204 consensus_peers.cc:597] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.716784 16204 consensus_peers.cc:597] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.702245 15641 consensus_peers.cc:597] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.731745 16204 consensus_peers.cc:597] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.736053 16204 consensus_peers.cc:597] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.759194 15567 consensus_peers.cc:597] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.786576 15641 consensus_peers.cc:597] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:20.814877 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:20.818751 14429 tablet_replica.cc:333] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:20.819660 14429 raft_consensus.cc:2243] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:20.823132 14429 raft_consensus.cc:2272] T ecb5853a05664246a26d3fc7ecf92777 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.826013 14429 tablet_replica.cc:333] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:20.826906 14429 raft_consensus.cc:2243] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:20.827572 14429 raft_consensus.cc:2272] T 0f8365fc4e2f4decb3d554a43e7d4d52 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.841271 14429 tablet_replica.cc:333] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:20.842255 14429 raft_consensus.cc:2243] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:20.843945 14429 raft_consensus.cc:2272] T 7dbabc834ad54c16ae25f79064932f42 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.856135 14429 tablet_replica.cc:333] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:20.857089 14429 raft_consensus.cc:2243] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:20.857820 14429 raft_consensus.cc:2272] T 7562b5f342134e39a5e6934d5ca16e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.861439 14429 tablet_replica.cc:333] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:20.862241 14429 raft_consensus.cc:2243] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:20.862854 14429 raft_consensus.cc:2272] T e67cef93bbdb4ff88a2ee2ac558ba090 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.869611 14429 tablet_replica.cc:333] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:20.870437 14429 raft_consensus.cc:2243] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:20.872699 14429 raft_consensus.cc:2272] T 7b8ed23a60014da39806e750cee98fa6 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.875662 14429 tablet_replica.cc:333] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:20.876521 14429 raft_consensus.cc:2243] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:21:20.880327 14429 raft_consensus.cc:2272] T 974a060267d24f00888c0942dfbca77e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.927438 14429 tablet_replica.cc:333] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:20.928334 14429 raft_consensus.cc:2243] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:20.929257 14429 raft_consensus.cc:2272] T d6670c20b1954b36a4920b44d1a8209d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.933570 14429 tablet_replica.cc:333] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:20.934422 14429 raft_consensus.cc:2243] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:20.935020 14429 raft_consensus.cc:2272] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.937788 14429 tablet_replica.cc:333] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:20.938606 14429 raft_consensus.cc:2243] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:20.939565 14429 raft_consensus.cc:2272] T 88a66a9fb8df432fb0d1d7ba3756235e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.943002 14429 tablet_replica.cc:333] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:20.943842 14429 raft_consensus.cc:2243] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:21:20.944814 14429 raft_consensus.cc:2272] T b99eaeae24b44aa78cfa8c90caca288e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.949177 14429 tablet_replica.cc:333] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:20.970852 14429 raft_consensus.cc:2243] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:20.971655 14429 raft_consensus.cc:2272] T 2c0b73c09a0849518f4bc6e3886a348e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.974414 14429 tablet_replica.cc:333] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:20.975188 14429 raft_consensus.cc:2243] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:20.975900 14429 raft_consensus.cc:2272] T 50726e0d396948b79be43aece292acb5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.978591 14429 tablet_replica.cc:333] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:20.979348 14429 raft_consensus.cc:2243] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:20.979974 14429 raft_consensus.cc:2272] T 4c48036212bb4277acf36aac79d8b35e P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.982812 14429 tablet_replica.cc:333] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:20.983557 14429 raft_consensus.cc:2243] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:21:20.984635 14429 raft_consensus.cc:2272] T 0af5649725244ae599dfc6ed7aeac5f1 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.996248 14429 tablet_replica.cc:333] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:20.998026 14429 raft_consensus.cc:2243] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:20.998875 14429 raft_consensus.cc:2272] T db313a4cd18a47a2b72587d04ed6e747 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.001643 14429 tablet_replica.cc:333] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.002480 14429 raft_consensus.cc:2243] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:21:21.003576 14429 raft_consensus.cc:2272] T 061ed80a3fca4096b7c13dc0e2256ef0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.092051 14429 tablet_replica.cc:333] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.092900 14429 raft_consensus.cc:2243] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:21:21.093825 14429 raft_consensus.cc:2272] T 8a81ba00c9bd4a45ae8d2379570072fe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.096698 14429 tablet_replica.cc:333] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.097437 14429 raft_consensus.cc:2243] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:21:21.149822 14429 raft_consensus.cc:2272] T 01ad9d7b8dba45be8fdf625f27e8d273 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.152554 14429 tablet_replica.cc:333] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.153405 14429 raft_consensus.cc:2243] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:21.153993 14429 raft_consensus.cc:2272] T 340e065cf2744fde9d65230a8ea91e14 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.156635 14429 tablet_replica.cc:333] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.157358 14429 raft_consensus.cc:2243] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:21.198489 22524 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:21.199080 22524 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:21:21.201339 22524 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:21.207118 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:21.211365 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b56c6369484d92b328d276d4fca9e3" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:21.213122 15493 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:21.213882 22524 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:21.157903 14429 raft_consensus.cc:2272] T 7a5315557f994ec689bb9a79498bb2d5 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.224311 14429 tablet_replica.cc:333] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.225131 14429 raft_consensus.cc:2243] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:21:21.226333 14429 raft_consensus.cc:2272] T 98787e13cd7a49528aeeb59c72559e46 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.228994 14429 tablet_replica.cc:333] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.238619 14429 raft_consensus.cc:2243] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:21.239347 14429 raft_consensus.cc:2272] T 5243f722506441269fac66aa989ce36d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.244184 14429 tablet_replica.cc:333] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.250169 14429 raft_consensus.cc:2243] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:21.250871 14429 raft_consensus.cc:2272] T 1fa44270fe124591b1e8c82c82b89efe P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.253654 14429 tablet_replica.cc:333] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.254442 14429 raft_consensus.cc:2243] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:21.255108 14429 raft_consensus.cc:2272] T 83022a40fade44a2988f3de3b6b36aec P d3c4f5bd76e0445795bad5f6dd93ff69 [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.257802 14429 tablet_replica.cc:333] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.258580 14429 raft_consensus.cc:2243] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:21:21.259617 14429 raft_consensus.cc:2272] T 3088d5241ec64ddc919925d5dccfeda0 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.287405 14429 tablet_replica.cc:333] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.288156 14429 raft_consensus.cc:2243] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:21.288810 14429 raft_consensus.cc:2272] T c1e80bd01b0c4006852da32ca0eeb0ae P d3c4f5bd76e0445795bad5f6dd93ff69 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.291476 14429 tablet_replica.cc:333] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.292219 14429 raft_consensus.cc:2243] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:21:21.293145 14429 raft_consensus.cc:2272] T d481434f60dc4276831163b535fb8159 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.298358 14429 tablet_replica.cc:333] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.299079 14429 raft_consensus.cc:2243] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:21:21.300127 14429 raft_consensus.cc:2272] T 6139553a47b64d798a29413921f18396 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.306108 14429 tablet_replica.cc:333] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.306922 14429 raft_consensus.cc:2243] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:21.307646 14429 raft_consensus.cc:2272] T 4344539a12c94917991c2f39d3ff5858 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.310550 14429 tablet_replica.cc:333] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.318912 22460 raft_consensus.cc:493] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:21.319526 22460 raft_consensus.cc:515] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:21.321708 22460 leader_election.cc:290] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:21.322825 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:21.322856 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4fd164d74054954b750fe9958e66778" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:21.328118 14429 raft_consensus.cc:2243] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:21:21.328275 15491 leader_election.cc:304] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:21.329001 22460 raft_consensus.cc:2749] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:21.334029 22460 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:21.334600 22460 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:21.337019 22460 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:21.338646 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:21.339054 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd36c2cfd99c45cea9e48096d54bed1a" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:21.333508 14429 raft_consensus.cc:2272] T b2c57bed49da41d795c8b21ed4461f1f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.347303 14429 tablet_replica.cc:333] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.348153 14429 raft_consensus.cc:2243] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:21:21.349323 14429 raft_consensus.cc:2272] T 40f71e7bfe1a4575846bcddbb0b9fd55 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.350164 22460 raft_consensus.cc:493] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader dd1a14c3173d49dc9b55bb116e3783c9)
I20260709 13:21:21.350666 22460 raft_consensus.cc:515] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:21.352807 22460 leader_election.cc:290] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:21.354245 15493 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:21.355033 22460 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:21.360448 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:21.360970 14429 tablet_replica.cc:333] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.361773 14429 raft_consensus.cc:2243] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:21.362593 14429 raft_consensus.cc:2272] T a3d461474623405ab8ee3ca7fdff1722 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.365320 14429 tablet_replica.cc:333] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.366110 14429 raft_consensus.cc:2243] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:21.366734 14429 raft_consensus.cc:2272] T bd36c2cfd99c45cea9e48096d54bed1a P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.369168 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38190179f6494b5591027e1c41f83176" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:21.378734 22460 raft_consensus.cc:493] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:21.379333 22460 raft_consensus.cc:515] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:21.381498 22460 leader_election.cc:290] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:21.382692 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:21.369443 14429 tablet_replica.cc:333] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.390579 14429 raft_consensus.cc:2243] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:21.394739 14429 raft_consensus.cc:2272] T 68dd26481a1b4442b702cb5976b1fb41 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.397678 14429 tablet_replica.cc:333] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.398447 14429 raft_consensus.cc:2243] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:21.401999 14429 raft_consensus.cc:2272] T 4da1202d21704a61a455cc91fa7c4022 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.410244 22460 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:21.410854 22460 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:21.412624 15491 leader_election.cc:304] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:21.412918 22460 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:21.413452 22524 raft_consensus.cc:2749] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:21.415396 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:21.418716 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271b5d375f0145b78b28a0c13328ce7f" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:21.419692 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db63c1e295d42d9938e0ef858365436" candidate_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:21.424044 15493 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:21.424830 22524 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:21.428534 15491 leader_election.cc:304] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3c4f5bd76e0445795bad5f6dd93ff69; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:21.431529 22524 raft_consensus.cc:2749] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:21.434276 14429 tablet_replica.cc:333] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.435173 14429 raft_consensus.cc:2243] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:21:21.448798 14429 raft_consensus.cc:2272] T ced58b89726545d3af99de36a1aa1f30 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.451788 14429 tablet_replica.cc:333] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.452517 14429 raft_consensus.cc:2243] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:21.456808 14429 raft_consensus.cc:2272] T 6db63c1e295d42d9938e0ef858365436 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.462616 14429 tablet_replica.cc:333] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.465056 14429 raft_consensus.cc:2243] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:21.465830 14429 raft_consensus.cc:2272] T 38190179f6494b5591027e1c41f83176 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.498086 14429 tablet_replica.cc:333] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.498903 14429 raft_consensus.cc:2243] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:21.499773 14429 raft_consensus.cc:2272] T d4fd164d74054954b750fe9958e66778 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.502347 14429 tablet_replica.cc:333] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.503023 14429 raft_consensus.cc:2243] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:21.508450 14429 raft_consensus.cc:2272] T 2a5f6755312b4dc09781ecf8a9468c17 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.525061 14429 tablet_replica.cc:333] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.525833 14429 raft_consensus.cc:2243] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:21.534324 14429 raft_consensus.cc:2272] T 12b56c6369484d92b328d276d4fca9e3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.537320 14429 tablet_replica.cc:333] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.541231 14429 raft_consensus.cc:2243] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 LEADER]: Raft consensus shutting down.
I20260709 13:21:21.542352 14429 raft_consensus.cc:2272] T 8153c7c094654fb492f5c47863ad170d P d3c4f5bd76e0445795bad5f6dd93ff69 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.549638 14429 tablet_replica.cc:333] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.550474 14429 raft_consensus.cc:2243] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:21.551110 14429 raft_consensus.cc:2272] T 271b5d375f0145b78b28a0c13328ce7f P d3c4f5bd76e0445795bad5f6dd93ff69 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.553684 14429 tablet_replica.cc:333] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69: stopping tablet replica
I20260709 13:21:21.566480 14429 raft_consensus.cc:2243] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:21.567248 14429 raft_consensus.cc:2272] T 2afa8e112a744f218227526d861b98c3 P d3c4f5bd76e0445795bad5f6dd93ff69 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:22.067273 22534 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.067845 22534 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:22.085609 16204 leader_election.cc:336] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.087684 22534 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:22.092941 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:22.094583 16205 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:22.095834 22495 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:22.122711 22504 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.123289 22504 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.125459 22504 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:22.127689 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:21:22.131181 15567 leader_election.cc:336] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.131735 15567 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:22.132748 22529 raft_consensus.cc:2749] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:22.210246 22495 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.210813 22495 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.211908 22545 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.212474 22545 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.214640 22545 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:22.214944 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
W20260709 13:21:22.222095 15641 leader_election.cc:336] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.222993 22495 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:22.223814 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:22.226301 15642 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:22.228299 22520 raft_consensus.cc:2749] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:21:22.233057 16204 leader_election.cc:336] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.233693 16204 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:22.239207 22493 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:22.273301 22495 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.277091 22495 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.282029 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:22.287776 22495 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:22.291806 22504 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.300946 22504 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:22.303288 22504 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:21:22.310914 16204 leader_election.cc:336] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.323875 16204 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:21:22.318514 15567 leader_election.cc:336] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.313653 22495 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.326838 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:22.335712 22495 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:22.338220 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 44486.3us
I20260709 13:21:22.348356 22534 raft_consensus.cc:2749] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:22.338822 15566 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:22.352028 22504 raft_consensus.cc:2749] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:22.349660 22493 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.356212 22493 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.358351 22493 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:22.360256 22534 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.360816 22534 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.363684 22534 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:22.364284 22493 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.367199 22493 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.369421 22493 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:22.370543 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:22.374668 15668 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 13.
I20260709 13:21:22.376181 16203 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:22.377121 22493 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:22.377686 22493 raft_consensus.cc:493] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.378230 22493 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:22.377225 22495 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:22.387847 22493 raft_consensus.cc:515] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.391551 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99eaeae24b44aa78cfa8c90caca288e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:22.392220 15668 raft_consensus.cc:3060] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:22.392604 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:22.393504 22520 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.394045 22520 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:22.394783 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:22.398152 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:22.398764 15595 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 18.
I20260709 13:21:22.398773 22493 leader_election.cc:290] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:22.400940 16205 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
W20260709 13:21:22.401687 15641 leader_election.cc:336] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.401787 22495 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:22.402526 22495 raft_consensus.cc:493] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.403774 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:22.412572 15642 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:22.413492 22545 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:22.404280 22520 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:22.414361 22547 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.415021 22495 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:22.414909 22547 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.419992 22520 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.420872 22520 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:22.422379 15567 leader_election.cc:336] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.423875 22547 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:22.428135 15641 leader_election.cc:336] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.429682 22520 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:22.404920 15668 raft_consensus.cc:2468] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:21:22.435878 22495 raft_consensus.cc:515] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.440274 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:22.460433 16235 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:21:22.449296 16203 leader_election.cc:304] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:22.461808 22548 raft_consensus.cc:2804] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:22.462352 22548 raft_consensus.cc:697] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:22.463256 22548 consensus_queue.cc:237] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.453907 22549 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.454334 22520 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.459838 22504 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.465085 22549 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.465138 22520 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:22.465720 22504 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.467358 22520 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:22.467806 22504 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:22.468544 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:22.469194 15670 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 20.
I20260709 13:21:22.465353 15568 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:22.471938 16203 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:22.472887 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:22.473030 22550 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:22.473564 22550 raft_consensus.cc:493] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
W20260709 13:21:22.473681 15641 leader_election.cc:336] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.474016 22547 raft_consensus.cc:2804] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:22.474427 22547 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.473994 22550 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:21:22.477260 15567 leader_election.cc:336] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.478402 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:22.479039 22547 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:22.478421 15642 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:22.480397 22520 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:22.482051 22549 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:22.479050 16235 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 11.
I20260709 13:21:22.492571 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:22.489295 15568 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:22.488706 22550 raft_consensus.cc:515] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.503261 22504 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:22.504546 22504 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.505018 22504 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:22.494408 22547 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.510445 16238 raft_consensus.cc:2393] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 21: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
W20260709 13:21:22.516086 15567 leader_election.cc:336] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.518407 22547 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:22.520628 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f71e7bfe1a4575846bcddbb0b9fd55" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:22.521304 15668 raft_consensus.cc:3060] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:22.522675 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974a060267d24f00888c0942dfbca77e" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:22.523258 15595 raft_consensus.cc:3060] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:22.531477 15668 raft_consensus.cc:2468] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 21.
I20260709 13:21:22.532891 16203 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:22.534724 22534 raft_consensus.cc:2804] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:22.535252 22534 raft_consensus.cc:697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:22.536108 22534 consensus_queue.cc:237] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.538908 22520 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.539469 22520 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.541678 22520 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:22.547482 15641 leader_election.cc:336] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.521034 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061ed80a3fca4096b7c13dc0e2256ef0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:22.550316 16238 raft_consensus.cc:3060] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:22.553428 22520 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.546437 15595 raft_consensus.cc:2468] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 19.
I20260709 13:21:22.554067 22520 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:22.555279 16205 leader_election.cc:304] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:22.556340 22549 raft_consensus.cc:2804] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:22.557140 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:22.557512 22549 raft_consensus.cc:697] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 LEADER]: Becoming Leader. State: Replica: dd1a14c3173d49dc9b55bb116e3783c9, State: Running, Role: LEADER
I20260709 13:21:22.557793 15601 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 16.
I20260709 13:21:22.559232 15642 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:21:22.560056 22545 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:22.559687 22549 consensus_queue.cc:237] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.560448 22545 raft_consensus.cc:493] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.561633 22545 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:21:22.564399 15641 leader_election.cc:336] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.560904 22520 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:21:22.568701 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 50699.7us
I20260709 13:21:22.578722 16238 raft_consensus.cc:2468] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
I20260709 13:21:22.576593 22550 leader_election.cc:290] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:22.663991 22529 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.578547 22547 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.672224 15568 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:22.569060 22495 leader_election.cc:290] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:21:22.672256 16204 leader_election.cc:336] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
W20260709 13:21:22.709291 16204 leader_election.cc:336] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.714192 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:22.715003 16238 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 14.
W20260709 13:21:22.716142 16204 leader_election.cc:336] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.731168 15418 catalog_manager.cc:5697] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 13 to 14, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 14 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:22.703758 22545 raft_consensus.cc:515] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:22.736213 22545 leader_election.cc:290] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:21:22.742549 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 70420.9us
I20260709 13:21:22.750770 22550 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.751426 22550 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.754256 22558 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.709064 22529 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.754841 22558 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.757030 22529 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:22.758143 22558 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:22.717697 22547 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:22.762683 22547 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:22.763207 22547 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.763644 22547 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.705361 22504 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.767889 22504 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:22.753799 22550 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:22.762604 22560 raft_consensus.cc:2804] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:22.770144 22560 raft_consensus.cc:697] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:22.771914 22560 consensus_queue.cc:237] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.765669 22547 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:22.787282 16204 leader_election.cc:336] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.787801 16204 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:21:22.794508 14429 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 15549)
I20260709 13:21:22.788898 22547 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
W20260709 13:21:22.811147 16204 leader_election.cc:336] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.811573 22547 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:21:22.811990 16204 leader_election.cc:336] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.812065 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:22.778005 22555 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.819590 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:22.820238 16235 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 19.
W20260709 13:21:22.812922 16204 leader_election.cc:336] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
W20260709 13:21:22.827414 16204 leader_election.cc:336] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.827966 16204 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:21:22.813647 15641 leader_election.cc:336] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
W20260709 13:21:22.795065 15567 leader_election.cc:336] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.819212 22547 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:22.812858 16238 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 21.
I20260709 13:21:22.834347 16235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af5649725244ae599dfc6ed7aeac5f1" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:21:22.832182 15567 leader_election.cc:336] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
W20260709 13:21:22.840006 15567 leader_election.cc:336] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.843161 15568 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:22.846365 22566 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:22.846817 22566 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.847157 22566 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:22.828125 22555 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.849895 22555 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:22.838404 22548 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:22.852265 16235 raft_consensus.cc:3060] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:22.854027 22565 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:22.841711 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 55732.7us
W20260709 13:21:22.857874 16204 leader_election.cc:336] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
W20260709 13:21:22.859054 16204 leader_election.cc:336] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.859162 16235 raft_consensus.cc:2468] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 12.
W20260709 13:21:22.860152 15567 leader_election.cc:336] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.861057 15568 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:22.862571 22546 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:22.862975 22546 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.863328 22546 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:21:22.859663 16204 consensus_peers.cc:597] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:22.866844 16204 consensus_peers.cc:597] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:22.868158 15641 consensus_peers.cc:597] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:22.870944 15642 leader_election.cc:304] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:22.875340 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:22.876075 15668 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd1a14c3173d49dc9b55bb116e3783c9 in term 14.
I20260709 13:21:22.884312 15642 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:22.880885 15568 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:22.894939 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:22.895682 15668 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:21:22.897564 15601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" is_pre_election: true
I20260709 13:21:22.897614 16203 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:22.893589 22562 raft_consensus.cc:2749] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:22.899405 22504 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.906278 22504 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.908553 22504 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:22.899991 15595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d481434f60dc4276831163b535fb8159" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5498ef3e89a941e39c25f68cf64e6b0f"
I20260709 13:21:22.909271 15595 raft_consensus.cc:3060] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:22.896651 22529 raft_consensus.cc:2804] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:22.901350 22520 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:22.902701 16237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:22.917367 16237 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 20.
W20260709 13:21:22.904543 15641 leader_election.cc:336] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.912770 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:22.919003 16205 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:22.920192 22534 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:22.920495 22493 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:21:22.921665 15595 raft_consensus.cc:2468] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 17.
I20260709 13:21:22.920746 22520 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.922240 22520 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:22.923578 15670 raft_consensus.cc:1275] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:21:22.919015 16238 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 18.
I20260709 13:21:22.920611 22534 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.930809 22534 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:22.930809 22559 consensus_queue.cc:1048] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
W20260709 13:21:22.938946 16204 consensus_peers.cc:597] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:22.939190 22546 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.937655 22520 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.941450 22546 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 22 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:22.901439 22566 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.946215 15423 catalog_manager.cc:5697] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 11 to 12, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 12 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:22.948154 15642 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:22.946645 22529 raft_consensus.cc:697] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:22.966645 22529 consensus_queue.cc:237] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:22.970572 15567 consensus_peers.cc:597] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:22.971153 15567 consensus_peers.cc:597] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:22.971550 15567 leader_election.cc:336] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.974452 15668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
I20260709 13:21:22.976895 22566 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:22.973412 22562 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:22.980604 22562 raft_consensus.cc:493] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:22.980993 22562 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:21:22.983340 15641 leader_election.cc:336] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:22.975220 15668 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 14.
I20260709 13:21:22.990345 22534 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:22.995239 22520 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:22.997710 15566 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:22.998963 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3088d5241ec64ddc919925d5dccfeda0" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:22.999563 16238 raft_consensus.cc:3060] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:23.007035 22566 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:23.007354 15566 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:23.008494 16239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6139553a47b64d798a29413921f18396" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:23.009084 16239 raft_consensus.cc:3060] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:23.009407 22566 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:23.009572 22534 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:23.009859 22566 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:23.010078 15674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:23.011067 15674 raft_consensus.cc:2393] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate dd1a14c3173d49dc9b55bb116e3783c9 in current term 15: Already voted for candidate 3ef186ec12554a1d87814af0a0ace9a4 in this term.
I20260709 13:21:23.021039 16239 raft_consensus.cc:2468] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 20.
I20260709 13:21:23.009210 22574 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:23.026746 22574 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:23.027156 22574 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:23.028647 22561 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:23.029224 22561 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:23.031934 15641 consensus_peers.cc:597] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:23.034569 15641 consensus_peers.cc:597] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:23.041647 16238 raft_consensus.cc:2468] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 22.
I20260709 13:21:23.042567 15642 leader_election.cc:304] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:21:23.045883 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 48422.4us
W20260709 13:21:23.046510 16204 consensus_peers.cc:597] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:23.050947 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:23.051700 16238 raft_consensus.cc:2393] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ef186ec12554a1d87814af0a0ace9a4 in current term 15: Already voted for candidate dd1a14c3173d49dc9b55bb116e3783c9 in this term.
I20260709 13:21:23.052654 15568 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:23.058974 22504 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:23.059433 22504 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:23.059801 22504 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:23.063268 22561 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
W20260709 13:21:23.066501 16204 consensus_peers.cc:597] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:23.067134 16204 consensus_peers.cc:597] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:23.068568 15568 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:23.069017 15642 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: d3c4f5bd76e0445795bad5f6dd93ff69, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:23.074322 22557 raft_consensus.cc:2804] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:23.075022 22557 raft_consensus.cc:697] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:23.075973 22557 consensus_queue.cc:237] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:23.077203 15567 consensus_peers.cc:597] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:23.074525 22563 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:23.050122 22520 raft_consensus.cc:2804] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:21:23.090246 15567 leader_election.cc:336] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
W20260709 13:21:23.091287 15567 consensus_peers.cc:597] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:23.091701 15677 raft_consensus.cc:1275] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 13:21:23.091771 15567 leader_election.cc:336] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:23.097994 15568 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:23.103021 22493 consensus_queue.cc:1048] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:21:23.103350 22546 raft_consensus.cc:2804] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:21:23.104064 22546 raft_consensus.cc:697] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 22 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:23.104929 22546 consensus_queue.cc:237] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:23.108464 16204 leader_election.cc:336] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:23.113306 16204 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:21:23.114542 15567 consensus_peers.cc:597] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:23.115077 15567 leader_election.cc:336] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
W20260709 13:21:23.116099 15567 consensus_peers.cc:597] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:23.116567 15567 consensus_peers.cc:597] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:23.108596 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:23.095829 22520 raft_consensus.cc:697] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
W20260709 13:21:23.117092 15641 consensus_peers.cc:597] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:23.120731 16238 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 9.
I20260709 13:21:23.118588 22504 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:23.121815 15641 consensus_peers.cc:597] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:23.123075 15568 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 
I20260709 13:21:23.125398 22520 consensus_queue.cc:237] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
W20260709 13:21:23.126597 15567 leader_election.cc:336] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
W20260709 13:21:23.128258 15567 consensus_peers.cc:597] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:23.128654 15567 consensus_peers.cc:597] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:23.131774 15641 consensus_peers.cc:597] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:23.121459 22493 raft_consensus.cc:2749] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:21:23.134348 16204 consensus_peers.cc:597] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:23.124629 22560 raft_consensus.cc:2804] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:21:23.142856 22562 raft_consensus.cc:515] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:23.145112 22504 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:23.147607 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c57bed49da41d795c8b21ed4461f1f" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:23.148298 16238 raft_consensus.cc:3060] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:23.150462 15567 leader_election.cc:336] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:23.151190 22574 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:23.153440 22574 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:23.140144 22560 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader d3c4f5bd76e0445795bad5f6dd93ff69)
I20260709 13:21:23.154572 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98787e13cd7a49528aeeb59c72559e46" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:23.155283 15677 raft_consensus.cc:3060] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:23.156972 15641 leader_election.cc:336] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:23.157995 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01ad9d7b8dba45be8fdf625f27e8d273" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:23.158569 16231 raft_consensus.cc:3060] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:21:23.159094 15567 leader_election.cc:336] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:23.155058 22560 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:21:23.163807 15677 raft_consensus.cc:2468] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:21:23.166002 15566 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:23.167021 22566 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:23.167900 16238 raft_consensus.cc:2468] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 19.
I20260709 13:21:23.169209 15568 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:23.154398 22562 leader_election.cc:290] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 21 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:23.169405 22566 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:23.177337 15677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a81ba00c9bd4a45ae8d2379570072fe" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4"
I20260709 13:21:23.176092 16231 raft_consensus.cc:2468] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 21.
I20260709 13:21:23.178153 15677 raft_consensus.cc:3060] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:23.181301 22546 raft_consensus.cc:2804] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:23.186270 15677 raft_consensus.cc:2468] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 15.
I20260709 13:21:23.190187 15566 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f; no voters: 
I20260709 13:21:23.191035 22574 raft_consensus.cc:2804] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:23.186813 22504 raft_consensus.cc:2804] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:23.187438 15642 leader_election.cc:304] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:23.188766 22546 raft_consensus.cc:697] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:23.191529 22574 raft_consensus.cc:697] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:21:23.191891 15567 leader_election.cc:336] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:23.193533 22562 raft_consensus.cc:2804] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:23.194836 22562 raft_consensus.cc:697] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:23.194801 22546 consensus_queue.cc:237] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:23.196054 22562 consensus_queue.cc:237] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:23.198254 22504 raft_consensus.cc:697] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
I20260709 13:21:23.200639 22574 consensus_queue.cc:237] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:23.204064 22504 consensus_queue.cc:237] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:23.212666 15641 consensus_peers.cc:597] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:23.217832 15567 consensus_peers.cc:597] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:23.221314 16204 consensus_peers.cc:597] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:23.223058 15641 consensus_peers.cc:597] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:23.227080 16231 raft_consensus.cc:1275] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:21:23.231920 22560 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:23.238070 22574 consensus_queue.cc:1048] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:21:23.242583 15567 leader_election.cc:336] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
W20260709 13:21:23.243335 15567 consensus_peers.cc:597] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:23.255710 22560 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:23.256597 16238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8153c7c094654fb492f5c47863ad170d" candidate_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:23.262907 16238 raft_consensus.cc:3060] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:21:23.262259 16204 consensus_peers.cc:597] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:23.268947 15567 consensus_peers.cc:597] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:23.274048 16238 raft_consensus.cc:2468] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498ef3e89a941e39c25f68cf64e6b0f in term 10.
I20260709 13:21:23.275331 15568 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:23.279858 22560 raft_consensus.cc:2804] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:21:23.280798 22560 raft_consensus.cc:697] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Becoming Leader. State: Replica: 5498ef3e89a941e39c25f68cf64e6b0f, State: Running, Role: LEADER
W20260709 13:21:23.287535 22584 log.cc:927] Time spent T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:21:23.296999 16204 consensus_peers.cc:597] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:23.295768 22560 consensus_queue.cc:237] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:23.319808 15418 catalog_manager.cc:5697] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 16 to 17, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 17 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:23.321573 15418 catalog_manager.cc:5697] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 20 to 21, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 21 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } } }
I20260709 13:21:23.323174 15418 catalog_manager.cc:5697] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 20 to 21, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 21 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:23.325881 16204 consensus_peers.cc:597] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:23.346616 15641 consensus_peers.cc:597] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:23.364230 15419 catalog_manager.cc:5697] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 11 to 12, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 12 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:23.366061 15419 catalog_manager.cc:5697] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 9 to 10, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 10 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:23.368012 15419 catalog_manager.cc:5697] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 19 to 20, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 20 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:23.371469 15423 catalog_manager.cc:5697] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 reported cstate change: term changed from 18 to 19, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68). New cstate: current_term: 19 leader_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:23.387478 15419 catalog_manager.cc:5697] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 21 to 22, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 22 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:23.389722 15419 catalog_manager.cc:5697] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 14 to 15, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 15 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:23.388758 22592 log.cc:927] Time spent T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.056s	user 0.000s	sys 0.002s
I20260709 13:21:23.392513 15419 catalog_manager.cc:5697] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 14 to 15, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 15 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:23.407563 15419 catalog_manager.cc:5697] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f reported cstate change: term changed from 18 to 19, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66). New cstate: current_term: 19 leader_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:23.417836 15641 consensus_peers.cc:597] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:23.430934 16204 consensus_peers.cc:597] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:23.447923 15595 raft_consensus.cc:1275] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Refusing update from remote peer dd1a14c3173d49dc9b55bb116e3783c9: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:21:23.449692 22559 consensus_queue.cc:1048] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:21:23.468163 16204 consensus_peers.cc:597] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:23.516817 16231 raft_consensus.cc:1275] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 22 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:21:23.518374 22546 consensus_queue.cc:1048] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:23.524127 16231 raft_consensus.cc:1275] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:21:23.526832 22560 consensus_queue.cc:1048] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:23.530668 16231 raft_consensus.cc:1275] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:21:23.532469 22546 consensus_queue.cc:1048] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:21:23.548264 15567 consensus_peers.cc:597] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:23.640487 15670 raft_consensus.cc:1275] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:21:23.659938 15641 consensus_peers.cc:597] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:23.643123 22574 consensus_queue.cc:1048] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:21:23.669659 15601 raft_consensus.cc:1275] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:21:23.671502 22568 consensus_queue.cc:1048] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:23.682562 16239 raft_consensus.cc:1275] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:21:23.684232 22568 consensus_queue.cc:1048] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:21:23.685609 15641 consensus_peers.cc:597] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:23.686015 15567 consensus_peers.cc:597] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:23.689574 15567 consensus_peers.cc:597] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:23.720513 16239 raft_consensus.cc:1275] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 13:21:23.723908 22560 consensus_queue.cc:1048] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:21:23.722962 15677 raft_consensus.cc:1275] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:21:23.730367 15567 consensus_peers.cc:597] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:23.738529 15567 consensus_peers.cc:597] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:23.739581 22566 consensus_queue.cc:1048] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:21:23.754403 15567 consensus_peers.cc:597] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:23.757468 15567 consensus_peers.cc:597] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:23.795087 14429 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 15549)
I20260709 13:21:23.883751 16231 raft_consensus.cc:1275] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Refusing update from remote peer 5498ef3e89a941e39c25f68cf64e6b0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:21:23.923305 22566 consensus_queue.cc:1048] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:21:24.135747 22605 log.cc:927] Time spent T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9: Append to log took a long time: real 0.075s	user 0.000s	sys 0.007s
W20260709 13:21:24.796384 14429 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 15549)
I20260709 13:21:24.830957 22520 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:24.832010 22520 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:24.839447 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
I20260709 13:21:24.840490 16231 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 15.
W20260709 13:21:24.840564 15641 leader_election.cc:336] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:24.838208 22520 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:24.843003 15642 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:24.844291 22520 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:24.844738 22520 raft_consensus.cc:493] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:24.845144 22520 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:24.868381 22520 raft_consensus.cc:515] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:24.885289 16231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced58b89726545d3af99de36a1aa1f30" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
I20260709 13:21:24.885882 16231 raft_consensus.cc:3060] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:24.889107 15641 leader_election.cc:336] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:24.894327 16231 raft_consensus.cc:2468] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 16.
I20260709 13:21:24.894899 22520 leader_election.cc:290] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 election: Requested vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:24.898967 15642 leader_election.cc:304] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:24.900321 22520 raft_consensus.cc:2804] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:24.901362 22520 raft_consensus.cc:697] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:24.902669 22520 consensus_queue.cc:237] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:25.021654 15423 catalog_manager.cc:5697] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 reported cstate change: term changed from 14 to 16, leader changed from d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65) to 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67). New cstate: current_term: 16 leader_uuid: "3ef186ec12554a1d87814af0a0ace9a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:25.182116 16204 consensus_peers.cc:597] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.203598 15567 consensus_peers.cc:597] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.233341 16204 consensus_peers.cc:597] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.294847 16204 consensus_peers.cc:597] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.295176 15641 consensus_peers.cc:597] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.319828 16204 consensus_peers.cc:597] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.322813 15567 consensus_peers.cc:597] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.326443 15567 consensus_peers.cc:597] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.339339 15567 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111) [suppressed 410 similar messages]
W20260709 13:21:25.366777 16204 consensus_peers.cc:597] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:25.369683 16238 raft_consensus.cc:1275] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Refusing update from remote peer 3ef186ec12554a1d87814af0a0ace9a4: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:21:25.394230 22568 consensus_queue.cc:1048] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:21:25.423668 15641 consensus_peers.cc:597] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.439994 15567 consensus_peers.cc:597] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.451145 16204 consensus_peers.cc:597] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:25.534865 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 56746.1us
W20260709 13:21:25.560513 15641 consensus_peers.cc:597] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.569344 15641 consensus_peers.cc:597] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:25.592908 15567 consensus_peers.cc:597] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:25.593444 15567 consensus_peers.cc:597] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.609334 16204 consensus_peers.cc:597] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:25.608510 15567 consensus_peers.cc:597] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.631098 15641 consensus_peers.cc:597] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.632560 15641 consensus_peers.cc:597] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.631549 15567 consensus_peers.cc:597] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.646184 15567 consensus_peers.cc:597] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.716295 15641 consensus_peers.cc:597] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.729614 15567 consensus_peers.cc:597] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.751569 15567 consensus_peers.cc:597] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.753320 16204 consensus_peers.cc:597] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.758589 16204 consensus_peers.cc:597] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.772025 15641 consensus_peers.cc:597] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.796371 15641 consensus_peers.cc:597] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.797042 14429 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 15549)
W20260709 13:21:25.825369 15567 consensus_peers.cc:597] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.823531 15641 consensus_peers.cc:597] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.911255 15641 consensus_peers.cc:597] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:25.913153 16204 consensus_peers.cc:597] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.918123 15567 consensus_peers.cc:597] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:25.978075 16204 consensus_peers.cc:597] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.999571 15641 consensus_peers.cc:597] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:26.076378 16204 consensus_peers.cc:597] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:26.145460 15641 consensus_peers.cc:597] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:26.147954 15567 consensus_peers.cc:597] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:26.227670 15567 consensus_peers.cc:597] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:26.237826 15567 consensus_peers.cc:597] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:26.283229 15567 consensus_peers.cc:597] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:26.296583 16204 consensus_peers.cc:597] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:26.421746 15567 consensus_peers.cc:597] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:26.482386 15567 consensus_peers.cc:597] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:26.797504 14429 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 15549)
W20260709 13:21:27.348378 16205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:41593, user_credentials={real_user=slave}} blocked reactor thread for 93187.3us
W20260709 13:21:27.392645 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 138667us
W20260709 13:21:27.517597 15641 consensus_peers.cc:597] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:27.573253 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 63769us
W20260709 13:21:27.718703 16204 consensus_peers.cc:597] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:27.761512 15641 consensus_peers.cc:597] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:27.770423 16204 consensus_peers.cc:597] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:27.797989 14429 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 15549)
W20260709 13:21:27.817023 15641 consensus_peers.cc:597] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:27.893178 16204 consensus_peers.cc:597] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:27.936594 16204 consensus_peers.cc:597] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:27.963498 16204 consensus_peers.cc:597] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:27.973675 15567 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 49168.9us
W20260709 13:21:27.982051 15567 consensus_peers.cc:597] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.006140 15641 consensus_peers.cc:597] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.074532 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 45962.7us
W20260709 13:21:28.112182 16204 consensus_peers.cc:597] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:28.198621 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 67903.7us
W20260709 13:21:28.204125 15567 consensus_peers.cc:597] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.204913 15567 consensus_peers.cc:597] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.240626 15567 consensus_peers.cc:597] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.241246 15567 consensus_peers.cc:597] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.241657 15567 consensus_peers.cc:597] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.242185 15567 consensus_peers.cc:597] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:28.245961 15567 consensus_peers.cc:597] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.263381 16204 consensus_peers.cc:597] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.270318 15641 consensus_peers.cc:597] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.270924 15641 consensus_peers.cc:597] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.298486 15641 consensus_peers.cc:597] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.302161 15641 consensus_peers.cc:597] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.302775 15641 consensus_peers.cc:597] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.325177 15567 consensus_peers.cc:597] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.331166 15567 consensus_peers.cc:597] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.352198 15641 consensus_peers.cc:597] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.413745 15641 consensus_peers.cc:597] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.456749 15641 consensus_peers.cc:597] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:28.459372 16204 consensus_peers.cc:597] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:28.483348 15567 consensus_peers.cc:597] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.575984 16204 consensus_peers.cc:597] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.623023 15641 consensus_peers.cc:597] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:28.628765 15567 consensus_peers.cc:597] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:28.625095 16204 consensus_peers.cc:597] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:28.715204 16204 consensus_peers.cc:597] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.720472 15567 consensus_peers.cc:597] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:28.724974 15567 consensus_peers.cc:597] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:28.740082 15567 consensus_peers.cc:597] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.749168 16204 consensus_peers.cc:597] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.763449 16204 consensus_peers.cc:597] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.826670 14429 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 15549)
W20260709 13:21:28.871861 15567 consensus_peers.cc:597] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:28.896415 15567 consensus_peers.cc:597] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:29.109983 15567 consensus_peers.cc:597] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:29.110481 15567 consensus_peers.cc:597] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:29.362178 15566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 74025.8us
W20260709 13:21:29.827157 14429 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 15549)
W20260709 13:21:30.273485 16263 debug-util.cc:398] Leaking SignalData structure 0x7b08003a5e20 after lost signal to thread 14432
W20260709 13:21:30.274830 16263 debug-util.cc:398] Leaking SignalData structure 0x7b08003a6320 after lost signal to thread 15460
W20260709 13:21:30.276216 16263 debug-util.cc:398] Leaking SignalData structure 0x7b080030cc20 after lost signal to thread 15627
W20260709 13:21:30.277408 16263 debug-util.cc:398] Leaking SignalData structure 0x7b080030c2a0 after lost signal to thread 15701
W20260709 13:21:30.278821 16263 debug-util.cc:398] Leaking SignalData structure 0x7b08003485c0 after lost signal to thread 16266
W20260709 13:21:30.474198 16204 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111) [suppressed 407 similar messages]
W20260709 13:21:30.506635 16204 consensus_peers.cc:597] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:30.519515 16204 consensus_peers.cc:597] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:30.682240 15641 consensus_peers.cc:597] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:30.686566 15641 consensus_peers.cc:597] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:30.704078 15642 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.822s	user 0.012s	sys 0.011s
W20260709 13:21:30.704595 15642 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.013s	sys 0.011s
W20260709 13:21:30.704882 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 823298us
W20260709 13:21:30.719638 15568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 175440us
W20260709 13:21:30.722889 15643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.841s	user 0.002s	sys 0.025s
W20260709 13:21:30.723291 15643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.002s	sys 0.025s
W20260709 13:21:30.750236 16204 consensus_peers.cc:597] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:30.755700 16204 consensus_peers.cc:597] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:30.756261 16204 consensus_peers.cc:597] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:30.756829 16204 consensus_peers.cc:597] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:30.755080 15641 consensus_peers.cc:597] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:30.795635 16204 consensus_peers.cc:597] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:30.806928 15567 consensus_peers.cc:597] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:30.827621 14429 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 15549)
W20260709 13:21:30.901279 15641 consensus_peers.cc:597] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:30.902133 15567 consensus_peers.cc:597] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:30.967967 16204 consensus_peers.cc:597] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:31.018781 16204 consensus_peers.cc:597] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.075882 15567 consensus_peers.cc:597] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.081116 16204 consensus_peers.cc:597] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.145617 15641 consensus_peers.cc:597] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.149643 15641 consensus_peers.cc:597] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.164016 15567 consensus_peers.cc:597] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.167433 15567 consensus_peers.cc:597] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.201752 15641 consensus_peers.cc:597] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.202800 16204 consensus_peers.cc:597] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.234774 15567 consensus_peers.cc:597] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.271986 15567 consensus_peers.cc:597] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:31.278872 15567 consensus_peers.cc:597] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.317540 15641 consensus_peers.cc:597] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.332799 15641 consensus_peers.cc:597] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.339434 16204 consensus_peers.cc:597] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.349261 15567 consensus_peers.cc:597] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.356726 15641 consensus_peers.cc:597] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.369177 15567 consensus_peers.cc:597] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:31.393637 15567 consensus_peers.cc:597] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:31.465282 15567 consensus_peers.cc:597] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:31.515967 15567 consensus_peers.cc:597] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.555802 15698 debug-util.cc:398] Leaking SignalData structure 0x7b080033c540 after lost signal to thread 14432
W20260709 13:21:31.556897 15698 debug-util.cc:398] Leaking SignalData structure 0x7b08003840a0 after lost signal to thread 15460
W20260709 13:21:31.561758 15698 debug-util.cc:398] Leaking SignalData structure 0x7b0800342400 after lost signal to thread 15627
W20260709 13:21:31.562961 15698 debug-util.cc:398] Leaking SignalData structure 0x7b08003ff7c0 after lost signal to thread 15701
W20260709 13:21:31.564146 15698 debug-util.cc:398] Leaking SignalData structure 0x7b08003fafa0 after lost signal to thread 16266
W20260709 13:21:31.588367 15567 consensus_peers.cc:597] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:31.609683 15567 consensus_peers.cc:597] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:31.639117 16204 consensus_peers.cc:597] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.690419 16203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:32779, user_credentials={real_user=slave}} blocked reactor thread for 59853.8us
W20260709 13:21:31.748800 15567 consensus_peers.cc:597] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.796029 15567 consensus_peers.cc:597] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:31.828100 14429 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 15549)
W20260709 13:21:31.853336 15641 consensus_peers.cc:597] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:31.884635 15567 consensus_peers.cc:597] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.922389 15641 consensus_peers.cc:597] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:32.033470 15641 consensus_peers.cc:597] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:32.181262 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 61276us
W20260709 13:21:32.201397 16204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 95541.5us
W20260709 13:21:32.235747 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 49485.1us
W20260709 13:21:32.365849 15567 consensus_peers.cc:597] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:32.562748 15641 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:42481, user_credentials={real_user=slave}} blocked reactor thread for 43770.1us
W20260709 13:21:32.828596 14429 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 15549)
W20260709 13:21:32.931130 16204 consensus_peers.cc:597] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.003018 15641 consensus_peers.cc:597] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.009703 16204 consensus_peers.cc:597] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.187135 16204 consensus_peers.cc:597] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:33.188594 15641 consensus_peers.cc:597] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.214913 16204 consensus_peers.cc:597] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.289075 16204 consensus_peers.cc:597] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.317070 16204 consensus_peers.cc:597] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:33.437664 15641 consensus_peers.cc:597] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.455766 16204 consensus_peers.cc:597] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.485219 15567 consensus_peers.cc:597] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.520995 15641 consensus_peers.cc:597] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.541543 15567 consensus_peers.cc:597] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.557982 16204 consensus_peers.cc:597] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:33.558564 16204 consensus_peers.cc:597] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.581758 15641 consensus_peers.cc:597] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.620662 15567 consensus_peers.cc:597] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.623183 15641 consensus_peers.cc:597] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:33.639652 15567 consensus_peers.cc:597] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:33.722810 16204 consensus_peers.cc:597] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.736570 15641 consensus_peers.cc:597] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.756953 15641 consensus_peers.cc:597] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.758822 15567 consensus_peers.cc:597] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.764182 15641 consensus_peers.cc:597] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.770202 15567 consensus_peers.cc:597] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.770823 15567 consensus_peers.cc:597] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.774538 16204 consensus_peers.cc:597] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.809176 15567 consensus_peers.cc:597] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.829111 14429 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 15549)
W20260709 13:21:33.871948 15567 consensus_peers.cc:597] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:33.981037 15641 consensus_peers.cc:597] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:34.123116 16204 consensus_peers.cc:597] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:34.131124 15641 consensus_peers.cc:597] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:34.323506 15641 consensus_peers.cc:597] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:34.344602 16204 consensus_peers.cc:597] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:34.561748 15641 consensus_peers.cc:597] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:34.693790 14429 tablet_server.cc:196] TabletServer@127.14.23.65:0 shutdown complete.
I20260709 13:21:34.899154 14429 tablet_server.cc:179] TabletServer@127.14.23.66:0 shutting down...
W20260709 13:21:34.939002 15642 consensus_peers.cc:597] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:34.940169 16205 consensus_peers.cc:597] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:34.952262 15642 consensus_peers.cc:597] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:34.956451 15642 consensus_peers.cc:597] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:34.961483 15567 consensus_peers.cc:597] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:34.962028 15567 consensus_peers.cc:597] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:34.973300 15567 consensus_peers.cc:597] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:34.973814 15567 consensus_peers.cc:597] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:34.974454 15567 consensus_peers.cc:597] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:34.975114 15567 consensus_peers.cc:597] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:34.978348 15567 consensus_peers.cc:597] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:34.980293 15567 consensus_peers.cc:597] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:35.020625 16205 consensus_peers.cc:597] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:35.033978 15642 consensus_peers.cc:597] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:35.049837 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:35.051079 14429 tablet_replica.cc:333] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.052152 14429 raft_consensus.cc:2243] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.053894 14429 raft_consensus.cc:2272] T ecb5853a05664246a26d3fc7ecf92777 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.060573 14429 tablet_replica.cc:333] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.066213 14429 raft_consensus.cc:2243] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.067549 14429 raft_consensus.cc:2272] T 7dbabc834ad54c16ae25f79064932f42 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.070466 14429 tablet_replica.cc:333] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.071300 14429 raft_consensus.cc:2243] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.072027 14429 raft_consensus.cc:2272] T f9328896aa5a4fdc8fa7c908efdae695 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.083918 14429 tablet_replica.cc:333] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.084964 14429 raft_consensus.cc:2243] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:21:35.088104 15642 consensus_peers.cc:597] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:35.079087 16205 consensus_peers.cc:597] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:35.094333 16205 consensus_peers.cc:597] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:35.098796 14429 raft_consensus.cc:2272] T 7b8ed23a60014da39806e750cee98fa6 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.102075 14429 tablet_replica.cc:333] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.103147 14429 raft_consensus.cc:2243] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.104301 14429 raft_consensus.cc:2272] T 974a060267d24f00888c0942dfbca77e P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:35.105819 16205 consensus_peers.cc:597] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:35.107414 14429 tablet_replica.cc:333] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.108326 14429 raft_consensus.cc:2243] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.108978 14429 raft_consensus.cc:2272] T e93d4a9ac6e844da87dcd8c6bfa7549b P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.111788 14429 tablet_replica.cc:333] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.112619 14429 raft_consensus.cc:2243] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.113435 14429 raft_consensus.cc:2272] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.117241 14429 tablet_replica.cc:333] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.121184 14429 raft_consensus.cc:2243] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.122699 14429 raft_consensus.cc:2272] T 88a66a9fb8df432fb0d1d7ba3756235e P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:35.124961 15642 consensus_peers.cc:597] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:35.125682 14429 tablet_replica.cc:333] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.126602 14429 raft_consensus.cc:2243] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.130836 14429 raft_consensus.cc:2272] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:35.133313 15642 consensus_peers.cc:597] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:35.134181 15642 consensus_peers.cc:597] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:35.134407 14429 tablet_replica.cc:333] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.135365 14429 raft_consensus.cc:2243] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.136023 14429 raft_consensus.cc:2272] T 2c0b73c09a0849518f4bc6e3886a348e P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:35.136991 15642 consensus_peers.cc:597] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:35.139038 14429 tablet_replica.cc:333] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.139957 14429 raft_consensus.cc:2243] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.140635 14429 raft_consensus.cc:2272] T ccb13c2addbf41569d0c2c23750bd4c9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.143637 14429 tablet_replica.cc:333] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.144518 14429 raft_consensus.cc:2243] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.146374 14429 raft_consensus.cc:2272] T 50726e0d396948b79be43aece292acb5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.149370 14429 tablet_replica.cc:333] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.150415 14429 raft_consensus.cc:2243] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.151753 14429 raft_consensus.cc:2272] T 0af5649725244ae599dfc6ed7aeac5f1 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.154794 14429 tablet_replica.cc:333] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.155723 14429 raft_consensus.cc:2243] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.156620 14429 raft_consensus.cc:2272] T 582343936346420b87ecf2eebd00dc25 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:35.156693 15567 consensus_peers.cc:597] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:35.160806 14429 tablet_replica.cc:333] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.161666 14429 raft_consensus.cc:2243] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.164100 14429 raft_consensus.cc:2272] T db313a4cd18a47a2b72587d04ed6e747 P 5498ef3e89a941e39c25f68cf64e6b0f [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.167048 14429 tablet_replica.cc:333] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.167837 14429 raft_consensus.cc:2243] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.169077 14429 raft_consensus.cc:2272] T 061ed80a3fca4096b7c13dc0e2256ef0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.171890 14429 tablet_replica.cc:333] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.172749 14429 raft_consensus.cc:2243] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.174080 14429 raft_consensus.cc:2272] T 8a81ba00c9bd4a45ae8d2379570072fe P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.178838 14429 tablet_replica.cc:333] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
W20260709 13:21:35.180430 16205 consensus_peers.cc:597] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:35.181464 14429 raft_consensus.cc:2243] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.182452 14429 raft_consensus.cc:2272] T 340e065cf2744fde9d65230a8ea91e14 P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.185586 14429 tablet_replica.cc:333] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.186443 14429 raft_consensus.cc:2243] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.187215 14429 raft_consensus.cc:2272] T 7a5315557f994ec689bb9a79498bb2d5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.190038 14429 tablet_replica.cc:333] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.190826 14429 raft_consensus.cc:2243] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.191524 14429 raft_consensus.cc:2272] T f25e63ed82fa4e008109b74fe8df71ba P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.194434 14429 tablet_replica.cc:333] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.195201 14429 raft_consensus.cc:2243] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.196560 14429 raft_consensus.cc:2272] T 98787e13cd7a49528aeeb59c72559e46 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.199491 14429 tablet_replica.cc:333] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.200248 14429 raft_consensus.cc:2243] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.200896 14429 raft_consensus.cc:2272] T 5243f722506441269fac66aa989ce36d P 5498ef3e89a941e39c25f68cf64e6b0f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.222024 14429 tablet_replica.cc:333] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.222833 14429 raft_consensus.cc:2243] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.223764 14429 raft_consensus.cc:2272] T 1fa44270fe124591b1e8c82c82b89efe P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.226575 14429 tablet_replica.cc:333] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.227355 14429 raft_consensus.cc:2243] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.232615 14429 raft_consensus.cc:2272] T 3088d5241ec64ddc919925d5dccfeda0 P 5498ef3e89a941e39c25f68cf64e6b0f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.245575 14429 tablet_replica.cc:333] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.246467 14429 raft_consensus.cc:2243] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.247447 14429 raft_consensus.cc:2272] T c1e80bd01b0c4006852da32ca0eeb0ae P 5498ef3e89a941e39c25f68cf64e6b0f [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:35.251827 15642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:39627, user_credentials={real_user=slave}} blocked reactor thread for 52535.7us
I20260709 13:21:35.272281 14429 tablet_replica.cc:333] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.273247 14429 raft_consensus.cc:2243] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.273983 14429 raft_consensus.cc:2272] T d481434f60dc4276831163b535fb8159 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.276899 14429 tablet_replica.cc:333] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.277803 14429 raft_consensus.cc:2243] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.278512 14429 raft_consensus.cc:2272] T b87d5c4641304e03a8a3fd4782b043e5 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.281827 14429 tablet_replica.cc:333] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.292131 14429 raft_consensus.cc:2243] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.292824 14429 raft_consensus.cc:2272] T 6db63c1e295d42d9938e0ef858365436 P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.295697 14429 tablet_replica.cc:333] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.296533 14429 raft_consensus.cc:2243] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.297799 14429 raft_consensus.cc:2272] T 4da1202d21704a61a455cc91fa7c4022 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:35.298247 16204 consensus_peers.cc:597] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:35.300479 14429 tablet_replica.cc:333] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.301335 14429 raft_consensus.cc:2243] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.302618 14429 raft_consensus.cc:2272] T bd36c2cfd99c45cea9e48096d54bed1a P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:35.303936 15642 consensus_peers.cc:597] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:35.306865 16205 consensus_peers.cc:597] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:35.309976 14429 tablet_replica.cc:333] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.310812 14429 raft_consensus.cc:2243] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.312018 14429 raft_consensus.cc:2272] T 8153c7c094654fb492f5c47863ad170d P 5498ef3e89a941e39c25f68cf64e6b0f [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.314689 14429 tablet_replica.cc:333] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.315439 14429 raft_consensus.cc:2243] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.316653 14429 raft_consensus.cc:2272] T 271b5d375f0145b78b28a0c13328ce7f P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.319433 14429 tablet_replica.cc:333] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.320176 14429 raft_consensus.cc:2243] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.320802 14429 raft_consensus.cc:2272] T ab68091b31d248a6ac99ca234a252f77 P 5498ef3e89a941e39c25f68cf64e6b0f [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.326102 14429 tablet_replica.cc:333] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.326845 14429 raft_consensus.cc:2243] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.327888 14429 raft_consensus.cc:2272] T 6139553a47b64d798a29413921f18396 P 5498ef3e89a941e39c25f68cf64e6b0f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.331246 14429 tablet_replica.cc:333] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.331967 14429 raft_consensus.cc:2243] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.333261 14429 raft_consensus.cc:2272] T d7b7542687ad43acb128a39bb1f40829 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.342520 14429 tablet_replica.cc:333] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.347609 14429 raft_consensus.cc:2243] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.348384 14429 raft_consensus.cc:2272] T 4344539a12c94917991c2f39d3ff5858 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.351257 14429 tablet_replica.cc:333] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
W20260709 13:21:35.351912 16204 consensus_peers.cc:597] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:35.353230 14429 raft_consensus.cc:2243] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.354081 14429 raft_consensus.cc:2272] T ef08c91ca56042df89e729f13df83ed9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.357112 14429 tablet_replica.cc:333] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.357863 14429 raft_consensus.cc:2243] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.363209 14429 raft_consensus.cc:2272] T b9a4c0b115544577a855e9c7383d9a48 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.370074 14429 tablet_replica.cc:333] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.370842 14429 raft_consensus.cc:2243] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:21:35.371989 15642 consensus_peers.cc:597] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:35.372063 14429 raft_consensus.cc:2272] T 12b56c6369484d92b328d276d4fca9e3 P 5498ef3e89a941e39c25f68cf64e6b0f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.376461 14429 tablet_replica.cc:333] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.379679 14429 raft_consensus.cc:2243] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.380805 14429 raft_consensus.cc:2272] T 3502bb7169aa41bb9ebdaeddf5068af9 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.384637 14429 tablet_replica.cc:333] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.385500 14429 raft_consensus.cc:2243] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 LEADER]: Raft consensus shutting down.
W20260709 13:21:35.386996 16205 consensus_peers.cc:597] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:35.387801 14429 raft_consensus.cc:2272] T b2c57bed49da41d795c8b21ed4461f1f P 5498ef3e89a941e39c25f68cf64e6b0f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.390813 14429 tablet_replica.cc:333] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.391511 14429 raft_consensus.cc:2243] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.392191 14429 raft_consensus.cc:2272] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 5498ef3e89a941e39c25f68cf64e6b0f [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:35.393139 15642 consensus_peers.cc:597] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:35.395336 14429 tablet_replica.cc:333] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.396121 14429 raft_consensus.cc:2243] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:21:35.397374 14429 raft_consensus.cc:2272] T d4fd164d74054954b750fe9958e66778 P 5498ef3e89a941e39c25f68cf64e6b0f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:35.400377 14429 tablet_replica.cc:333] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f: stopping tablet replica
I20260709 13:21:35.401166 14429 raft_consensus.cc:2243] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:35.401868 14429 raft_consensus.cc:2272] T 4b73eb6a0096455b87f6109b059cf0ee P 5498ef3e89a941e39c25f68cf64e6b0f [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:35.462276 16205 consensus_peers.cc:597] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:35.474308 16205 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 445 similar messages]
W20260709 13:21:35.539732 15641 consensus_peers.cc:597] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:35.560287 16204 consensus_peers.cc:597] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:35.914119 16204 consensus_peers.cc:597] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:35.916587 16204 consensus_peers.cc:597] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:35.917068 16204 consensus_peers.cc:597] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:35.919365 16204 consensus_peers.cc:597] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:35.929327 16204 consensus_peers.cc:597] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:35.944964 15641 consensus_peers.cc:597] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:35.945796 15641 consensus_peers.cc:597] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:36.079761 15641 consensus_peers.cc:597] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:36.112438 22685 raft_consensus.cc:493] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:36.112991 22685 raft_consensus.cc:515] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:36.115157 22685 leader_election.cc:290] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:21:36.120793 15641 leader_election.cc:336] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
W20260709 13:21:36.121469 15641 consensus_peers.cc:597] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:36.121417 15642 leader_election.cc:336] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
I20260709 13:21:36.126958 15642 leader_election.cc:304] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:36.127830 22685 raft_consensus.cc:2749] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:21:36.138288 16204 consensus_peers.cc:597] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:36.139477 22680 raft_consensus.cc:493] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:36.140075 22680 raft_consensus.cc:515] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:36.142302 22680 leader_election.cc:290] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:21:36.146087 16205 leader_election.cc:336] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
W20260709 13:21:36.146605 16204 leader_election.cc:336] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:36.146958 16204 leader_election.cc:304] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:36.147519 22680 raft_consensus.cc:2749] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:36.151947 16204 consensus_peers.cc:597] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:36.255402 22680 raft_consensus.cc:493] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:36.256093 22680 raft_consensus.cc:515] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:36.258397 22680 leader_election.cc:290] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:21:36.260139 16204 consensus_peers.cc:597] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:36.262488 16204 leader_election.cc:336] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
W20260709 13:21:36.263186 16205 leader_election.cc:336] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
I20260709 13:21:36.263630 16205 leader_election.cc:304] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:36.264494 22680 raft_consensus.cc:2749] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:36.269686 14429 tablet_server.cc:196] TabletServer@127.14.23.66:0 shutdown complete.
I20260709 13:21:36.301468 22679 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:36.302198 22679 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:36.305140 22679 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:36.305786 15675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3ef186ec12554a1d87814af0a0ace9a4" is_pre_election: true
W20260709 13:21:36.314342 16205 leader_election.cc:336] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
I20260709 13:21:36.314916 16205 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:36.315740 22679 raft_consensus.cc:2749] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:21:36.352458 15641 consensus_peers.cc:597] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:36.355170 22672 raft_consensus.cc:493] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:36.355770 22672 raft_consensus.cc:515] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } }
I20260709 13:21:36.358917 22672 leader_election.cc:290] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:21:36.364930 15642 leader_election.cc:336] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
W20260709 13:21:36.365408 15641 leader_election.cc:336] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:36.365835 15641 leader_election.cc:304] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:36.366544 22685 raft_consensus.cc:2749] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:36.374233 22685 raft_consensus.cc:493] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:36.374811 22685 raft_consensus.cc:515] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:36.376927 22685 leader_election.cc:290] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:21:36.382169 15642 leader_election.cc:336] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
W20260709 13:21:36.386250 15641 leader_election.cc:336] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:36.386813 15641 leader_election.cc:304] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:36.387653 22685 raft_consensus.cc:2749] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:36.389233 14429 tablet_server.cc:179] TabletServer@127.14.23.67:0 shutting down...
I20260709 13:21:36.398814 22679 raft_consensus.cc:493] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:36.399535 22679 raft_consensus.cc:515] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:36.403980 22679 leader_election.cc:290] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:36.404425 22685 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:36.404968 22685 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:36.406581 16205 leader_election.cc:336] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
I20260709 13:21:36.412534 22685 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:36.413384 22679 raft_consensus.cc:493] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:36.414055 22679 raft_consensus.cc:515] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:36.415818 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:21:36.416337 15642 leader_election.cc:336] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
I20260709 13:21:36.416517 16236 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 21.
W20260709 13:21:36.417691 16204 leader_election.cc:336] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:36.418179 16204 leader_election.cc:304] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:36.419582 22680 raft_consensus.cc:2749] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:36.420661 22679 leader_election.cc:290] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:21:36.421800 15641 consensus_peers.cc:597] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:36.425284 16205 leader_election.cc:336] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
I20260709 13:21:36.428238 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:21:36.428346 16203 consensus_peers.cc:597] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Couldn't send request to peer 3ef186ec12554a1d87814af0a0ace9a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:36.429543 22672 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:36.429977 22672 raft_consensus.cc:493] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:36.430344 22672 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:21:36.437412 16203 consensus_peers.cc:597] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Couldn't send request to peer 3ef186ec12554a1d87814af0a0ace9a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:36.439855 16204 leader_election.cc:336] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:36.440330 16204 leader_election.cc:304] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:36.441277 22679 raft_consensus.cc:493] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:36.441951 22679 raft_consensus.cc:515] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:36.443137 22680 raft_consensus.cc:2749] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:36.446317 22679 leader_election.cc:290] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
I20260709 13:21:36.451679 22686 raft_consensus.cc:493] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
W20260709 13:21:36.452620 16205 leader_election.cc:336] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
I20260709 13:21:36.452373 22686 raft_consensus.cc:515] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:36.455943 22686 leader_election.cc:290] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:21:36.474861 16203 leader_election.cc:336] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:36.475315 16203 leader_election.cc:304] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:21:36.479290 16203 consensus_peers.cc:597] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Couldn't send request to peer 3ef186ec12554a1d87814af0a0ace9a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:36.479537 15641 leader_election.cc:336] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
W20260709 13:21:36.481951 15642 leader_election.cc:336] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
I20260709 13:21:36.482174 22687 raft_consensus.cc:2749] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:36.483162 15642 leader_election.cc:304] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
W20260709 13:21:36.480129 15641 consensus_peers.cc:597] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:36.484803 22684 raft_consensus.cc:2749] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:36.485467 15641 consensus_peers.cc:597] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:36.476190 22672 raft_consensus.cc:515] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
W20260709 13:21:36.480342 16203 consensus_peers.cc:597] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Couldn't send request to peer 3ef186ec12554a1d87814af0a0ace9a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:36.489636 22672 leader_election.cc:290] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 22 election: Requested vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:36.491788 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c436bd48e6b4ad9a1bb596e2ecb15fc" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9"
W20260709 13:21:36.492682 15641 consensus_peers.cc:597] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:36.492903 16236 raft_consensus.cc:3060] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:21:36.494161 15642 leader_election.cc:336] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
I20260709 13:21:36.497193 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:36.498081 14429 tablet_replica.cc:333] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.499044 14429 raft_consensus.cc:2243] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.499369 16236 raft_consensus.cc:2468] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef186ec12554a1d87814af0a0ace9a4 in term 22.
I20260709 13:21:36.499701 14429 raft_consensus.cc:2272] T 0f8365fc4e2f4decb3d554a43e7d4d52 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.501017 15642 leader_election.cc:304] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4, dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f
W20260709 13:21:36.491632 16203 consensus_peers.cc:597] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Couldn't send request to peer 3ef186ec12554a1d87814af0a0ace9a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:36.501902 22688 raft_consensus.cc:2804] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:21:36.502724 22680 raft_consensus.cc:493] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:36.502528 22688 raft_consensus.cc:697] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 22 LEADER]: Becoming Leader. State: Replica: 3ef186ec12554a1d87814af0a0ace9a4, State: Running, Role: LEADER
I20260709 13:21:36.503247 22680 raft_consensus.cc:515] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:36.504453 22688 consensus_queue.cc:237] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:36.505425 22680 leader_election.cc:290] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:36.509438 14429 tablet_replica.cc:333] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.510386 14429 raft_consensus.cc:2243] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.511014 14429 raft_consensus.cc:2272] T 7562b5f342134e39a5e6934d5ca16e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:36.511108 16205 leader_election.cc:336] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
W20260709 13:21:36.512696 16204 leader_election.cc:336] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:36.513820 14429 tablet_replica.cc:333] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.514815 14429 raft_consensus.cc:2243] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:21:36.514886 16204 leader_election.cc:304] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:36.515982 14429 raft_consensus.cc:2272] T ef08c91ca56042df89e729f13df83ed9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.518555 14429 tablet_replica.cc:333] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.519336 14429 raft_consensus.cc:2243] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.520072 14429 raft_consensus.cc:2272] T f9328896aa5a4fdc8fa7c908efdae695 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.523942 14429 tablet_replica.cc:333] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.524773 14429 raft_consensus.cc:2243] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.526255 22680 raft_consensus.cc:2749] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:36.526512 14429 raft_consensus.cc:2272] T d6670c20b1954b36a4920b44d1a8209d P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:36.532370 16204 consensus_peers.cc:597] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:36.535811 22688 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:36.536780 14429 tablet_replica.cc:333] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.536422 22688 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:36.537673 14429 raft_consensus.cc:2243] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:21:36.538575 22688 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), dd1a14c3173d49dc9b55bb116e3783c9 (127.14.23.68:39627)
I20260709 13:21:36.538780 14429 raft_consensus.cc:2272] T e93d4a9ac6e844da87dcd8c6bfa7549b P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:36.540627 16203 consensus_peers.cc:597] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Couldn't send request to peer 3ef186ec12554a1d87814af0a0ace9a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:36.541404 16236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3502bb7169aa41bb9ebdaeddf5068af9" candidate_uuid: "3ef186ec12554a1d87814af0a0ace9a4" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" is_pre_election: true
W20260709 13:21:36.542212 15642 leader_election.cc:336] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
I20260709 13:21:36.542441 14429 tablet_replica.cc:333] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.542910 15642 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, dd1a14c3173d49dc9b55bb116e3783c9
I20260709 13:21:36.543077 14429 raft_consensus.cc:2243] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.543504 22688 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:36.543717 14429 raft_consensus.cc:2272] T 88a66a9fb8df432fb0d1d7ba3756235e P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.545830 14429 tablet_replica.cc:333] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.546443 14429 raft_consensus.cc:2243] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.547044 14429 raft_consensus.cc:2272] T 50726e0d396948b79be43aece292acb5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.548372 22688 raft_consensus.cc:493] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:36.548784 22688 raft_consensus.cc:515] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:36.549232 14429 tablet_replica.cc:333] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.549824 14429 raft_consensus.cc:2243] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:21:36.550552 14429 pending_rounds.cc:70] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: Trying to abort 1 pending ops.
I20260709 13:21:36.550684 22688 leader_election.cc:290] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:36.550825 14429 pending_rounds.cc:77] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4: Aborting op as it isn't in flight: id { term: 22 index: 17 } timestamp: 7305639102491664384 op_type: NO_OP noop_request { }
I20260709 13:21:36.551369 14429 raft_consensus.cc:2889] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 22 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:21:36.551698 14429 raft_consensus.cc:2272] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P 3ef186ec12554a1d87814af0a0ace9a4 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.552886 22680 raft_consensus.cc:493] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:36.553417 22680 raft_consensus.cc:515] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:36.554663 22688 raft_consensus.cc:493] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:36.554944 14429 tablet_replica.cc:333] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
W20260709 13:21:36.555186 15641 leader_election.cc:336] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:36.555855 14429 raft_consensus.cc:2243] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.555393 22680 leader_election.cc:290] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:36.555307 22688 raft_consensus.cc:515] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } }
I20260709 13:21:36.556845 14429 raft_consensus.cc:2272] T b99eaeae24b44aa78cfa8c90caca288e P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:36.557583 15642 leader_election.cc:336] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
I20260709 13:21:36.557988 15642 leader_election.cc:304] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:36.558542 22672 raft_consensus.cc:2749] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:36.559665 14429 tablet_replica.cc:333] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.560528 14429 raft_consensus.cc:2243] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:21:36.561601 14429 raft_consensus.cc:2272] T 2c0b73c09a0849518f4bc6e3886a348e P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.561949 22688 leader_election.cc:290] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:21:36.563723 15641 leader_election.cc:336] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:36.564361 14429 tablet_replica.cc:333] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
W20260709 13:21:36.562315 16204 leader_election.cc:336] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:36.565155 14429 raft_consensus.cc:2243] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Raft consensus shutting down.
W20260709 13:21:36.565243 15642 leader_election.cc:336] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
I20260709 13:21:36.565702 15642 leader_election.cc:304] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ef186ec12554a1d87814af0a0ace9a4; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:36.565812 14429 raft_consensus.cc:2272] T ccb13c2addbf41569d0c2c23750bd4c9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.565891 22680 raft_consensus.cc:493] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:36.566890 22688 raft_consensus.cc:2749] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:36.566695 22680 raft_consensus.cc:515] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:36.568467 14429 tablet_replica.cc:333] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.569255 22680 leader_election.cc:290] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:36.569334 14429 raft_consensus.cc:2243] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:21:36.570654 14429 raft_consensus.cc:2272] T 4c48036212bb4277acf36aac79d8b35e P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:36.571046 16205 leader_election.cc:336] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
I20260709 13:21:36.571453 16205 leader_election.cc:304] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:36.572304 22687 raft_consensus.cc:2749] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:36.573519 14429 tablet_replica.cc:333] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.574330 14429 raft_consensus.cc:2243] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:21:36.575403 14429 raft_consensus.cc:2272] T e67cef93bbdb4ff88a2ee2ac558ba090 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.577955 14429 tablet_replica.cc:333] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
W20260709 13:21:36.577821 16205 leader_election.cc:336] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
W20260709 13:21:36.578579 16203 consensus_peers.cc:597] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Couldn't send request to peer 3ef186ec12554a1d87814af0a0ace9a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:36.578706 16204 leader_election.cc:336] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:36.578786 14429 raft_consensus.cc:2243] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.579156 16204 leader_election.cc:304] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:36.579882 22680 raft_consensus.cc:2749] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:36.580119 14429 raft_consensus.cc:2272] T 582343936346420b87ecf2eebd00dc25 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.582340 14429 tablet_replica.cc:333] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.583009 14429 raft_consensus.cc:2243] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.583671 14429 raft_consensus.cc:2272] T 8a81ba00c9bd4a45ae8d2379570072fe P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.586293 14429 tablet_replica.cc:333] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.587001 14429 raft_consensus.cc:2243] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:21:36.587338 22680 raft_consensus.cc:493] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:36.587847 22680 raft_consensus.cc:515] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:36.588230 14429 raft_consensus.cc:2272] T 01ad9d7b8dba45be8fdf625f27e8d273 P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.591069 14429 tablet_replica.cc:333] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.591096 22680 leader_election.cc:290] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
I20260709 13:21:36.591881 14429 raft_consensus.cc:2243] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.592533 14429 raft_consensus.cc:2272] T f25e63ed82fa4e008109b74fe8df71ba P 3ef186ec12554a1d87814af0a0ace9a4 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.595340 14429 tablet_replica.cc:333] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
W20260709 13:21:36.595852 16205 leader_election.cc:336] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
I20260709 13:21:36.596117 14429 raft_consensus.cc:2243] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.596868 14429 raft_consensus.cc:2272] T 98787e13cd7a49528aeeb59c72559e46 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:36.598027 16204 leader_election.cc:336] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
I20260709 13:21:36.598510 16204 leader_election.cc:304] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:36.599197 22680 raft_consensus.cc:2749] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:36.599558 14429 tablet_replica.cc:333] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.600288 14429 raft_consensus.cc:2243] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:21:36.601251 14429 raft_consensus.cc:2272] T 340e065cf2744fde9d65230a8ea91e14 P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.603279 14429 tablet_replica.cc:333] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.603829 14429 raft_consensus.cc:2243] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:21:36.604636 14429 raft_consensus.cc:2272] T 7a5315557f994ec689bb9a79498bb2d5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.606549 14429 tablet_replica.cc:333] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
W20260709 13:21:36.606554 16203 consensus_peers.cc:597] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Couldn't send request to peer 3ef186ec12554a1d87814af0a0ace9a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:36.607327 14429 raft_consensus.cc:2243] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.607788 14429 raft_consensus.cc:2272] T 83022a40fade44a2988f3de3b6b36aec P 3ef186ec12554a1d87814af0a0ace9a4 [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.609571 14429 tablet_replica.cc:333] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.610157 14429 raft_consensus.cc:2243] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:21:36.610957 14429 raft_consensus.cc:2272] T 5243f722506441269fac66aa989ce36d P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.612845 14429 tablet_replica.cc:333] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.613389 14429 raft_consensus.cc:2243] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.613811 14429 raft_consensus.cc:2272] T c1e80bd01b0c4006852da32ca0eeb0ae P 3ef186ec12554a1d87814af0a0ace9a4 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.615629 14429 tablet_replica.cc:333] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.616174 14429 raft_consensus.cc:2243] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:21:36.616995 14429 raft_consensus.cc:2272] T d481434f60dc4276831163b535fb8159 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.618796 14429 tablet_replica.cc:333] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.619331 14429 raft_consensus.cc:2243] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:21:36.620083 14429 raft_consensus.cc:2272] T 6db63c1e295d42d9938e0ef858365436 P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.621902 14429 tablet_replica.cc:333] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.622474 14429 raft_consensus.cc:2243] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.622984 14429 raft_consensus.cc:2272] T 38190179f6494b5591027e1c41f83176 P 3ef186ec12554a1d87814af0a0ace9a4 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.624732 14429 tablet_replica.cc:333] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.625298 14429 raft_consensus.cc:2243] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:21:36.626089 14429 raft_consensus.cc:2272] T b87d5c4641304e03a8a3fd4782b043e5 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.628113 14429 tablet_replica.cc:333] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.628825 14429 raft_consensus.cc:2243] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.629343 14429 raft_consensus.cc:2272] T b9a4c0b115544577a855e9c7383d9a48 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.631420 14429 tablet_replica.cc:333] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.631932 14429 raft_consensus.cc:2243] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.632402 14429 raft_consensus.cc:2272] T 12b56c6369484d92b328d276d4fca9e3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.634297 14429 tablet_replica.cc:333] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.634788 14429 raft_consensus.cc:2243] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:21:36.635563 14429 raft_consensus.cc:2272] T 68dd26481a1b4442b702cb5976b1fb41 P 3ef186ec12554a1d87814af0a0ace9a4 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.637372 14429 tablet_replica.cc:333] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.637859 14429 raft_consensus.cc:2243] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.638350 14429 raft_consensus.cc:2272] T 4da1202d21704a61a455cc91fa7c4022 P 3ef186ec12554a1d87814af0a0ace9a4 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.640025 14429 tablet_replica.cc:333] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.640502 14429 raft_consensus.cc:2243] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.640947 14429 raft_consensus.cc:2272] T 40f71e7bfe1a4575846bcddbb0b9fd55 P 3ef186ec12554a1d87814af0a0ace9a4 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.642937 14429 tablet_replica.cc:333] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.643426 14429 raft_consensus.cc:2243] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.643860 14429 raft_consensus.cc:2272] T a3d461474623405ab8ee3ca7fdff1722 P 3ef186ec12554a1d87814af0a0ace9a4 [term 10 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:36.644264 16203 consensus_peers.cc:597] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Couldn't send request to peer 3ef186ec12554a1d87814af0a0ace9a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:36.645606 14429 tablet_replica.cc:333] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.646204 14429 raft_consensus.cc:2243] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:21:36.646996 14429 raft_consensus.cc:2272] T ab68091b31d248a6ac99ca234a252f77 P 3ef186ec12554a1d87814af0a0ace9a4 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.648708 14429 tablet_replica.cc:333] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.649216 14429 raft_consensus.cc:2243] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:21:36.650352 14429 raft_consensus.cc:2272] T ced58b89726545d3af99de36a1aa1f30 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.652130 14429 tablet_replica.cc:333] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.652629 14429 raft_consensus.cc:2243] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.653141 14429 raft_consensus.cc:2272] T d7b7542687ad43acb128a39bb1f40829 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.655531 14429 tablet_replica.cc:333] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.656206 14429 raft_consensus.cc:2243] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:21:36.657308 14429 raft_consensus.cc:2272] T 4344539a12c94917991c2f39d3ff5858 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.660013 14429 tablet_replica.cc:333] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.660701 14429 raft_consensus.cc:2243] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.661295 14429 raft_consensus.cc:2272] T 4b73eb6a0096455b87f6109b059cf0ee P 3ef186ec12554a1d87814af0a0ace9a4 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:36.662602 16203 consensus_peers.cc:597] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Couldn't send request to peer 3ef186ec12554a1d87814af0a0ace9a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:36.663733 14429 tablet_replica.cc:333] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.664445 14429 raft_consensus.cc:2243] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:21:36.665529 14429 raft_consensus.cc:2272] T 4b9f476ac2ad44cd8124fde4f1aadc78 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.667747 14429 tablet_replica.cc:333] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.668208 14429 raft_consensus.cc:2243] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.668637 14429 raft_consensus.cc:2272] T d4fd164d74054954b750fe9958e66778 P 3ef186ec12554a1d87814af0a0ace9a4 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.670439 14429 tablet_replica.cc:333] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.670858 14429 raft_consensus.cc:2243] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.671331 14429 raft_consensus.cc:2272] T 2a5f6755312b4dc09781ecf8a9468c17 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.672976 14429 tablet_replica.cc:333] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.673386 14429 raft_consensus.cc:2243] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:36.673959 14429 raft_consensus.cc:2272] T 3502bb7169aa41bb9ebdaeddf5068af9 P 3ef186ec12554a1d87814af0a0ace9a4 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.675648 14429 tablet_replica.cc:333] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4: stopping tablet replica
I20260709 13:21:36.676081 14429 raft_consensus.cc:2243] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:21:36.676898 14429 raft_consensus.cc:2272] T 2afa8e112a744f218227526d861b98c3 P 3ef186ec12554a1d87814af0a0ace9a4 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:36.693883 22680 raft_consensus.cc:493] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:36.694550 22680 raft_consensus.cc:515] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:36.696828 22680 leader_election.cc:290] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:21:36.699858 16203 leader_election.cc:336] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:36.700862 16205 leader_election.cc:336] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
I20260709 13:21:36.701301 16205 leader_election.cc:304] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:36.702039 22680 raft_consensus.cc:2749] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:21:36.718065 16203 consensus_peers.cc:597] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Couldn't send request to peer 3ef186ec12554a1d87814af0a0ace9a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:36.726965 22680 raft_consensus.cc:493] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:36.727468 22680 raft_consensus.cc:515] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:36.729210 22680 leader_election.cc:290] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:21:36.733129 16204 leader_election.cc:336] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
W20260709 13:21:36.733626 16205 leader_election.cc:336] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
I20260709 13:21:36.733978 16205 leader_election.cc:304] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:36.734542 22680 raft_consensus.cc:2749] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:21:36.753463 22680 raft_consensus.cc:493] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:36.754005 22680 raft_consensus.cc:515] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:36.755757 22680 leader_election.cc:290] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:21:36.759258 16203 leader_election.cc:336] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Network error: Client connection negotiation failed: client connection to 127.14.23.67:32779: connect: Connection refused (error 111)
W20260709 13:21:36.759698 16205 leader_election.cc:336] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
W20260709 13:21:36.759756 16203 consensus_peers.cc:597] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Couldn't send request to peer 3ef186ec12554a1d87814af0a0ace9a4. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.67:32779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:36.760046 16205 leader_election.cc:304] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:36.760617 22687 raft_consensus.cc:2749] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:36.866044 22687 raft_consensus.cc:493] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:36.866611 22687 raft_consensus.cc:515] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:36.869123 22687 leader_election.cc:290] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:21:36.871786 16204 leader_election.cc:336] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
W20260709 13:21:36.874037 16205 leader_election.cc:336] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
W20260709 13:21:36.873586 16203 consensus_peers.cc:597] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Couldn't send request to peer 3ef186ec12554a1d87814af0a0ace9a4. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.67:32779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:36.874558 16205 leader_election.cc:304] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:36.876019 22687 raft_consensus.cc:2749] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:21:36.881044 15457 debug-util.cc:398] Leaking SignalData structure 0x7b0800177780 after lost signal to thread 14432
W20260709 13:21:36.882045 15457 debug-util.cc:398] Leaking SignalData structure 0x7b08000fade0 after lost signal to thread 15460
W20260709 13:21:36.883036 15457 debug-util.cc:398] Leaking SignalData structure 0x7b08000faee0 after lost signal to thread 15701
W20260709 13:21:36.883921 15457 debug-util.cc:398] Leaking SignalData structure 0x7b08003e9c40 after lost signal to thread 16266
I20260709 13:21:36.898475 22687 raft_consensus.cc:493] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5498ef3e89a941e39c25f68cf64e6b0f)
I20260709 13:21:36.899078 22687 raft_consensus.cc:515] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d3c4f5bd76e0445795bad5f6dd93ff69" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 42481 } } peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:36.903191 22687 leader_election.cc:290] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481), 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593)
W20260709 13:21:36.906631 16204 leader_election.cc:336] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111)
W20260709 13:21:36.908566 16205 leader_election.cc:336] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
I20260709 13:21:36.908996 16205 leader_election.cc:304] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 5498ef3e89a941e39c25f68cf64e6b0f, d3c4f5bd76e0445795bad5f6dd93ff69
I20260709 13:21:36.909690 22679 raft_consensus.cc:2749] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:21:36.974720 16204 consensus_peers.cc:597] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer d3c4f5bd76e0445795bad5f6dd93ff69 (127.14.23.65:42481): Couldn't send request to peer d3c4f5bd76e0445795bad5f6dd93ff69. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:42481: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:37.004778 16203 consensus_peers.cc:597] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Couldn't send request to peer 3ef186ec12554a1d87814af0a0ace9a4. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.67:32779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:37.132211 16205 consensus_peers.cc:597] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:37.430557 16205 consensus_peers.cc:597] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:37.472292 14429 tablet_server.cc:196] TabletServer@127.14.23.67:0 shutdown complete.
I20260709 13:21:37.502025 22680 raft_consensus.cc:493] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ef186ec12554a1d87814af0a0ace9a4)
I20260709 13:21:37.502846 22680 raft_consensus.cc:515] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5498ef3e89a941e39c25f68cf64e6b0f" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41593 } } peers { permanent_uuid: "3ef186ec12554a1d87814af0a0ace9a4" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 32779 } } peers { permanent_uuid: "dd1a14c3173d49dc9b55bb116e3783c9" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 39627 } attrs { promote: false } }
I20260709 13:21:37.505297 22680 leader_election.cc:290] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593), 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779)
W20260709 13:21:37.508304 16205 consensus_peers.cc:597] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:37.511385 16205 leader_election.cc:336] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111)
W20260709 13:21:37.511965 16203 leader_election.cc:336] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3ef186ec12554a1d87814af0a0ace9a4 (127.14.23.67:32779): Network error: Client connection negotiation failed: client connection to 127.14.23.67:32779: connect: Connection refused (error 111)
I20260709 13:21:37.512501 16203 leader_election.cc:304] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd1a14c3173d49dc9b55bb116e3783c9; no voters: 3ef186ec12554a1d87814af0a0ace9a4, 5498ef3e89a941e39c25f68cf64e6b0f
I20260709 13:21:37.513602 22680 raft_consensus.cc:2749] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:37.610448 14429 tablet_server.cc:179] TabletServer@127.14.23.68:0 shutting down...
W20260709 13:21:37.636291 16205 consensus_peers.cc:597] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:37.644860 16205 consensus_peers.cc:597] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:37.653191 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:37.654172 14429 tablet_replica.cc:333] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.655148 14429 raft_consensus.cc:2243] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.655884 14429 raft_consensus.cc:2272] T bd36c2cfd99c45cea9e48096d54bed1a P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.658879 14429 tablet_replica.cc:333] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
W20260709 13:21:37.659382 16205 consensus_peers.cc:597] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 -> Peer 5498ef3e89a941e39c25f68cf64e6b0f (127.14.23.66:41593): Couldn't send request to peer 5498ef3e89a941e39c25f68cf64e6b0f. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.66:41593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:37.659698 14429 raft_consensus.cc:2243] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.660837 14429 raft_consensus.cc:2272] T b99eaeae24b44aa78cfa8c90caca288e P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.663666 14429 tablet_replica.cc:333] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.664505 14429 raft_consensus.cc:2243] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.665158 14429 raft_consensus.cc:2272] T 2afa8e112a744f218227526d861b98c3 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.668236 14429 tablet_replica.cc:333] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.669080 14429 raft_consensus.cc:2243] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.670178 14429 raft_consensus.cc:2272] T 4b73eb6a0096455b87f6109b059cf0ee P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.673134 14429 tablet_replica.cc:333] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.673998 14429 raft_consensus.cc:2243] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.674651 14429 raft_consensus.cc:2272] T b2c57bed49da41d795c8b21ed4461f1f P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.677430 14429 tablet_replica.cc:333] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.678215 14429 raft_consensus.cc:2243] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.679453 14429 raft_consensus.cc:2272] T d6670c20b1954b36a4920b44d1a8209d P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.682101 14429 tablet_replica.cc:333] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.682843 14429 raft_consensus.cc:2243] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.683468 14429 raft_consensus.cc:2272] T e93d4a9ac6e844da87dcd8c6bfa7549b P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.686249 14429 tablet_replica.cc:333] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.687011 14429 raft_consensus.cc:2243] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.687604 14429 raft_consensus.cc:2272] T 7dbabc834ad54c16ae25f79064932f42 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.690517 14429 tablet_replica.cc:333] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.691294 14429 raft_consensus.cc:2243] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.692413 14429 raft_consensus.cc:2272] T 1fa44270fe124591b1e8c82c82b89efe P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.695238 14429 tablet_replica.cc:333] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.696046 14429 raft_consensus.cc:2243] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.697108 14429 raft_consensus.cc:2272] T 83022a40fade44a2988f3de3b6b36aec P dd1a14c3173d49dc9b55bb116e3783c9 [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.699903 14429 tablet_replica.cc:333] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.700697 14429 raft_consensus.cc:2243] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.701303 14429 raft_consensus.cc:2272] T ced58b89726545d3af99de36a1aa1f30 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.704170 14429 tablet_replica.cc:333] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.704985 14429 raft_consensus.cc:2243] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.706025 14429 raft_consensus.cc:2272] T f25e63ed82fa4e008109b74fe8df71ba P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.708591 14429 tablet_replica.cc:333] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.709395 14429 raft_consensus.cc:2243] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.710037 14429 raft_consensus.cc:2272] T 68dd26481a1b4442b702cb5976b1fb41 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.713127 14429 tablet_replica.cc:333] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.713874 14429 raft_consensus.cc:2243] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.714975 14429 raft_consensus.cc:2272] T f9328896aa5a4fdc8fa7c908efdae695 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.717640 14429 tablet_replica.cc:333] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.718403 14429 raft_consensus.cc:2243] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.719455 14429 raft_consensus.cc:2272] T a3d461474623405ab8ee3ca7fdff1722 P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.722126 14429 tablet_replica.cc:333] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.722882 14429 raft_consensus.cc:2243] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.723945 14429 raft_consensus.cc:2272] T 38190179f6494b5591027e1c41f83176 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.726965 14429 tablet_replica.cc:333] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.727808 14429 raft_consensus.cc:2243] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.728439 14429 raft_consensus.cc:2272] T 061ed80a3fca4096b7c13dc0e2256ef0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.731295 14429 tablet_replica.cc:333] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.732026 14429 raft_consensus.cc:2243] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.732682 14429 raft_consensus.cc:2272] T b87d5c4641304e03a8a3fd4782b043e5 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.735298 14429 tablet_replica.cc:333] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.736011 14429 raft_consensus.cc:2243] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.736554 14429 raft_consensus.cc:2272] T ab68091b31d248a6ac99ca234a252f77 P dd1a14c3173d49dc9b55bb116e3783c9 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.739110 14429 tablet_replica.cc:333] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.739811 14429 raft_consensus.cc:2243] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.740414 14429 raft_consensus.cc:2272] T 4c48036212bb4277acf36aac79d8b35e P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.743209 14429 tablet_replica.cc:333] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.744024 14429 raft_consensus.cc:2243] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.744608 14429 raft_consensus.cc:2272] T 0af5649725244ae599dfc6ed7aeac5f1 P dd1a14c3173d49dc9b55bb116e3783c9 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.747270 14429 tablet_replica.cc:333] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.748085 14429 raft_consensus.cc:2243] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.748692 14429 raft_consensus.cc:2272] T 8153c7c094654fb492f5c47863ad170d P dd1a14c3173d49dc9b55bb116e3783c9 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.751473 14429 tablet_replica.cc:333] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.752228 14429 raft_consensus.cc:2243] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.752797 14429 raft_consensus.cc:2272] T 3502bb7169aa41bb9ebdaeddf5068af9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.755507 14429 tablet_replica.cc:333] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.756064 14429 raft_consensus.cc:2243] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.756486 14429 raft_consensus.cc:2272] T d7b7542687ad43acb128a39bb1f40829 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.758436 14429 tablet_replica.cc:333] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.758934 14429 raft_consensus.cc:2243] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.759318 14429 raft_consensus.cc:2272] T 1c436bd48e6b4ad9a1bb596e2ecb15fc P dd1a14c3173d49dc9b55bb116e3783c9 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.761111 14429 tablet_replica.cc:333] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.761636 14429 raft_consensus.cc:2243] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.762439 14429 raft_consensus.cc:2272] T 2a5f6755312b4dc09781ecf8a9468c17 P dd1a14c3173d49dc9b55bb116e3783c9 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.764261 14429 tablet_replica.cc:333] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.764767 14429 raft_consensus.cc:2243] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.765262 14429 raft_consensus.cc:2272] T 4b9f476ac2ad44cd8124fde4f1aadc78 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.767091 14429 tablet_replica.cc:333] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.767558 14429 raft_consensus.cc:2243] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.768327 14429 raft_consensus.cc:2272] T 15e5bd79113c48c5a0b0f265bdd1d7b2 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.770238 14429 tablet_replica.cc:333] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.770761 14429 raft_consensus.cc:2243] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.771175 14429 raft_consensus.cc:2272] T 7b8ed23a60014da39806e750cee98fa6 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.772971 14429 tablet_replica.cc:333] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.773458 14429 raft_consensus.cc:2243] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.774250 14429 raft_consensus.cc:2272] T 0f8365fc4e2f4decb3d554a43e7d4d52 P dd1a14c3173d49dc9b55bb116e3783c9 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.776100 14429 tablet_replica.cc:333] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.776580 14429 raft_consensus.cc:2243] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.777377 14429 raft_consensus.cc:2272] T 40f71e7bfe1a4575846bcddbb0b9fd55 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.779176 14429 tablet_replica.cc:333] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.779659 14429 raft_consensus.cc:2243] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.780372 14429 raft_consensus.cc:2272] T 7562b5f342134e39a5e6934d5ca16e46 P dd1a14c3173d49dc9b55bb116e3783c9 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.782169 14429 tablet_replica.cc:333] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.782671 14429 raft_consensus.cc:2243] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.783393 14429 raft_consensus.cc:2272] T ecb5853a05664246a26d3fc7ecf92777 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.785264 14429 tablet_replica.cc:333] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.785764 14429 raft_consensus.cc:2243] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.786194 14429 raft_consensus.cc:2272] T 01ad9d7b8dba45be8fdf625f27e8d273 P dd1a14c3173d49dc9b55bb116e3783c9 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.787978 14429 tablet_replica.cc:333] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.788457 14429 raft_consensus.cc:2243] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.789232 14429 raft_consensus.cc:2272] T ccb13c2addbf41569d0c2c23750bd4c9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.791143 14429 tablet_replica.cc:333] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.791680 14429 raft_consensus.cc:2243] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.792430 14429 raft_consensus.cc:2272] T 974a060267d24f00888c0942dfbca77e P dd1a14c3173d49dc9b55bb116e3783c9 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.794242 14429 tablet_replica.cc:333] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.794704 14429 raft_consensus.cc:2243] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.795109 14429 raft_consensus.cc:2272] T 271b5d375f0145b78b28a0c13328ce7f P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.796908 14429 tablet_replica.cc:333] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.797361 14429 raft_consensus.cc:2243] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.797756 14429 raft_consensus.cc:2272] T db313a4cd18a47a2b72587d04ed6e747 P dd1a14c3173d49dc9b55bb116e3783c9 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.799834 14429 tablet_replica.cc:333] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.800364 14429 raft_consensus.cc:2243] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.800786 14429 raft_consensus.cc:2272] T 3088d5241ec64ddc919925d5dccfeda0 P dd1a14c3173d49dc9b55bb116e3783c9 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.802608 14429 tablet_replica.cc:333] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.803093 14429 raft_consensus.cc:2243] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.803781 14429 raft_consensus.cc:2272] T b9a4c0b115544577a855e9c7383d9a48 P dd1a14c3173d49dc9b55bb116e3783c9 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.805613 14429 tablet_replica.cc:333] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.806084 14429 raft_consensus.cc:2243] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.806568 14429 raft_consensus.cc:2272] T 6139553a47b64d798a29413921f18396 P dd1a14c3173d49dc9b55bb116e3783c9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.808427 14429 tablet_replica.cc:333] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.808847 14429 raft_consensus.cc:2243] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.809233 14429 raft_consensus.cc:2272] T ef08c91ca56042df89e729f13df83ed9 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.811019 14429 tablet_replica.cc:333] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.811457 14429 raft_consensus.cc:2243] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.812206 14429 raft_consensus.cc:2272] T 582343936346420b87ecf2eebd00dc25 P dd1a14c3173d49dc9b55bb116e3783c9 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.814071 14429 tablet_replica.cc:333] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9: stopping tablet replica
I20260709 13:21:37.814476 14429 raft_consensus.cc:2243] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.814874 14429 raft_consensus.cc:2272] T e67cef93bbdb4ff88a2ee2ac558ba090 P dd1a14c3173d49dc9b55bb116e3783c9 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.163173 14429 tablet_server.cc:196] TabletServer@127.14.23.68:0 shutdown complete.
I20260709 13:21:38.225798 14429 master.cc:562] Master@127.14.23.126:39251 shutting down...
I20260709 13:21:38.254999 14429 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:38.255573 14429 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.255859 14429 tablet_replica.cc:333] T 00000000000000000000000000000000 P 88b9585660f045bda650d7ffa0d7a0b7: stopping tablet replica
I20260709 13:21:38.438629 14429 master.cc:584] Master@127.14.23.126:39251 shutdown complete.
[       OK ] LeaderRebalancerTest.AddTserver (220079 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:21:38.527143 14429 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.23.126:41523
I20260709 13:21:38.528234 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:38.533898 22693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:38.533782 22694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:38.536017 22696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:38.536402 14429 server_base.cc:1061] running on GCE node
I20260709 13:21:38.538782 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:38.539000 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:38.539141 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603298539124 us; error 0 us; skew 500 ppm
I20260709 13:21:38.539796 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:38.542697 14429 webserver.cc:533] Webserver started at http://127.14.23.126:43575/ using document root <none> and password file <none>
I20260709 13:21:38.543217 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:38.543387 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:38.543654 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:38.544775 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/master-0-root/instance:
uuid: "597f4a6dec454e9386f272dfcba9216c"
format_stamp: "Formatted at 2026-07-09 13:21:38 on dist-test-slave-twc1"
I20260709 13:21:38.549305 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.003s
I20260709 13:21:38.552739 22701 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:38.553467 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:21:38.553766 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/master-0-root
uuid: "597f4a6dec454e9386f272dfcba9216c"
format_stamp: "Formatted at 2026-07-09 13:21:38 on dist-test-slave-twc1"
I20260709 13:21:38.554064 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:38.564883 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:38.566133 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:38.604238 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.126:41523
I20260709 13:21:38.604344 22752 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.126:41523 every 8 connection(s)
I20260709 13:21:38.608358 22753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:38.618659 22753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c: Bootstrap starting.
I20260709 13:21:38.623200 22753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.627351 22753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c: No bootstrap required, opened a new log
I20260709 13:21:38.629418 22753 raft_consensus.cc:359] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597f4a6dec454e9386f272dfcba9216c" member_type: VOTER }
I20260709 13:21:38.629796 22753 raft_consensus.cc:385] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:38.630016 22753 raft_consensus.cc:740] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 597f4a6dec454e9386f272dfcba9216c, State: Initialized, Role: FOLLOWER
I20260709 13:21:38.630543 22753 consensus_queue.cc:260] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597f4a6dec454e9386f272dfcba9216c" member_type: VOTER }
I20260709 13:21:38.630955 22753 raft_consensus.cc:399] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:21:38.631138 22753 raft_consensus.cc:493] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:21:38.631353 22753 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:38.635778 22753 raft_consensus.cc:515] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597f4a6dec454e9386f272dfcba9216c" member_type: VOTER }
I20260709 13:21:38.636273 22753 leader_election.cc:304] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 597f4a6dec454e9386f272dfcba9216c; no voters: 
I20260709 13:21:38.637375 22753 leader_election.cc:290] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:21:38.637759 22756 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:38.639117 22756 raft_consensus.cc:697] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c [term 1 LEADER]: Becoming Leader. State: Replica: 597f4a6dec454e9386f272dfcba9216c, State: Running, Role: LEADER
I20260709 13:21:38.639799 22756 consensus_queue.cc:237] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597f4a6dec454e9386f272dfcba9216c" member_type: VOTER }
I20260709 13:21:38.640270 22753 sys_catalog.cc:565] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:21:38.644711 22758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 597f4a6dec454e9386f272dfcba9216c. Latest consensus state: current_term: 1 leader_uuid: "597f4a6dec454e9386f272dfcba9216c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597f4a6dec454e9386f272dfcba9216c" member_type: VOTER } }
I20260709 13:21:38.644788 22757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "597f4a6dec454e9386f272dfcba9216c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597f4a6dec454e9386f272dfcba9216c" member_type: VOTER } }
I20260709 13:21:38.645663 22758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c [sys.catalog]: This master's current role is: LEADER
I20260709 13:21:38.645715 22757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c [sys.catalog]: This master's current role is: LEADER
I20260709 13:21:38.649427 22763 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:21:38.655215 22763 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:21:38.656424 14429 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:21:38.664239 22763 catalog_manager.cc:1383] Generated new cluster ID: 3455cf129c1e41e6b799143a386eb154
I20260709 13:21:38.664552 22763 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:21:38.693655 22763 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:21:38.695580 22763 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:21:38.711222 22763 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c: Generated new TSK 0
I20260709 13:21:38.711921 22763 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:21:38.723281 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:38.729501 22774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:38.730053 22775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:38.733290 14429 server_base.cc:1061] running on GCE node
W20260709 13:21:38.733639 22777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:38.735327 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:38.735549 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:38.735700 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603298735680 us; error 0 us; skew 500 ppm
I20260709 13:21:38.736222 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:38.738643 14429 webserver.cc:533] Webserver started at http://127.14.23.65:36953/ using document root <none> and password file <none>
I20260709 13:21:38.739142 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:38.739333 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:38.739578 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:38.740725 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-0-root/instance:
uuid: "91f33bdbdd63438294071eb3776db679"
format_stamp: "Formatted at 2026-07-09 13:21:38 on dist-test-slave-twc1"
I20260709 13:21:38.745393 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:21:38.748783 22782 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:38.749518 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:21:38.749809 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-0-root
uuid: "91f33bdbdd63438294071eb3776db679"
format_stamp: "Formatted at 2026-07-09 13:21:38 on dist-test-slave-twc1"
I20260709 13:21:38.750141 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:38.759876 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:38.761046 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:38.762544 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:38.764779 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:38.764993 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:38.765221 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:38.765393 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:38.804901 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.65:43155
I20260709 13:21:38.805001 22844 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.65:43155 every 8 connection(s)
I20260709 13:21:38.810029 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:38.818239 22849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:38.818614 22850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:38.826020 22852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:38.826045 14429 server_base.cc:1061] running on GCE node
I20260709 13:21:38.826767 22845 heartbeater.cc:344] Connected to a master server at 127.14.23.126:41523
I20260709 13:21:38.827178 22845 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:38.828055 22845 heartbeater.cc:507] Master 127.14.23.126:41523 requested a full tablet report, sending...
I20260709 13:21:38.829224 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:38.829521 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:38.829717 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603298829699 us; error 0 us; skew 500 ppm
I20260709 13:21:38.830371 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:38.830536 22718 ts_manager.cc:194] Registered new tserver with Master: 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:38.832561 22718 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50708
I20260709 13:21:38.832916 14429 webserver.cc:533] Webserver started at http://127.14.23.66:44215/ using document root <none> and password file <none>
I20260709 13:21:38.833456 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:38.833691 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:38.834029 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:38.835543 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-1-root/instance:
uuid: "49cbf11e18ce4268a7fcca76872afb23"
format_stamp: "Formatted at 2026-07-09 13:21:38 on dist-test-slave-twc1"
I20260709 13:21:38.839949 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 13:21:38.843405 22857 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:38.844244 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:21:38.844514 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-1-root
uuid: "49cbf11e18ce4268a7fcca76872afb23"
format_stamp: "Formatted at 2026-07-09 13:21:38 on dist-test-slave-twc1"
I20260709 13:21:38.844758 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:38.857697 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:38.858922 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:38.860404 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:38.862681 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:38.862876 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:38.863114 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:38.863255 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:38.904387 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.66:45323
I20260709 13:21:38.904498 22919 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.66:45323 every 8 connection(s)
I20260709 13:21:38.909103 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:38.916697 22923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:38.917757 22924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:38.921221 22926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:38.921459 14429 server_base.cc:1061] running on GCE node
I20260709 13:21:38.922053 22920 heartbeater.cc:344] Connected to a master server at 127.14.23.126:41523
I20260709 13:21:38.922511 22920 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:38.922678 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:38.922956 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:38.923153 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603298923136 us; error 0 us; skew 500 ppm
I20260709 13:21:38.923453 22920 heartbeater.cc:507] Master 127.14.23.126:41523 requested a full tablet report, sending...
I20260709 13:21:38.923951 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:38.925782 22718 ts_manager.cc:194] Registered new tserver with Master: 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:38.926679 14429 webserver.cc:533] Webserver started at http://127.14.23.67:42127/ using document root <none> and password file <none>
I20260709 13:21:38.927225 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:38.927403 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:38.927313 22718 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50724
I20260709 13:21:38.927760 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:38.929068 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-2-root/instance:
uuid: "8ef3887fb3874f8b81079e542d252944"
format_stamp: "Formatted at 2026-07-09 13:21:38 on dist-test-slave-twc1"
I20260709 13:21:38.933552 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:21:38.936760 22931 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:38.937502 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:21:38.937791 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-2-root
uuid: "8ef3887fb3874f8b81079e542d252944"
format_stamp: "Formatted at 2026-07-09 13:21:38 on dist-test-slave-twc1"
I20260709 13:21:38.938102 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:38.948618 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:38.949819 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:38.951624 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:38.954124 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:38.954329 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:38.954519 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:38.954651 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:38.994709 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.67:43011
I20260709 13:21:38.994822 22993 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.67:43011 every 8 connection(s)
I20260709 13:21:38.999508 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:39.006841 22997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:39.007694 22998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:39.011054 14429 server_base.cc:1061] running on GCE node
W20260709 13:21:39.012020 23001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:39.012538 22994 heartbeater.cc:344] Connected to a master server at 127.14.23.126:41523
I20260709 13:21:39.012912 22994 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:39.013674 22994 heartbeater.cc:507] Master 127.14.23.126:41523 requested a full tablet report, sending...
I20260709 13:21:39.013732 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:39.014081 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:39.014261 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603299014241 us; error 0 us; skew 500 ppm
I20260709 13:21:39.014963 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:39.016055 22718 ts_manager.cc:194] Registered new tserver with Master: 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:39.017536 22718 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50740
I20260709 13:21:39.018029 14429 webserver.cc:533] Webserver started at http://127.14.23.68:36879/ using document root <none> and password file <none>
I20260709 13:21:39.018671 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:39.018901 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:39.019155 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:39.020262 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-3-root/instance:
uuid: "27745ef236cd494684f62a1269411a68"
format_stamp: "Formatted at 2026-07-09 13:21:39 on dist-test-slave-twc1"
I20260709 13:21:39.024781 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:21:39.028157 23005 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:39.028934 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:21:39.029246 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-3-root
uuid: "27745ef236cd494684f62a1269411a68"
format_stamp: "Formatted at 2026-07-09 13:21:39 on dist-test-slave-twc1"
I20260709 13:21:39.029520 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:39.055452 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:39.056828 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:39.058147 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:39.060431 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:39.060648 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:39.060871 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:39.061023 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:39.101966 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.68:44509
I20260709 13:21:39.102059 23067 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.68:44509 every 8 connection(s)
I20260709 13:21:39.115886 23068 heartbeater.cc:344] Connected to a master server at 127.14.23.126:41523
I20260709 13:21:39.116299 23068 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:39.117070 23068 heartbeater.cc:507] Master 127.14.23.126:41523 requested a full tablet report, sending...
I20260709 13:21:39.119151 22718 ts_manager.cc:194] Registered new tserver with Master: 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:39.120108 14429 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.014936277s
I20260709 13:21:39.120626 22718 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50746
I20260709 13:21:39.147114 22718 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50762:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:21:39.378773 23033 tablet_service.cc:1511] Processing CreateTablet for tablet 02259962b5504debb700dd6b7c12915b (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:21:39.378777 23032 tablet_service.cc:1511] Processing CreateTablet for tablet 084ee2ebbf564ed583f84c1227d29bd4 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:21:39.381896 22959 tablet_service.cc:1511] Processing CreateTablet for tablet 02259962b5504debb700dd6b7c12915b (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:21:39.383320 22959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02259962b5504debb700dd6b7c12915b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.383231 23031 tablet_service.cc:1511] Processing CreateTablet for tablet 086a7123d3ad43d79101a7c5de5158fe (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:21:39.383720 22958 tablet_service.cc:1511] Processing CreateTablet for tablet 7c3139446c554b419fc28cad31653600 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:21:39.380280 23033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02259962b5504debb700dd6b7c12915b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.385106 22958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c3139446c554b419fc28cad31653600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.388892 23030 tablet_service.cc:1511] Processing CreateTablet for tablet 6e09daa4f2dd474d96ae9c4dfa1a8758 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:21:39.390362 23030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e09daa4f2dd474d96ae9c4dfa1a8758. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.394892 23031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 086a7123d3ad43d79101a7c5de5158fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.396049 23029 tablet_service.cc:1511] Processing CreateTablet for tablet cd78248be3954373ad4fd60b8ae7593d (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:21:39.397495 23029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd78248be3954373ad4fd60b8ae7593d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.398507 23032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 084ee2ebbf564ed583f84c1227d29bd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.399231 23028 tablet_service.cc:1511] Processing CreateTablet for tablet 94800e302e4641bb857f2d160f1f60ea (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:21:39.400682 23028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94800e302e4641bb857f2d160f1f60ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.402559 22957 tablet_service.cc:1511] Processing CreateTablet for tablet 086a7123d3ad43d79101a7c5de5158fe (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:21:39.404120 22957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 086a7123d3ad43d79101a7c5de5158fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.407313 23027 tablet_service.cc:1511] Processing CreateTablet for tablet 8f631a34c02c4de8ae5bd70760fba972 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:21:39.409911 23026 tablet_service.cc:1511] Processing CreateTablet for tablet 69e638dc505d4b04b802e91b8863d833 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:21:39.414884 23026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69e638dc505d4b04b802e91b8863d833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.415642 23025 tablet_service.cc:1511] Processing CreateTablet for tablet 0957b803701e45648f538789ec62e507 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:21:39.416706 23027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f631a34c02c4de8ae5bd70760fba972. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.418749 23024 tablet_service.cc:1511] Processing CreateTablet for tablet 03a064be2ec640a4a8283957d0b9b98c (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:21:39.427865 22956 tablet_service.cc:1511] Processing CreateTablet for tablet 5e7b082dcf344fbf83e0ddec4b08b6de (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:21:39.429373 22956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e7b082dcf344fbf83e0ddec4b08b6de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.430187 23025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0957b803701e45648f538789ec62e507. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.430972 23024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03a064be2ec640a4a8283957d0b9b98c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.432209 22955 tablet_service.cc:1511] Processing CreateTablet for tablet cd78248be3954373ad4fd60b8ae7593d (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:21:39.433589 22955 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd78248be3954373ad4fd60b8ae7593d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.438966 22954 tablet_service.cc:1511] Processing CreateTablet for tablet 94800e302e4641bb857f2d160f1f60ea (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:21:39.440704 22954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94800e302e4641bb857f2d160f1f60ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.457526 22952 tablet_service.cc:1511] Processing CreateTablet for tablet 69e638dc505d4b04b802e91b8863d833 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:21:39.459108 22952 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69e638dc505d4b04b802e91b8863d833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.463167 22885 tablet_service.cc:1511] Processing CreateTablet for tablet 02259962b5504debb700dd6b7c12915b (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:21:39.466122 22884 tablet_service.cc:1511] Processing CreateTablet for tablet 7c3139446c554b419fc28cad31653600 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:21:39.467109 22951 tablet_service.cc:1511] Processing CreateTablet for tablet 0957b803701e45648f538789ec62e507 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:21:39.482107 22951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0957b803701e45648f538789ec62e507. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.485206 23027 tablet_service.cc:1511] Processing CreateTablet for tablet 1e4148bbaf3347bb942721170efe1ecc (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:21:39.468855 22950 tablet_service.cc:1511] Processing CreateTablet for tablet 505eab84b06f46ee9d336abf01353dee (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:21:39.469632 22882 tablet_service.cc:1511] Processing CreateTablet for tablet 5e7b082dcf344fbf83e0ddec4b08b6de (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:21:39.470592 22953 tablet_service.cc:1511] Processing CreateTablet for tablet 206c4910f2f9410d9c27ed152cb96087 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:21:39.471367 22881 tablet_service.cc:1511] Processing CreateTablet for tablet 6e09daa4f2dd474d96ae9c4dfa1a8758 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:21:39.473044 22880 tablet_service.cc:1511] Processing CreateTablet for tablet 94800e302e4641bb857f2d160f1f60ea (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:21:39.474757 22879 tablet_service.cc:1511] Processing CreateTablet for tablet 206c4910f2f9410d9c27ed152cb96087 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:21:39.476461 22878 tablet_service.cc:1511] Processing CreateTablet for tablet 8f631a34c02c4de8ae5bd70760fba972 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:21:39.478292 22877 tablet_service.cc:1511] Processing CreateTablet for tablet 0957b803701e45648f538789ec62e507 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:21:39.479991 22876 tablet_service.cc:1511] Processing CreateTablet for tablet 505eab84b06f46ee9d336abf01353dee (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:21:39.464623 22885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02259962b5504debb700dd6b7c12915b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.504884 22807 tablet_service.cc:1511] Processing CreateTablet for tablet 5e7b082dcf344fbf83e0ddec4b08b6de (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:21:39.467935 22883 tablet_service.cc:1511] Processing CreateTablet for tablet 084ee2ebbf564ed583f84c1227d29bd4 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:21:39.515605 22807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e7b082dcf344fbf83e0ddec4b08b6de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.497313 22950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 505eab84b06f46ee9d336abf01353dee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.516448 23030 tablet_service.cc:1511] Processing CreateTablet for tablet 8e230e64c6984b72a2ba543f2a017e25 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:21:39.517966 22953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 206c4910f2f9410d9c27ed152cb96087. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.517622 22802 tablet_service.cc:1511] Processing CreateTablet for tablet 69e638dc505d4b04b802e91b8863d833 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:21:39.519029 22802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69e638dc505d4b04b802e91b8863d833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.520916 22801 tablet_service.cc:1511] Processing CreateTablet for tablet 505eab84b06f46ee9d336abf01353dee (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:21:39.497541 22810 tablet_service.cc:1511] Processing CreateTablet for tablet 7c3139446c554b419fc28cad31653600 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:21:39.522373 22801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 505eab84b06f46ee9d336abf01353dee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.498658 22809 tablet_service.cc:1511] Processing CreateTablet for tablet 084ee2ebbf564ed583f84c1227d29bd4 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:21:39.523463 22809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 084ee2ebbf564ed583f84c1227d29bd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.512173 22803 tablet_service.cc:1511] Processing CreateTablet for tablet 8f631a34c02c4de8ae5bd70760fba972 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:21:39.501727 23027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e4148bbaf3347bb942721170efe1ecc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.503044 22808 tablet_service.cc:1511] Processing CreateTablet for tablet 086a7123d3ad43d79101a7c5de5158fe (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:21:39.506160 22806 tablet_service.cc:1511] Processing CreateTablet for tablet 6e09daa4f2dd474d96ae9c4dfa1a8758 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:21:39.507645 23029 tablet_service.cc:1511] Processing CreateTablet for tablet b154956cc17f4ba68f906daffbcedff6 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:21:39.507838 22805 tablet_service.cc:1511] Processing CreateTablet for tablet cd78248be3954373ad4fd60b8ae7593d (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:21:39.504884 23033 tablet_service.cc:1511] Processing CreateTablet for tablet cabd9adbf414496bbaab3946b413d313 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:21:39.509449 22804 tablet_service.cc:1511] Processing CreateTablet for tablet 206c4910f2f9410d9c27ed152cb96087 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:21:39.513645 22951 tablet_service.cc:1511] Processing CreateTablet for tablet 1e4148bbaf3347bb942721170efe1ecc (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:21:39.515199 22876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 505eab84b06f46ee9d336abf01353dee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.532547 22951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e4148bbaf3347bb942721170efe1ecc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.526400 22810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c3139446c554b419fc28cad31653600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.534852 22805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd78248be3954373ad4fd60b8ae7593d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.535524 22806 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e09daa4f2dd474d96ae9c4dfa1a8758. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.538436 22808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 086a7123d3ad43d79101a7c5de5158fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.542629 22804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 206c4910f2f9410d9c27ed152cb96087. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.547281 22803 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f631a34c02c4de8ae5bd70760fba972. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.550019 22951 tablet_service.cc:1511] Processing CreateTablet for tablet cabd9adbf414496bbaab3946b413d313 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:21:39.551585 22951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cabd9adbf414496bbaab3946b413d313. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.552616 22882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e7b082dcf344fbf83e0ddec4b08b6de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.553328 22877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0957b803701e45648f538789ec62e507. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.553996 22883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 084ee2ebbf564ed583f84c1227d29bd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.554400 23085 tablet_bootstrap.cc:492] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:39.554684 22878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f631a34c02c4de8ae5bd70760fba972. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.557356 22879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 206c4910f2f9410d9c27ed152cb96087. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.558322 22881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e09daa4f2dd474d96ae9c4dfa1a8758. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.560386 22880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94800e302e4641bb857f2d160f1f60ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.571472 22951 tablet_service.cc:1511] Processing CreateTablet for tablet 955a6d8dbbe94ef9be9c1f69b10675d8 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:21:39.573711 22951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 955a6d8dbbe94ef9be9c1f69b10675d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.583038 22958 tablet_service.cc:1511] Processing CreateTablet for tablet 8e230e64c6984b72a2ba543f2a017e25 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:21:39.584439 22958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e230e64c6984b72a2ba543f2a017e25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.591109 22951 tablet_service.cc:1511] Processing CreateTablet for tablet 968e16cef40146509d654274b9fd2687 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:21:39.592504 22951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 968e16cef40146509d654274b9fd2687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.532610 23084 tablet_bootstrap.cc:492] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:39.614398 22884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c3139446c554b419fc28cad31653600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.620200 22951 tablet_service.cc:1511] Processing CreateTablet for tablet 050e9ef11f664c849a40cbb6acb25f19 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:21:39.622123 23031 tablet_service.cc:1511] Processing CreateTablet for tablet 968e16cef40146509d654274b9fd2687 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:21:39.625895 23032 tablet_service.cc:1511] Processing CreateTablet for tablet 56df9023b48a4dc5ae4b437c358860ed (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:21:39.627408 22951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 050e9ef11f664c849a40cbb6acb25f19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.639568 23030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e230e64c6984b72a2ba543f2a017e25. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:21:39.659966 22766 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:39.665458 22877 tablet_service.cc:1511] Processing CreateTablet for tablet 03a064be2ec640a4a8283957d0b9b98c (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:21:39.675033 22877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03a064be2ec640a4a8283957d0b9b98c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.673899 22803 tablet_service.cc:1511] Processing CreateTablet for tablet 1e4148bbaf3347bb942721170efe1ecc (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:21:39.683375 22803 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e4148bbaf3347bb942721170efe1ecc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.710443 23033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cabd9adbf414496bbaab3946b413d313. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.689342 22808 tablet_service.cc:1511] Processing CreateTablet for tablet 03a064be2ec640a4a8283957d0b9b98c (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:21:39.650983 23025 tablet_service.cc:1511] Processing CreateTablet for tablet 6972fb562de24785b0feffe6bcd061c5 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:21:39.681551 22879 tablet_service.cc:1511] Processing CreateTablet for tablet cabd9adbf414496bbaab3946b413d313 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:21:39.726877 22879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cabd9adbf414496bbaab3946b413d313. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.728998 22808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03a064be2ec640a4a8283957d0b9b98c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.732242 22959 tablet_service.cc:1511] Processing CreateTablet for tablet 6972fb562de24785b0feffe6bcd061c5 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:21:39.733717 22959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6972fb562de24785b0feffe6bcd061c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.736938 22951 tablet_service.cc:1511] Processing CreateTablet for tablet 1a4c294dbaeb4040b157ad72102a9e56 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:21:39.738397 22951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a4c294dbaeb4040b157ad72102a9e56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.739667 23032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56df9023b48a4dc5ae4b437c358860ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.742597 22807 tablet_service.cc:1511] Processing CreateTablet for tablet b154956cc17f4ba68f906daffbcedff6 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:21:39.743981 22807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b154956cc17f4ba68f906daffbcedff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.748184 23087 tablet_bootstrap.cc:492] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:39.749238 22806 tablet_service.cc:1511] Processing CreateTablet for tablet 050e9ef11f664c849a40cbb6acb25f19 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:21:39.750262 23025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6972fb562de24785b0feffe6bcd061c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.750610 22806 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 050e9ef11f664c849a40cbb6acb25f19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.752261 23086 tablet_bootstrap.cc:492] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:39.755087 22885 tablet_service.cc:1511] Processing CreateTablet for tablet 955a6d8dbbe94ef9be9c1f69b10675d8 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:21:39.756426 22804 tablet_service.cc:1511] Processing CreateTablet for tablet 955a6d8dbbe94ef9be9c1f69b10675d8 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:21:39.757767 22804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 955a6d8dbbe94ef9be9c1f69b10675d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.758929 23024 tablet_service.cc:1511] Processing CreateTablet for tablet 1a4c294dbaeb4040b157ad72102a9e56 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:21:39.760385 23024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a4c294dbaeb4040b157ad72102a9e56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.762867 23084 tablet_bootstrap.cc:654] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:39.762656 22957 tablet_service.cc:1511] Processing CreateTablet for tablet 98ea5c83aada45d3a98027069e96658c (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:21:39.763967 22957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98ea5c83aada45d3a98027069e96658c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.764153 22878 tablet_service.cc:1511] Processing CreateTablet for tablet b154956cc17f4ba68f906daffbcedff6 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:21:39.767448 22881 tablet_service.cc:1511] Processing CreateTablet for tablet 968e16cef40146509d654274b9fd2687 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:21:39.774277 22805 tablet_service.cc:1511] Processing CreateTablet for tablet 56df9023b48a4dc5ae4b437c358860ed (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:21:39.774718 22881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 968e16cef40146509d654274b9fd2687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.775627 22805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56df9023b48a4dc5ae4b437c358860ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.777473 22885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 955a6d8dbbe94ef9be9c1f69b10675d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.782678 22801 tablet_service.cc:1511] Processing CreateTablet for tablet 6972fb562de24785b0feffe6bcd061c5 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:21:39.783373 22809 tablet_service.cc:1511] Processing CreateTablet for tablet 98ea5c83aada45d3a98027069e96658c (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:21:39.784026 22801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6972fb562de24785b0feffe6bcd061c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.784629 22809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98ea5c83aada45d3a98027069e96658c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.769110 23085 tablet_bootstrap.cc:654] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:39.755681 22810 tablet_service.cc:1511] Processing CreateTablet for tablet 8e230e64c6984b72a2ba543f2a017e25 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:21:39.789065 23028 tablet_service.cc:1511] Processing CreateTablet for tablet 10066ccf8d174f4a9dd69253613857b8 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:21:39.788400 22876 tablet_service.cc:1511] Processing CreateTablet for tablet 050e9ef11f664c849a40cbb6acb25f19 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:21:39.816515 22876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 050e9ef11f664c849a40cbb6acb25f19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.798868 22810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e230e64c6984b72a2ba543f2a017e25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.818624 23086 tablet_bootstrap.cc:654] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:39.819527 22880 tablet_service.cc:1511] Processing CreateTablet for tablet 1a4c294dbaeb4040b157ad72102a9e56 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:21:39.820956 22880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a4c294dbaeb4040b157ad72102a9e56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.792110 23031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 968e16cef40146509d654274b9fd2687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.824554 22950 tablet_service.cc:1511] Processing CreateTablet for tablet c8be856e3d604f8d8a1c56a88ea18643 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:21:39.826186 22950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8be856e3d604f8d8a1c56a88ea18643. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.826714 23029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b154956cc17f4ba68f906daffbcedff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.830179 23086 tablet_bootstrap.cc:492] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:39.830503 23028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10066ccf8d174f4a9dd69253613857b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.830766 23086 ts_tablet_manager.cc:1403] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.079s	user 0.014s	sys 0.000s
I20260709 13:21:39.833747 23026 tablet_service.cc:1511] Processing CreateTablet for tablet c8be856e3d604f8d8a1c56a88ea18643 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:21:39.835158 23026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8be856e3d604f8d8a1c56a88ea18643. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.834764 22803 tablet_service.cc:1511] Processing CreateTablet for tablet c8be856e3d604f8d8a1c56a88ea18643 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:21:39.836264 22803 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8be856e3d604f8d8a1c56a88ea18643. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.838234 22845 heartbeater.cc:499] Master 127.14.23.126:41523 was elected leader, sending a full tablet report...
I20260709 13:21:39.838702 23086 raft_consensus.cc:359] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:39.846154 23086 raft_consensus.cc:385] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:39.846496 23086 raft_consensus.cc:740] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:39.803009 22802 tablet_service.cc:1511] Processing CreateTablet for tablet 10066ccf8d174f4a9dd69253613857b8 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:21:39.851445 22802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10066ccf8d174f4a9dd69253613857b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.852487 22878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b154956cc17f4ba68f906daffbcedff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.853473 23086 consensus_queue.cc:260] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:39.859930 22952 tablet_service.cc:1511] Processing CreateTablet for tablet 89ecf8a5022f43c787aaff1cf7679264 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:21:39.807155 22882 tablet_service.cc:1511] Processing CreateTablet for tablet 56df9023b48a4dc5ae4b437c358860ed (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:21:39.863343 22882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56df9023b48a4dc5ae4b437c358860ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.866720 23087 tablet_bootstrap.cc:654] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:39.869204 23033 tablet_service.cc:1511] Processing CreateTablet for tablet 89ecf8a5022f43c787aaff1cf7679264 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:21:39.870990 22806 tablet_service.cc:1511] Processing CreateTablet for tablet bb2b82eb33bf425d94d0d14dfba05ecb (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:21:39.912293 22806 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb2b82eb33bf425d94d0d14dfba05ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.871966 22883 tablet_service.cc:1511] Processing CreateTablet for tablet 98ea5c83aada45d3a98027069e96658c (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:21:39.877770 22952 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89ecf8a5022f43c787aaff1cf7679264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.918503 22883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98ea5c83aada45d3a98027069e96658c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.932610 22920 heartbeater.cc:499] Master 127.14.23.126:41523 was elected leader, sending a full tablet report...
I20260709 13:21:39.933579 22877 tablet_service.cc:1511] Processing CreateTablet for tablet 10066ccf8d174f4a9dd69253613857b8 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:21:39.934986 22877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10066ccf8d174f4a9dd69253613857b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.953553 22884 tablet_service.cc:1511] Processing CreateTablet for tablet 89ecf8a5022f43c787aaff1cf7679264 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:21:39.954964 22884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89ecf8a5022f43c787aaff1cf7679264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.953557 22885 tablet_service.cc:1511] Processing CreateTablet for tablet bb2b82eb33bf425d94d0d14dfba05ecb (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:21:39.957868 22885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb2b82eb33bf425d94d0d14dfba05ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.912293 23033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89ecf8a5022f43c787aaff1cf7679264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:39.974807 23085 tablet_bootstrap.cc:492] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:39.975405 23085 ts_tablet_manager.cc:1403] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.421s	user 0.012s	sys 0.017s
I20260709 13:21:39.995571 23032 tablet_service.cc:1511] Processing CreateTablet for tablet 536d31f168a54e19aa9aa819e6818d64 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:21:39.998836 23032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 536d31f168a54e19aa9aa819e6818d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.002229 23085 raft_consensus.cc:359] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:40.003060 23085 raft_consensus.cc:385] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:40.003355 23085 raft_consensus.cc:740] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.006753 23086 ts_tablet_manager.cc:1434] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.176s	user 0.014s	sys 0.004s
I20260709 13:21:40.007932 23086 tablet_bootstrap.cc:492] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:40.010304 22957 tablet_service.cc:1511] Processing CreateTablet for tablet bb2b82eb33bf425d94d0d14dfba05ecb (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:21:40.011714 22957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb2b82eb33bf425d94d0d14dfba05ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.018324 23084 tablet_bootstrap.cc:492] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:40.018923 23084 ts_tablet_manager.cc:1403] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.487s	user 0.015s	sys 0.010s
I20260709 13:21:40.020097 22994 heartbeater.cc:499] Master 127.14.23.126:41523 was elected leader, sending a full tablet report...
I20260709 13:21:40.022856 22958 tablet_service.cc:1511] Processing CreateTablet for tablet 745e7e3bed5b4d59949aeddcede543f3 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:21:40.021189 22959 tablet_service.cc:1511] Processing CreateTablet for tablet 50b783c62bd54d4c8f4f1795b7409958 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:21:40.024405 22958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 745e7e3bed5b4d59949aeddcede543f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.026161 23084 raft_consensus.cc:359] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:40.026932 23084 raft_consensus.cc:385] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:40.027220 23084 raft_consensus.cc:740] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.028034 23084 consensus_queue.cc:260] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:40.029771 22807 tablet_service.cc:1511] Processing CreateTablet for tablet 536d31f168a54e19aa9aa819e6818d64 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:21:40.029779 22804 tablet_service.cc:1511] Processing CreateTablet for tablet 50b783c62bd54d4c8f4f1795b7409958 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:21:40.031447 22959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50b783c62bd54d4c8f4f1795b7409958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.032894 22809 tablet_service.cc:1511] Processing CreateTablet for tablet 9ecffc3cb18a4a6eb87a67130c728681 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:21:40.004104 23085 consensus_queue.cc:260] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:40.044080 22804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50b783c62bd54d4c8f4f1795b7409958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.045264 23087 tablet_bootstrap.cc:492] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:40.050454 22809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ecffc3cb18a4a6eb87a67130c728681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.045861 23087 ts_tablet_manager.cc:1403] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.298s	user 0.025s	sys 0.007s
I20260709 13:21:40.050944 23029 tablet_service.cc:1511] Processing CreateTablet for tablet 50b783c62bd54d4c8f4f1795b7409958 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
W20260709 13:21:40.060181 22846 tablet.cc:2406] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:21:40.069068 22953 tablet_service.cc:1511] Processing CreateTablet for tablet 9ecffc3cb18a4a6eb87a67130c728681 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:21:40.069437 23025 tablet_service.cc:1511] Processing CreateTablet for tablet be220d19020b428a8b1cc356f32ff5ad (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:21:40.066857 23029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50b783c62bd54d4c8f4f1795b7409958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.070506 22953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ecffc3cb18a4a6eb87a67130c728681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.079109 23025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be220d19020b428a8b1cc356f32ff5ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.084950 23086 tablet_bootstrap.cc:654] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:40.085522 22802 tablet_service.cc:1511] Processing CreateTablet for tablet be220d19020b428a8b1cc356f32ff5ad (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:21:40.087694 23085 ts_tablet_manager.cc:1434] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.112s	user 0.007s	sys 0.004s
I20260709 13:21:40.095374 22802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be220d19020b428a8b1cc356f32ff5ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.123073 22807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 536d31f168a54e19aa9aa819e6818d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.128607 23031 tablet_service.cc:1511] Processing CreateTablet for tablet 745e7e3bed5b4d59949aeddcede543f3 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:21:40.095162 22956 tablet_service.cc:1511] Processing CreateTablet for tablet 39611d2ff3524bc0840e4fd222d96a84 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:21:40.097931 23026 tablet_service.cc:1511] Processing CreateTablet for tablet 39611d2ff3524bc0840e4fd222d96a84 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:21:40.102821 23068 heartbeater.cc:499] Master 127.14.23.126:41523 was elected leader, sending a full tablet report...
I20260709 13:21:40.130959 22956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39611d2ff3524bc0840e4fd222d96a84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.114980 23084 ts_tablet_manager.cc:1434] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.096s	user 0.012s	sys 0.010s
I20260709 13:21:40.133764 23084 tablet_bootstrap.cc:492] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:40.129786 22884 tablet_service.cc:1511] Processing CreateTablet for tablet 745e7e3bed5b4d59949aeddcede543f3 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:21:40.135304 22884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 745e7e3bed5b4d59949aeddcede543f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.093963 23087 raft_consensus.cc:359] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:40.139609 23087 raft_consensus.cc:385] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:40.139968 22879 tablet_service.cc:1511] Processing CreateTablet for tablet 536d31f168a54e19aa9aa819e6818d64 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:21:40.141388 22879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 536d31f168a54e19aa9aa819e6818d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.146297 22882 tablet_service.cc:1511] Processing CreateTablet for tablet 9ecffc3cb18a4a6eb87a67130c728681 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:21:40.147358 23026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39611d2ff3524bc0840e4fd222d96a84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.147599 22882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ecffc3cb18a4a6eb87a67130c728681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.164685 22877 tablet_service.cc:1511] Processing CreateTablet for tablet be220d19020b428a8b1cc356f32ff5ad (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:21:40.172331 22877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be220d19020b428a8b1cc356f32ff5ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.175273 23085 tablet_bootstrap.cc:492] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:40.178498 23031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 745e7e3bed5b4d59949aeddcede543f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.140007 23087 raft_consensus.cc:740] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.182853 23087 consensus_queue.cc:260] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:40.185343 23085 tablet_bootstrap.cc:654] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:40.206244 22809 tablet_service.cc:1511] Processing CreateTablet for tablet 39611d2ff3524bc0840e4fd222d96a84 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:21:40.206694 22881 tablet_service.cc:1511] Processing CreateTablet for tablet 171526d2fe264ee8b1168192bec31c7a (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:21:40.207890 22809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39611d2ff3524bc0840e4fd222d96a84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.208055 22881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 171526d2fe264ee8b1168192bec31c7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.217790 22959 tablet_service.cc:1511] Processing CreateTablet for tablet 171526d2fe264ee8b1168192bec31c7a (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:21:40.219461 22959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 171526d2fe264ee8b1168192bec31c7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.220710 23087 ts_tablet_manager.cc:1434] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.169s	user 0.009s	sys 0.002s
I20260709 13:21:40.227679 23087 tablet_bootstrap.cc:492] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:40.224035 23084 tablet_bootstrap.cc:654] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:40.232107 23086 tablet_bootstrap.cc:492] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:40.232689 23086 ts_tablet_manager.cc:1403] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.225s	user 0.030s	sys 0.035s
I20260709 13:21:40.233950 23094 raft_consensus.cc:493] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.234469 23094 raft_consensus.cc:515] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:40.298887 23086 raft_consensus.cc:359] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
W20260709 13:21:40.306242 22702 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 70460.2us
W20260709 13:21:40.309388 22704 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 64124.4us
I20260709 13:21:40.300698 23094 leader_election.cc:290] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:40.306159 23086 raft_consensus.cc:385] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:40.314661 23085 tablet_bootstrap.cc:492] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:40.320617 23086 raft_consensus.cc:740] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.320940 23085 ts_tablet_manager.cc:1403] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.146s	user 0.020s	sys 0.011s
I20260709 13:21:40.321647 23086 consensus_queue.cc:260] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:40.323830 23086 ts_tablet_manager.cc:1434] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.091s	user 0.015s	sys 0.015s
I20260709 13:21:40.325727 23086 tablet_bootstrap.cc:492] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:40.326540 22885 tablet_service.cc:1511] Processing CreateTablet for tablet 087d64074b5943f0a7d789ead06f4783 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:21:40.327960 22885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 087d64074b5943f0a7d789ead06f4783. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.329617 23091 raft_consensus.cc:493] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.329411 23085 raft_consensus.cc:359] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:40.334120 23085 raft_consensus.cc:385] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:40.334424 23085 raft_consensus.cc:740] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.335151 23085 consensus_queue.cc:260] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:40.336089 23091 raft_consensus.cc:515] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:40.330649 22807 tablet_service.cc:1511] Processing CreateTablet for tablet 087d64074b5943f0a7d789ead06f4783 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:21:40.337347 23085 ts_tablet_manager.cc:1434] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 13:21:40.338361 23085 tablet_bootstrap.cc:492] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:40.337484 22807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 087d64074b5943f0a7d789ead06f4783. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.335155 22959 tablet_service.cc:1511] Processing CreateTablet for tablet 087d64074b5943f0a7d789ead06f4783 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:21:40.339426 23087 tablet_bootstrap.cc:654] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:40.335930 23086 tablet_bootstrap.cc:654] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:40.351756 23084 tablet_bootstrap.cc:492] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:40.352325 23084 ts_tablet_manager.cc:1403] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.219s	user 0.010s	sys 0.009s
I20260709 13:21:40.352393 23091 leader_election.cc:290] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:40.354425 22959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 087d64074b5943f0a7d789ead06f4783. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.355557 23084 raft_consensus.cc:359] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:40.356418 23084 raft_consensus.cc:385] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:40.356719 23084 raft_consensus.cc:740] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.357753 23084 consensus_queue.cc:260] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:40.361690 23084 ts_tablet_manager.cc:1434] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:21:40.362776 23084 tablet_bootstrap.cc:492] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:40.363345 22809 tablet_service.cc:1511] Processing CreateTablet for tablet d1934129c8764c018495eff97d43a3b5 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:21:40.364691 22809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1934129c8764c018495eff97d43a3b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.367683 22881 tablet_service.cc:1511] Processing CreateTablet for tablet d1934129c8764c018495eff97d43a3b5 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:21:40.368364 22957 tablet_service.cc:1511] Processing CreateTablet for tablet c005b0f2343140f68e3069174596c1ad (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:21:40.369050 22881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1934129c8764c018495eff97d43a3b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.369655 22957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c005b0f2343140f68e3069174596c1ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.382004 23085 tablet_bootstrap.cc:654] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:40.384946 23093 raft_consensus.cc:493] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.385502 23093 raft_consensus.cc:515] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:40.386329 22805 tablet_service.cc:1511] Processing CreateTablet for tablet c005b0f2343140f68e3069174596c1ad (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:21:40.387766 22805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c005b0f2343140f68e3069174596c1ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.387876 22884 tablet_service.cc:1511] Processing CreateTablet for tablet 44841295bfd54159a48e285582824bad (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:21:40.388203 23093 leader_election.cc:290] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:40.389449 22884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44841295bfd54159a48e285582824bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.394829 23086 tablet_bootstrap.cc:492] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:40.395476 23086 ts_tablet_manager.cc:1403] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.070s	user 0.012s	sys 0.002s
I20260709 13:21:40.396754 23096 raft_consensus.cc:493] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.397585 23087 tablet_bootstrap.cc:492] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:40.397298 23096 raft_consensus.cc:515] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:40.398168 23087 ts_tablet_manager.cc:1403] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.171s	user 0.012s	sys 0.004s
I20260709 13:21:40.400044 23096 leader_election.cc:290] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:40.411832 23084 tablet_bootstrap.cc:654] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:40.437005 23085 tablet_bootstrap.cc:492] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:40.437606 23085 ts_tablet_manager.cc:1403] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.099s	user 0.013s	sys 0.020s
I20260709 13:21:40.437950 22802 tablet_service.cc:1511] Processing CreateTablet for tablet 7802e4d0fd394596b42b661c9eb4c7ce (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:21:40.439301 22802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7802e4d0fd394596b42b661c9eb4c7ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.440866 23086 raft_consensus.cc:359] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:40.441709 23086 raft_consensus.cc:385] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:40.442039 23086 raft_consensus.cc:740] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.442782 23086 consensus_queue.cc:260] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:40.432320 23087 raft_consensus.cc:359] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:40.452553 23087 raft_consensus.cc:385] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:40.478844 23086 ts_tablet_manager.cc:1434] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.083s	user 0.008s	sys 0.001s
I20260709 13:21:40.471153 23087 raft_consensus.cc:740] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.462090 22804 tablet_service.cc:1511] Processing CreateTablet for tablet 21e1e18beffe450f87ff940a4e9165ea (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:21:40.494638 23086 tablet_bootstrap.cc:492] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:40.495613 22880 tablet_service.cc:1511] Processing CreateTablet for tablet 7802e4d0fd394596b42b661c9eb4c7ce (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:21:40.496194 22879 tablet_service.cc:1511] Processing CreateTablet for tablet 21e1e18beffe450f87ff940a4e9165ea (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:21:40.502031 23087 consensus_queue.cc:260] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:40.502568 23085 raft_consensus.cc:359] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:40.509150 22876 tablet_service.cc:1511] Processing CreateTablet for tablet 78234c90229d4615ab9e2c2a811bd0f8 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:21:40.526722 22879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21e1e18beffe450f87ff940a4e9165ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.509105 22878 tablet_service.cc:1511] Processing CreateTablet for tablet a28cd6c417f74c9ea3a4172ad8f3c5e6 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:21:40.505112 22803 tablet_service.cc:1511] Processing CreateTablet for tablet a28cd6c417f74c9ea3a4172ad8f3c5e6 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:21:40.540333 23087 ts_tablet_manager.cc:1434] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.142s	user 0.011s	sys 0.002s
I20260709 13:21:40.508018 22885 tablet_service.cc:1511] Processing CreateTablet for tablet 8ad9b8807851437d9a22c0a33f567c15 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:21:40.542410 22806 tablet_service.cc:1511] Processing CreateTablet for tablet 78234c90229d4615ab9e2c2a811bd0f8 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:21:40.543551 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02259962b5504debb700dd6b7c12915b" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:40.544504 22881 tablet_service.cc:1511] Processing CreateTablet for tablet ee6a9c5a5fdc42e78dbbe6b411416c1a (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:21:40.547730 22876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78234c90229d4615ab9e2c2a811bd0f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.549748 22880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7802e4d0fd394596b42b661c9eb4c7ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.506281 22804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21e1e18beffe450f87ff940a4e9165ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.554605 22878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a28cd6c417f74c9ea3a4172ad8f3c5e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.564472 23031 tablet_service.cc:1511] Processing CreateTablet for tablet 171526d2fe264ee8b1168192bec31c7a (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:21:40.505573 22810 tablet_service.cc:1511] Processing CreateTablet for tablet abacd5d0d1e54db38a397e3c35c0cb81 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:21:40.565840 23031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 171526d2fe264ee8b1168192bec31c7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.541404 22801 tablet_service.cc:1511] Processing CreateTablet for tablet 49b752c38e1b47a489a3d720801661d1 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:21:40.531137 23085 raft_consensus.cc:385] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:40.566715 23085 raft_consensus.cc:740] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.567518 23085 consensus_queue.cc:260] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:40.569139 23043 raft_consensus.cc:2468] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 0.
I20260709 13:21:40.574358 22858 leader_election.cc:304] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:40.575553 23096 raft_consensus.cc:2804] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:40.575946 23096 raft_consensus.cc:493] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:40.576274 23096 raft_consensus.cc:3060] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:40.541448 23087 tablet_bootstrap.cc:492] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:40.566375 22806 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78234c90229d4615ab9e2c2a811bd0f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.582484 22810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abacd5d0d1e54db38a397e3c35c0cb81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.584033 23030 tablet_service.cc:1511] Processing CreateTablet for tablet d1934129c8764c018495eff97d43a3b5 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:21:40.585453 23030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1934129c8764c018495eff97d43a3b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.585685 22801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49b752c38e1b47a489a3d720801661d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.597625 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:40.599922 22803 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a28cd6c417f74c9ea3a4172ad8f3c5e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.600180 23096 raft_consensus.cc:515] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:40.601292 22809 tablet_service.cc:1511] Processing CreateTablet for tablet ac141e204e3f4d3c84d6651893bdfc6f (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:21:40.602662 22809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac141e204e3f4d3c84d6651893bdfc6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.603159 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02259962b5504debb700dd6b7c12915b" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:40.603812 23043 raft_consensus.cc:3060] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:40.606837 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:40.609505 23026 tablet_service.cc:1511] Processing CreateTablet for tablet c005b0f2343140f68e3069174596c1ad (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:21:40.610872 23026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c005b0f2343140f68e3069174596c1ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.618213 23085 ts_tablet_manager.cc:1434] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.180s	user 0.009s	sys 0.003s
I20260709 13:21:40.619454 23029 tablet_service.cc:1511] Processing CreateTablet for tablet 44841295bfd54159a48e285582824bad (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
W20260709 13:21:40.620591 22785 leader_election.cc:343] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.620836 23029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44841295bfd54159a48e285582824bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.621675 23096 leader_election.cc:290] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:40.624573 23043 raft_consensus.cc:2468] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:40.621024 22879 tablet_service.cc:1511] Processing CreateTablet for tablet ac141e204e3f4d3c84d6651893bdfc6f (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:21:40.619397 23085 tablet_bootstrap.cc:492] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:40.626739 22858 leader_election.cc:304] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:40.630931 22881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee6a9c5a5fdc42e78dbbe6b411416c1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.638100 23085 tablet_bootstrap.cc:654] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:40.630591 22785 leader_election.cc:343] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.638319 23093 raft_consensus.cc:493] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.638772 22785 leader_election.cc:304] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:40.638902 23093 raft_consensus.cc:515] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:40.641371 23093 leader_election.cc:290] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:40.641953 23091 raft_consensus.cc:2749] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:40.643363 22885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ad9b8807851437d9a22c0a33f567c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.645814 22802 tablet_service.cc:1511] Processing CreateTablet for tablet 630a5e60917545b3a8072fc8358207c5 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:21:40.647287 22802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 630a5e60917545b3a8072fc8358207c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.647480 23087 tablet_bootstrap.cc:654] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:40.638319 23086 tablet_bootstrap.cc:654] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:40.654112 23096 raft_consensus.cc:2804] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:40.659354 22884 tablet_service.cc:1511] Processing CreateTablet for tablet 630a5e60917545b3a8072fc8358207c5 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
W20260709 13:21:40.662672 22766 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:40.662703 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02259962b5504debb700dd6b7c12915b" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:40.663679 23043 raft_consensus.cc:2393] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 in current term 1: Already voted for candidate 49cbf11e18ce4268a7fcca76872afb23 in this term.
I20260709 13:21:40.669523 22807 tablet_service.cc:1511] Processing CreateTablet for tablet 0e55b4e727be4089ba7454b662d03b9c (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:21:40.672719 22807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e55b4e727be4089ba7454b662d03b9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.673777 22878 tablet_service.cc:1511] Processing CreateTablet for tablet b74a5704470044c1bb9cd7c18e3b6000 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:21:40.681243 23031 tablet_service.cc:1511] Processing CreateTablet for tablet 7802e4d0fd394596b42b661c9eb4c7ce (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:21:40.682679 23031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7802e4d0fd394596b42b661c9eb4c7ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.687923 22801 tablet_service.cc:1511] Processing CreateTablet for tablet 989cd1a042864c419d8c10665cb8afd0 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:21:40.689352 23096 raft_consensus.cc:697] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:40.691601 23084 tablet_bootstrap.cc:492] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:40.692977 23084 ts_tablet_manager.cc:1403] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.330s	user 0.007s	sys 0.019s
I20260709 13:21:40.693230 23096 consensus_queue.cc:237] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:40.691792 22876 tablet_service.cc:1511] Processing CreateTablet for tablet 989cd1a042864c419d8c10665cb8afd0 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:21:40.674134 22879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac141e204e3f4d3c84d6651893bdfc6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.697288 23033 tablet_service.cc:1511] Processing CreateTablet for tablet abacd5d0d1e54db38a397e3c35c0cb81 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:21:40.701725 23033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abacd5d0d1e54db38a397e3c35c0cb81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.701838 22959 tablet_service.cc:1511] Processing CreateTablet for tablet 44841295bfd54159a48e285582824bad (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:21:40.703238 22959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44841295bfd54159a48e285582824bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.705943 23024 tablet_service.cc:1511] Processing CreateTablet for tablet 8ad9b8807851437d9a22c0a33f567c15 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:21:40.707283 23024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ad9b8807851437d9a22c0a33f567c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.706945 23084 raft_consensus.cc:359] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:40.708611 23084 raft_consensus.cc:385] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:40.709204 23084 raft_consensus.cc:740] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.709376 23025 tablet_service.cc:1511] Processing CreateTablet for tablet 78234c90229d4615ab9e2c2a811bd0f8 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:21:40.710814 23025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78234c90229d4615ab9e2c2a811bd0f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.718425 22884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 630a5e60917545b3a8072fc8358207c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.719317 22801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 989cd1a042864c419d8c10665cb8afd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.722473 22957 tablet_service.cc:1511] Processing CreateTablet for tablet 21e1e18beffe450f87ff940a4e9165ea (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:21:40.722854 22956 tablet_service.cc:1511] Processing CreateTablet for tablet abacd5d0d1e54db38a397e3c35c0cb81 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:21:40.723943 22957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21e1e18beffe450f87ff940a4e9165ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.727708 22878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b74a5704470044c1bb9cd7c18e3b6000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.728139 22956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abacd5d0d1e54db38a397e3c35c0cb81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.728456 23032 tablet_service.cc:1511] Processing CreateTablet for tablet 49b752c38e1b47a489a3d720801661d1 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:21:40.726188 22958 tablet_service.cc:1511] Processing CreateTablet for tablet 8ad9b8807851437d9a22c0a33f567c15 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:21:40.729717 23032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49b752c38e1b47a489a3d720801661d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.730562 22958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ad9b8807851437d9a22c0a33f567c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.710758 23084 consensus_queue.cc:260] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:40.746097 22803 tablet_service.cc:1511] Processing CreateTablet for tablet 014651e80f9f44e599d2d8e4d308d2a6 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:21:40.747560 22803 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 014651e80f9f44e599d2d8e4d308d2a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.750988 22880 tablet_service.cc:1511] Processing CreateTablet for tablet 014651e80f9f44e599d2d8e4d308d2a6 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:21:40.752002 23085 tablet_bootstrap.cc:492] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:40.752543 23085 ts_tablet_manager.cc:1403] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.133s	user 0.013s	sys 0.003s
I20260709 13:21:40.753857 23028 tablet_service.cc:1511] Processing CreateTablet for tablet 630a5e60917545b3a8072fc8358207c5 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:21:40.755196 23028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 630a5e60917545b3a8072fc8358207c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.756273 23084 ts_tablet_manager.cc:1434] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.063s	user 0.009s	sys 0.000s
I20260709 13:21:40.757299 23084 tablet_bootstrap.cc:492] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:40.758206 23027 tablet_service.cc:1511] Processing CreateTablet for tablet 0e55b4e727be4089ba7454b662d03b9c (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:21:40.757324 22876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 989cd1a042864c419d8c10665cb8afd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.761488 23027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e55b4e727be4089ba7454b662d03b9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.767645 23086 tablet_bootstrap.cc:492] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:40.768198 23086 ts_tablet_manager.cc:1403] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.274s	user 0.005s	sys 0.021s
I20260709 13:21:40.771277 23086 raft_consensus.cc:359] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:40.772002 23086 raft_consensus.cc:385] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:40.772208 22880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 014651e80f9f44e599d2d8e4d308d2a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.772302 23086 raft_consensus.cc:740] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.776525 23086 consensus_queue.cc:260] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:40.779160 22953 tablet_service.cc:1511] Processing CreateTablet for tablet a28cd6c417f74c9ea3a4172ad8f3c5e6 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:21:40.780546 22953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a28cd6c417f74c9ea3a4172ad8f3c5e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.780951 22950 tablet_service.cc:1511] Processing CreateTablet for tablet 49b752c38e1b47a489a3d720801661d1 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:21:40.781710 22954 tablet_service.cc:1511] Processing CreateTablet for tablet b74a5704470044c1bb9cd7c18e3b6000 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:21:40.780172 22951 tablet_service.cc:1511] Processing CreateTablet for tablet ac141e204e3f4d3c84d6651893bdfc6f (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:21:40.782502 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3139446c554b419fc28cad31653600" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:40.782207 22950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49b752c38e1b47a489a3d720801661d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.783373 22952 tablet_service.cc:1511] Processing CreateTablet for tablet ee6a9c5a5fdc42e78dbbe6b411416c1a (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:21:40.784600 22952 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee6a9c5a5fdc42e78dbbe6b411416c1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.786245 22951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac141e204e3f4d3c84d6651893bdfc6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.786926 22955 tablet_service.cc:1511] Processing CreateTablet for tablet 0e55b4e727be4089ba7454b662d03b9c (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:21:40.788195 22955 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e55b4e727be4089ba7454b662d03b9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.789446 22954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b74a5704470044c1bb9cd7c18e3b6000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.792057 23087 tablet_bootstrap.cc:492] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:40.792608 23087 ts_tablet_manager.cc:1403] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.251s	user 0.015s	sys 0.000s
I20260709 13:21:40.794345 23091 raft_consensus.cc:493] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.794870 23091 raft_consensus.cc:515] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:40.797282 23091 leader_election.cc:290] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:40.799696 23085 raft_consensus.cc:359] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:40.800522 23085 raft_consensus.cc:385] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:40.800817 23085 raft_consensus.cc:740] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.802896 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f631a34c02c4de8ae5bd70760fba972" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
W20260709 13:21:40.804683 22785 leader_election.cc:343] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.808913 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02259962b5504debb700dd6b7c12915b" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:40.809674 22969 raft_consensus.cc:2468] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 0.
I20260709 13:21:40.810899 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3139446c554b419fc28cad31653600" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:40.812813 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02259962b5504debb700dd6b7c12915b" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:40.819983 23084 tablet_bootstrap.cc:654] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:40.819711 23087 raft_consensus.cc:359] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:40.821012 23087 raft_consensus.cc:385] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:40.821426 23087 raft_consensus.cc:740] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.822387 23087 consensus_queue.cc:260] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:40.822849 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02259962b5504debb700dd6b7c12915b" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:40.823592 22968 raft_consensus.cc:3060] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:40.825261 23087 ts_tablet_manager.cc:1434] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 13:21:40.824352 23030 tablet_service.cc:1511] Processing CreateTablet for tablet ee6a9c5a5fdc42e78dbbe6b411416c1a (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:21:40.831681 23026 tablet_service.cc:1511] Processing CreateTablet for tablet 014651e80f9f44e599d2d8e4d308d2a6 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
W20260709 13:21:40.843931 22934 leader_election.cc:343] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.845778 22968 raft_consensus.cc:2468] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:40.848685 22957 tablet_service.cc:1511] Processing CreateTablet for tablet 9d8b119f5c5844d1be439423b333ec60 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:21:40.779023 23086 ts_tablet_manager.cc:1434] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 13:21:40.865819 22957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d8b119f5c5844d1be439423b333ec60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.866513 23030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee6a9c5a5fdc42e78dbbe6b411416c1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.871662 23026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 014651e80f9f44e599d2d8e4d308d2a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.874871 23086 tablet_bootstrap.cc:492] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:40.853250 22802 tablet_service.cc:1511] Processing CreateTablet for tablet 9d8b119f5c5844d1be439423b333ec60 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:21:40.876094 22802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d8b119f5c5844d1be439423b333ec60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.855660 22885 tablet_service.cc:1511] Processing CreateTablet for tablet d217f95123f542e886aa049a608a88d4 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:21:40.858523 22959 tablet_service.cc:1511] Processing CreateTablet for tablet 989cd1a042864c419d8c10665cb8afd0 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:21:40.897969 23094 raft_consensus.cc:493] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.801542 23085 consensus_queue.cc:260] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:40.882656 23087 tablet_bootstrap.cc:492] T 8f631a34c02c4de8ae5bd70760fba972 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:40.886336 22934 leader_election.cc:304] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:40.888010 23121 raft_consensus.cc:493] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.890154 22807 tablet_service.cc:1511] Processing CreateTablet for tablet b1394978547a410199b4f798a10edb2d (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:21:40.899755 23094 raft_consensus.cc:515] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:40.908221 23093 raft_consensus.cc:2749] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:40.908905 22878 tablet_service.cc:1511] Processing CreateTablet for tablet b1394978547a410199b4f798a10edb2d (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:21:40.904583 23121 raft_consensus.cc:515] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:40.895102 23024 tablet_service.cc:1511] Processing CreateTablet for tablet b74a5704470044c1bb9cd7c18e3b6000 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:21:40.913492 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd78248be3954373ad4fd60b8ae7593d" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:40.915683 23094 leader_election.cc:290] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:40.924041 22807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1394978547a410199b4f798a10edb2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.930137 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd78248be3954373ad4fd60b8ae7593d" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:40.916288 23084 tablet_bootstrap.cc:492] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:40.917526 23121 leader_election.cc:290] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:40.918215 22878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1394978547a410199b4f798a10edb2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.936328 22885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d217f95123f542e886aa049a608a88d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.920756 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0957b803701e45648f538789ec62e507" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
W20260709 13:21:40.938220 22858 leader_election.cc:343] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.922920 23024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b74a5704470044c1bb9cd7c18e3b6000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.952621 23027 tablet_service.cc:1511] Processing CreateTablet for tablet 9d8b119f5c5844d1be439423b333ec60 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:21:40.952968 23029 tablet_service.cc:1511] Processing CreateTablet for tablet d217f95123f542e886aa049a608a88d4 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:21:40.954157 23027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d8b119f5c5844d1be439423b333ec60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.954845 23029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d217f95123f542e886aa049a608a88d4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:21:40.959129 23008 leader_election.cc:343] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:40.959925 23008 leader_election.cc:343] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.960376 23008 leader_election.cc:304] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:40.960930 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f631a34c02c4de8ae5bd70760fba972" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:40.961395 23084 ts_tablet_manager.cc:1403] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.204s	user 0.011s	sys 0.003s
I20260709 13:21:40.919353 22959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 989cd1a042864c419d8c10665cb8afd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:40.964659 23043 raft_consensus.cc:2468] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
I20260709 13:21:40.965838 23091 raft_consensus.cc:493] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.966349 23091 raft_consensus.cc:515] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:40.963901 23094 raft_consensus.cc:2749] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:40.965700 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f631a34c02c4de8ae5bd70760fba972" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:40.969069 22820 raft_consensus.cc:2468] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 0.
I20260709 13:21:40.969200 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e7b082dcf344fbf83e0ddec4b08b6de" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:40.968513 23091 leader_election.cc:290] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
W20260709 13:21:40.971459 22785 leader_election.cc:343] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.972373 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e7b082dcf344fbf83e0ddec4b08b6de" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
W20260709 13:21:40.973762 22785 leader_election.cc:343] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.974224 22785 leader_election.cc:304] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:40.974939 23091 raft_consensus.cc:2749] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:21:40.963797 22934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 77588.1us
I20260709 13:21:40.978518 23084 raft_consensus.cc:359] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:40.979360 23084 raft_consensus.cc:385] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:40.979681 23084 raft_consensus.cc:740] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.980513 23084 consensus_queue.cc:260] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:40.981221 22783 leader_election.cc:304] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:40.980924 22716 catalog_manager.cc:5697] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:40.982719 23091 raft_consensus.cc:2804] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:40.983182 23091 raft_consensus.cc:493] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:40.983556 23091 raft_consensus.cc:3060] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:40.982813 23084 ts_tablet_manager.cc:1434] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.003s
I20260709 13:21:40.990779 23084 tablet_bootstrap.cc:492] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:40.992241 23087 tablet_bootstrap.cc:654] T 8f631a34c02c4de8ae5bd70760fba972 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:40.992151 23027 tablet_service.cc:1511] Processing CreateTablet for tablet c52d63f523d24195bc8127113598ff91 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:21:40.986043 23008 leader_election.cc:304] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:40.983191 23086 tablet_bootstrap.cc:654] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:40.979733 22934 leader_election.cc:343] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.007594 23094 raft_consensus.cc:2804] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:41.012451 23094 raft_consensus.cc:493] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:41.012878 23094 raft_consensus.cc:3060] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.018385 23027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c52d63f523d24195bc8127113598ff91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:41.022102 22934 leader_election.cc:304] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
W20260709 13:21:41.023198 22934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 43514.6us
I20260709 13:21:41.023324 22809 tablet_service.cc:1511] Processing CreateTablet for tablet 0abfe80c1fa642ec8a0d075f75264816 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:21:41.023728 23093 raft_consensus.cc:2749] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:41.024515 23093 raft_consensus.cc:493] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.025022 23093 raft_consensus.cc:515] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:41.025978 22878 tablet_service.cc:1511] Processing CreateTablet for tablet c52d63f523d24195bc8127113598ff91 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:21:41.027275 22878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c52d63f523d24195bc8127113598ff91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:41.024672 22809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0abfe80c1fa642ec8a0d075f75264816. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:41.032985 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f631a34c02c4de8ae5bd70760fba972" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:41.035614 23028 tablet_service.cc:1511] Processing CreateTablet for tablet 0abfe80c1fa642ec8a0d075f75264816 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:21:41.037164 23028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0abfe80c1fa642ec8a0d075f75264816. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:41.036825 23091 raft_consensus.cc:515] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.046921 23093 leader_election.cc:290] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:41.011270 23085 ts_tablet_manager.cc:1434] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.258s	user 0.012s	sys 0.012s
I20260709 13:21:41.047752 23084 tablet_bootstrap.cc:654] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.047727 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
W20260709 13:21:41.048884 23008 leader_election.cc:343] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.058228 23093 raft_consensus.cc:493] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.065317 22959 tablet_service.cc:1511] Processing CreateTablet for tablet d217f95123f542e886aa049a608a88d4 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:21:41.065783 23093 raft_consensus.cc:515] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.060879 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0957b803701e45648f538789ec62e507" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
W20260709 13:21:41.066833 22932 leader_election.cc:343] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.065191 22801 tablet_service.cc:1511] Processing CreateTablet for tablet c52d63f523d24195bc8127113598ff91 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:21:41.073714 23121 raft_consensus.cc:493] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.066843 22968 raft_consensus.cc:2468] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 0.
I20260709 13:21:41.079960 22959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d217f95123f542e886aa049a608a88d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:41.080684 23091 leader_election.cc:290] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:41.081624 23085 tablet_bootstrap.cc:492] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:41.074944 23121 raft_consensus.cc:515] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.084383 23091 raft_consensus.cc:493] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.090178 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f631a34c02c4de8ae5bd70760fba972" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:41.091125 22955 tablet_service.cc:1511] Processing CreateTablet for tablet 0abfe80c1fa642ec8a0d075f75264816 (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:21:41.091490 23121 leader_election.cc:290] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:41.092474 22955 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0abfe80c1fa642ec8a0d075f75264816. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:41.087630 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f631a34c02c4de8ae5bd70760fba972" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:41.095117 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0957b803701e45648f538789ec62e507" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:41.095587 23087 tablet_bootstrap.cc:492] T 8f631a34c02c4de8ae5bd70760fba972 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:41.096235 23087 ts_tablet_manager.cc:1403] T 8f631a34c02c4de8ae5bd70760fba972 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.214s	user 0.033s	sys 0.023s
W20260709 13:21:41.096998 22932 leader_election.cc:343] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.097784 23093 leader_election.cc:290] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:41.080801 22957 tablet_service.cc:1511] Processing CreateTablet for tablet b1394978547a410199b4f798a10edb2d (DEFAULT_TABLE table=test-workload [id=c1289a9189604bea9a8576c1900cdcc3]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
W20260709 13:21:41.098951 22785 leader_election.cc:343] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.099092 22957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1394978547a410199b4f798a10edb2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:41.089663 22801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c52d63f523d24195bc8127113598ff91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:41.112165 23086 tablet_bootstrap.cc:492] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:41.112815 23086 ts_tablet_manager.cc:1403] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.238s	user 0.009s	sys 0.016s
I20260709 13:21:41.118700 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:41.114312 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0957b803701e45648f538789ec62e507" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:41.122499 22895 raft_consensus.cc:2468] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 0.
I20260709 13:21:41.128317 22934 leader_election.cc:304] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68
I20260709 13:21:41.129722 23093 raft_consensus.cc:2804] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:41.132349 23093 raft_consensus.cc:493] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:41.132738 23093 raft_consensus.cc:3060] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.100811 23091 raft_consensus.cc:515] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.125183 23087 raft_consensus.cc:359] T 8f631a34c02c4de8ae5bd70760fba972 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.140271 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086a7123d3ad43d79101a7c5de5158fe" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
W20260709 13:21:41.142160 22783 leader_election.cc:343] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.142817 23091 leader_election.cc:290] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:41.145666 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086a7123d3ad43d79101a7c5de5158fe" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:41.135389 23087 raft_consensus.cc:385] T 8f631a34c02c4de8ae5bd70760fba972 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.146413 23087 raft_consensus.cc:740] T 8f631a34c02c4de8ae5bd70760fba972 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.147198 23087 consensus_queue.cc:260] T 8f631a34c02c4de8ae5bd70760fba972 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
W20260709 13:21:41.148371 22785 leader_election.cc:343] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.149849 22785 leader_election.cc:304] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:41.149402 23087 ts_tablet_manager.cc:1434] T 8f631a34c02c4de8ae5bd70760fba972 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
I20260709 13:21:41.152644 23091 raft_consensus.cc:2749] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:41.150846 22860 leader_election.cc:304] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68
I20260709 13:21:41.154090 23121 raft_consensus.cc:2804] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:41.154428 23121 raft_consensus.cc:493] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:41.154686 23121 raft_consensus.cc:3060] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:41.149658 22934 leader_election.cc:343] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.158471 22934 leader_election.cc:304] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:41.159524 23128 raft_consensus.cc:2749] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:41.152359 23087 tablet_bootstrap.cc:492] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:41.162492 23086 raft_consensus.cc:359] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.166105 23086 raft_consensus.cc:385] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.146119 23094 raft_consensus.cc:515] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.169019 23043 raft_consensus.cc:2393] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 in current term 1: Already voted for candidate 27745ef236cd494684f62a1269411a68 in this term.
I20260709 13:21:41.171201 22783 leader_election.cc:304] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:41.167937 23093 raft_consensus.cc:515] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.172597 23091 raft_consensus.cc:2749] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:41.167937 23121 raft_consensus.cc:515] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.174518 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:41.168064 23086 raft_consensus.cc:740] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.177093 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0957b803701e45648f538789ec62e507" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:41.166379 23085 tablet_bootstrap.cc:654] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.179258 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0957b803701e45648f538789ec62e507" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:41.179787 23096 consensus_queue.cc:1048] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:21:41.185534 22932 leader_election.cc:343] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.186688 23086 consensus_queue.cc:260] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.180914 23094 leader_election.cc:290] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:41.181062 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f631a34c02c4de8ae5bd70760fba972" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:41.190465 22820 raft_consensus.cc:2393] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 in current term 1: Already voted for candidate 91f33bdbdd63438294071eb3776db679 in this term.
W20260709 13:21:41.191175 22860 leader_election.cc:343] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.190435 23084 tablet_bootstrap.cc:492] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:41.192313 23084 ts_tablet_manager.cc:1403] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.202s	user 0.021s	sys 0.019s
I20260709 13:21:41.182296 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:41.193609 22819 raft_consensus.cc:2468] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 0.
I20260709 13:21:41.194741 22860 leader_election.cc:304] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:41.195540 23084 raft_consensus.cc:359] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.183435 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f631a34c02c4de8ae5bd70760fba972" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:41.196538 22894 raft_consensus.cc:3060] T 8f631a34c02c4de8ae5bd70760fba972 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.200738 23085 tablet_bootstrap.cc:492] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:41.201349 23085 ts_tablet_manager.cc:1403] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.129s	user 0.007s	sys 0.008s
I20260709 13:21:41.195906 23096 raft_consensus.cc:2804] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:41.202121 23096 raft_consensus.cc:493] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:41.202527 23096 raft_consensus.cc:3060] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.204692 23085 raft_consensus.cc:359] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.205626 23085 raft_consensus.cc:385] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.206004 23085 raft_consensus.cc:740] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.206423 23086 ts_tablet_manager.cc:1434] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.093s	user 0.007s	sys 0.001s
I20260709 13:21:41.179744 23093 leader_election.cc:290] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:41.209300 23096 raft_consensus.cc:515] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.213498 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0957b803701e45648f538789ec62e507" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:41.214221 23096 leader_election.cc:290] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:41.213989 22895 raft_consensus.cc:2393] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 in current term 1: Already voted for candidate 49cbf11e18ce4268a7fcca76872afb23 in this term.
I20260709 13:21:41.212330 22894 raft_consensus.cc:2468] T 8f631a34c02c4de8ae5bd70760fba972 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
W20260709 13:21:41.215384 22858 leader_election.cc:343] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.215693 23084 raft_consensus.cc:385] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.216063 23084 raft_consensus.cc:740] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.207176 23085 consensus_queue.cc:260] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.216850 23084 consensus_queue.cc:260] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.217947 22934 leader_election.cc:304] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:41.218676 23085 ts_tablet_manager.cc:1434] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.004s
I20260709 13:21:41.219110 23128 raft_consensus.cc:2749] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:41.216768 23008 leader_election.cc:304] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:41.220008 23094 raft_consensus.cc:2804] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:41.220485 23094 raft_consensus.cc:697] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:41.221299 23084 ts_tablet_manager.cc:1434] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:21:41.221347 23094 consensus_queue.cc:237] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.222433 23084 tablet_bootstrap.cc:492] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:41.224301 23091 raft_consensus.cc:493] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.224867 23091 raft_consensus.cc:515] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.210269 23086 tablet_bootstrap.cc:492] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:41.227552 23087 tablet_bootstrap.cc:654] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.228253 23121 leader_election.cc:290] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:41.228197 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e09daa4f2dd474d96ae9c4dfa1a8758" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:41.230939 23043 raft_consensus.cc:2468] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
I20260709 13:21:41.233276 22783 leader_election.cc:304] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:41.234973 23085 tablet_bootstrap.cc:492] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:41.236873 23091 leader_election.cc:290] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:41.239081 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e09daa4f2dd474d96ae9c4dfa1a8758" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:41.240191 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:41.240346 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0957b803701e45648f538789ec62e507" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
W20260709 13:21:41.240876 22785 leader_election.cc:343] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.241217 23130 raft_consensus.cc:2804] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:41.241082 22968 raft_consensus.cc:2393] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 in current term 1: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:41.241706 23130 raft_consensus.cc:493] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:41.242162 23130 raft_consensus.cc:3060] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.242189 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:41.243723 23086 tablet_bootstrap.cc:654] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:41.243858 22860 leader_election.cc:343] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.240859 22819 raft_consensus.cc:3060] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.250373 23084 tablet_bootstrap.cc:654] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.252952 22860 leader_election.cc:304] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:41.255898 23096 raft_consensus.cc:2749] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:41.256938 23096 consensus_queue.cc:1048] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:41.267686 22819 raft_consensus.cc:2468] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:41.271442 23087 tablet_bootstrap.cc:492] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:41.273053 23087 ts_tablet_manager.cc:1403] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.121s	user 0.018s	sys 0.009s
I20260709 13:21:41.274814 23138 raft_consensus.cc:493] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.275483 23138 raft_consensus.cc:515] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.278508 23138 leader_election.cc:290] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:41.279199 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cabd9adbf414496bbaab3946b413d313" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:41.279357 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cabd9adbf414496bbaab3946b413d313" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:41.279557 22860 leader_election.cc:304] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:41.280517 23137 raft_consensus.cc:2804] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:21:41.280897 22932 leader_election.cc:343] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.281035 23137 raft_consensus.cc:697] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:41.277078 23087 raft_consensus.cc:359] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
W20260709 13:21:41.281579 22934 leader_election.cc:343] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.281885 23087 raft_consensus.cc:385] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.281997 23137 consensus_queue.cc:237] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.283135 23087 raft_consensus.cc:740] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.284404 23087 consensus_queue.cc:260] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.282142 22934 leader_election.cc:304] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:41.290207 23138 raft_consensus.cc:2749] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:41.295166 23130 raft_consensus.cc:515] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.296530 23087 ts_tablet_manager.cc:1434] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.023s	user 0.004s	sys 0.004s
I20260709 13:21:41.297940 23087 tablet_bootstrap.cc:492] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:41.298511 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e09daa4f2dd474d96ae9c4dfa1a8758" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:41.299252 23042 raft_consensus.cc:3060] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.305449 23087 tablet_bootstrap.cc:654] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.319689 23085 tablet_bootstrap.cc:654] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.322886 23042 raft_consensus.cc:2468] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:41.323839 23130 leader_election.cc:290] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:41.324196 22783 leader_election.cc:304] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:41.329458 23086 tablet_bootstrap.cc:492] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:41.330132 23086 ts_tablet_manager.cc:1403] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.120s	user 0.022s	sys 0.016s
I20260709 13:21:41.330253 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e09daa4f2dd474d96ae9c4dfa1a8758" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:41.330871 22895 raft_consensus.cc:3060] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.326118 23130 raft_consensus.cc:2804] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:41.336555 23130 raft_consensus.cc:697] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:41.338205 23130 consensus_queue.cc:237] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.349512 22715 catalog_manager.cc:5697] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 1 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:41.357651 23084 tablet_bootstrap.cc:492] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:41.358268 23084 ts_tablet_manager.cc:1403] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.136s	user 0.026s	sys 0.029s
I20260709 13:21:41.359480 23085 tablet_bootstrap.cc:492] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:41.360131 23085 ts_tablet_manager.cc:1403] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.140s	user 0.016s	sys 0.017s
I20260709 13:21:41.345898 23086 raft_consensus.cc:359] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.362332 23086 raft_consensus.cc:385] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.362651 23086 raft_consensus.cc:740] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.363739 23085 raft_consensus.cc:359] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.364598 23085 raft_consensus.cc:385] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.364935 23085 raft_consensus.cc:740] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.365824 22895 raft_consensus.cc:2468] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:41.366927 23086 consensus_queue.cc:260] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.369408 23086 ts_tablet_manager.cc:1434] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.000s
I20260709 13:21:41.370473 23086 tablet_bootstrap.cc:492] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:41.371508 22717 catalog_manager.cc:5697] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:41.361271 23084 raft_consensus.cc:359] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.374159 23084 raft_consensus.cc:385] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.374500 23084 raft_consensus.cc:740] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.375401 23084 consensus_queue.cc:260] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.377745 23084 ts_tablet_manager.cc:1434] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.000s
I20260709 13:21:41.365835 23085 consensus_queue.cc:260] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.378829 23084 tablet_bootstrap.cc:492] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:41.380247 23085 ts_tablet_manager.cc:1434] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:21:41.381135 23085 tablet_bootstrap.cc:492] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:41.385736 23084 tablet_bootstrap.cc:654] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.385844 22718 catalog_manager.cc:5697] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:41.388325 23087 tablet_bootstrap.cc:492] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:41.388839 23087 ts_tablet_manager.cc:1403] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.091s	user 0.021s	sys 0.007s
I20260709 13:21:41.391901 23087 raft_consensus.cc:359] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.392606 23087 raft_consensus.cc:385] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.392930 23087 raft_consensus.cc:740] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.393324 23086 tablet_bootstrap.cc:654] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.393639 23087 consensus_queue.cc:260] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.394861 23085 tablet_bootstrap.cc:654] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.396132 23087 ts_tablet_manager.cc:1434] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:21:41.404608 23086 tablet_bootstrap.cc:492] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:41.405215 23086 ts_tablet_manager.cc:1403] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.001s
I20260709 13:21:41.407090 23085 tablet_bootstrap.cc:492] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:41.407894 23085 ts_tablet_manager.cc:1403] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.004s
I20260709 13:21:41.408245 23086 raft_consensus.cc:359] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:41.409055 23086 raft_consensus.cc:385] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.409375 23086 raft_consensus.cc:740] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.410324 23086 consensus_queue.cc:260] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:41.411826 23085 raft_consensus.cc:359] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.412767 23086 ts_tablet_manager.cc:1434] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:21:41.412796 23085 raft_consensus.cc:385] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.413334 23085 raft_consensus.cc:740] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.413909 23086 tablet_bootstrap.cc:492] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:41.420892 23086 tablet_bootstrap.cc:654] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.414198 23085 consensus_queue.cc:260] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.428275 23087 tablet_bootstrap.cc:492] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:41.431807 23085 ts_tablet_manager.cc:1434] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.023s	user 0.000s	sys 0.006s
I20260709 13:21:41.432935 23086 tablet_bootstrap.cc:492] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:41.433480 23086 ts_tablet_manager.cc:1403] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.020s	user 0.004s	sys 0.012s
I20260709 13:21:41.436331 23086 raft_consensus.cc:359] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.437003 23086 raft_consensus.cc:385] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.437453 23086 raft_consensus.cc:740] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.436879 23087 tablet_bootstrap.cc:654] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.438190 23086 consensus_queue.cc:260] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.442735 23084 tablet_bootstrap.cc:492] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:41.443248 23084 ts_tablet_manager.cc:1403] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.065s	user 0.023s	sys 0.037s
I20260709 13:21:41.443965 23086 ts_tablet_manager.cc:1434] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:21:41.444993 23086 tablet_bootstrap.cc:492] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:41.445880 23085 tablet_bootstrap.cc:492] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:41.446458 23084 raft_consensus.cc:359] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.447232 23084 raft_consensus.cc:385] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.447502 23084 raft_consensus.cc:740] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.448372 23084 consensus_queue.cc:260] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.451505 23086 tablet_bootstrap.cc:654] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.454407 23084 ts_tablet_manager.cc:1434] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:21:41.456425 23084 tablet_bootstrap.cc:492] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:41.458768 23087 tablet_bootstrap.cc:492] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:41.459357 23087 ts_tablet_manager.cc:1403] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.000s
I20260709 13:21:41.462656 23087 raft_consensus.cc:359] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.463285 23084 tablet_bootstrap.cc:654] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.463419 23087 raft_consensus.cc:385] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.464004 23087 raft_consensus.cc:740] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.464958 23087 consensus_queue.cc:260] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.466251 23085 tablet_bootstrap.cc:654] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.476287 23087 ts_tablet_manager.cc:1434] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:21:41.476795 23086 tablet_bootstrap.cc:492] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:41.477316 23086 ts_tablet_manager.cc:1403] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.033s	user 0.020s	sys 0.008s
I20260709 13:21:41.477396 23084 tablet_bootstrap.cc:492] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:41.477849 23085 tablet_bootstrap.cc:492] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:41.478013 23084 ts_tablet_manager.cc:1403] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.007s
I20260709 13:21:41.478375 23085 ts_tablet_manager.cc:1403] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.033s	user 0.005s	sys 0.011s
I20260709 13:21:41.480134 23086 raft_consensus.cc:359] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.480809 23087 tablet_bootstrap.cc:492] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:41.480887 23086 raft_consensus.cc:385] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.481323 23086 raft_consensus.cc:740] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.481775 23084 raft_consensus.cc:359] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.482125 23086 consensus_queue.cc:260] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.482810 23085 raft_consensus.cc:359] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.483567 23084 raft_consensus.cc:385] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.485343 23130 raft_consensus.cc:493] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.485594 23086 ts_tablet_manager.cc:1434] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:21:41.485894 23130 raft_consensus.cc:515] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.484002 23084 raft_consensus.cc:740] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.483880 23085 raft_consensus.cc:385] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.487211 23085 raft_consensus.cc:740] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.487546 23087 tablet_bootstrap.cc:654] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.486701 23086 tablet_bootstrap.cc:492] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:41.487962 23130 leader_election.cc:290] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:41.487998 23085 consensus_queue.cc:260] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.489997 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505eab84b06f46ee9d336abf01353dee" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:41.490545 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505eab84b06f46ee9d336abf01353dee" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:41.490675 22895 raft_consensus.cc:2468] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
W20260709 13:21:41.491993 22785 leader_election.cc:343] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.487547 23084 consensus_queue.cc:260] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.492748 22785 leader_election.cc:304] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:41.494032 23084 ts_tablet_manager.cc:1434] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:21:41.495074 23084 tablet_bootstrap.cc:492] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:41.498054 23130 raft_consensus.cc:2804] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:41.498549 23130 raft_consensus.cc:493] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:41.498914 23130 raft_consensus.cc:3060] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.501142 23084 tablet_bootstrap.cc:654] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.507212 23085 ts_tablet_manager.cc:1434] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.028s	user 0.004s	sys 0.002s
I20260709 13:21:41.513442 23086 tablet_bootstrap.cc:654] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.515336 23091 raft_consensus.cc:493] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.515967 23091 raft_consensus.cc:515] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.517354 23085 tablet_bootstrap.cc:492] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:41.518381 23091 leader_election.cc:290] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:41.520215 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "084ee2ebbf564ed583f84c1227d29bd4" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:41.520964 23043 raft_consensus.cc:2468] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
I20260709 13:21:41.522089 23130 raft_consensus.cc:515] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.522362 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "084ee2ebbf564ed583f84c1227d29bd4" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:41.523214 22783 leader_election.cc:304] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
W20260709 13:21:41.525300 22785 leader_election.cc:343] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.525906 23130 leader_election.cc:290] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:41.526707 23130 raft_consensus.cc:2804] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:41.527103 23130 raft_consensus.cc:493] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:41.527462 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505eab84b06f46ee9d336abf01353dee" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:41.528183 22895 raft_consensus.cc:3060] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.530167 23085 tablet_bootstrap.cc:654] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.527439 23130 raft_consensus.cc:3060] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.538691 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505eab84b06f46ee9d336abf01353dee" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
W20260709 13:21:41.540524 22785 leader_election.cc:343] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.541312 22895 raft_consensus.cc:2468] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:41.542757 22785 leader_election.cc:304] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:41.543668 23091 raft_consensus.cc:2804] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:41.543908 23130 raft_consensus.cc:515] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.545394 23087 tablet_bootstrap.cc:492] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:41.545931 23085 tablet_bootstrap.cc:492] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:41.546181 23087 ts_tablet_manager.cc:1403] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.066s	user 0.030s	sys 0.025s
I20260709 13:21:41.546418 23085 ts_tablet_manager.cc:1403] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.029s	user 0.007s	sys 0.005s
I20260709 13:21:41.547245 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "084ee2ebbf564ed583f84c1227d29bd4" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:41.547837 23043 raft_consensus.cc:3060] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.549291 23085 raft_consensus.cc:359] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:41.550004 23085 raft_consensus.cc:385] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.549616 23087 raft_consensus.cc:359] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.543880 23084 tablet_bootstrap.cc:492] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:41.550429 23087 raft_consensus.cc:385] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.550850 23084 ts_tablet_manager.cc:1403] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.056s	user 0.023s	sys 0.011s
I20260709 13:21:41.550809 23087 raft_consensus.cc:740] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.552582 23087 consensus_queue.cc:260] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.553378 23086 tablet_bootstrap.cc:492] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:41.553865 23086 ts_tablet_manager.cc:1403] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.067s	user 0.011s	sys 0.004s
I20260709 13:21:41.553625 23084 raft_consensus.cc:359] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:41.554338 23084 raft_consensus.cc:385] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.554601 23084 raft_consensus.cc:740] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.554605 23087 ts_tablet_manager.cc:1434] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:21:41.556880 23087 tablet_bootstrap.cc:492] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:41.550311 23085 raft_consensus.cc:740] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.555306 23084 consensus_queue.cc:260] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:41.557041 23086 raft_consensus.cc:359] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:41.561321 23086 raft_consensus.cc:385] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.561617 23086 raft_consensus.cc:740] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.562386 23086 consensus_queue.cc:260] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:41.564342 23086 ts_tablet_manager.cc:1434] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:21:41.559854 23043 raft_consensus.cc:2468] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:41.560235 23091 raft_consensus.cc:697] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:41.566912 22783 leader_election.cc:304] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:41.561754 23084 ts_tablet_manager.cc:1434] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:21:41.567785 23130 leader_election.cc:290] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:41.568361 23130 raft_consensus.cc:2804] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:41.568563 23084 tablet_bootstrap.cc:492] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:41.568876 23130 raft_consensus.cc:697] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:41.569747 23130 consensus_queue.cc:237] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.565397 23087 tablet_bootstrap.cc:654] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.560225 23085 consensus_queue.cc:260] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:41.573001 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "084ee2ebbf564ed583f84c1227d29bd4" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
W20260709 13:21:41.574702 22785 leader_election.cc:343] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.565254 23086 tablet_bootstrap.cc:492] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:41.566676 23091 consensus_queue.cc:237] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.574837 23085 ts_tablet_manager.cc:1434] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 13:21:41.582505 23094 raft_consensus.cc:493] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.583123 23094 raft_consensus.cc:515] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.585333 23094 leader_election.cc:290] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:41.582161 23085 tablet_bootstrap.cc:492] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:41.587678 23086 tablet_bootstrap.cc:654] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.589855 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086a7123d3ad43d79101a7c5de5158fe" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:41.590607 22819 raft_consensus.cc:2468] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 0.
I20260709 13:21:41.591629 23085 tablet_bootstrap.cc:654] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.592543 23008 leader_election.cc:304] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:41.593461 23094 raft_consensus.cc:2804] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:41.593909 23094 raft_consensus.cc:493] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:41.591735 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086a7123d3ad43d79101a7c5de5158fe" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:41.594336 23094 raft_consensus.cc:3060] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.595366 23087 tablet_bootstrap.cc:492] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:41.595538 23084 tablet_bootstrap.cc:654] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:41.595703 23008 leader_election.cc:343] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.598516 23086 tablet_bootstrap.cc:492] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:41.599017 23086 ts_tablet_manager.cc:1403] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.000s
I20260709 13:21:41.595867 23087 ts_tablet_manager.cc:1403] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.039s	user 0.014s	sys 0.006s
I20260709 13:21:41.601783 23086 raft_consensus.cc:359] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:41.602515 23086 raft_consensus.cc:385] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.602818 23086 raft_consensus.cc:740] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.603510 23087 raft_consensus.cc:359] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.603634 23086 consensus_queue.cc:260] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:41.604243 23087 raft_consensus.cc:385] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.602327 23094 raft_consensus.cc:515] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.605685 23086 ts_tablet_manager.cc:1434] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:21:41.606712 23086 tablet_bootstrap.cc:492] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:41.606968 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086a7123d3ad43d79101a7c5de5158fe" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:41.607622 22819 raft_consensus.cc:3060] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.604542 23087 raft_consensus.cc:740] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.609874 23087 consensus_queue.cc:260] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.611912 23087 ts_tablet_manager.cc:1434] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:21:41.612285 23086 tablet_bootstrap.cc:654] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.614168 22716 catalog_manager.cc:5697] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:41.617542 22819 raft_consensus.cc:2468] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
I20260709 13:21:41.619011 23008 leader_election.cc:304] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:41.620309 23125 raft_consensus.cc:2804] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:41.629510 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086a7123d3ad43d79101a7c5de5158fe" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
W20260709 13:21:41.631354 23008 leader_election.cc:343] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.633152 23094 leader_election.cc:290] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:41.635311 23087 tablet_bootstrap.cc:492] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:41.637190 23125 raft_consensus.cc:697] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:41.638219 23125 consensus_queue.cc:237] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.640996 22717 catalog_manager.cc:5697] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:41.642558 23087 tablet_bootstrap.cc:654] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.644565 23085 tablet_bootstrap.cc:492] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:41.645184 23085 ts_tablet_manager.cc:1403] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.063s	user 0.013s	sys 0.000s
I20260709 13:21:41.646317 23173 raft_consensus.cc:493] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.646960 23173 raft_consensus.cc:515] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.650693 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0957b803701e45648f538789ec62e507" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:41.651619 22895 raft_consensus.cc:2393] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 in current term 1: Already voted for candidate 49cbf11e18ce4268a7fcca76872afb23 in this term.
I20260709 13:21:41.653345 23085 raft_consensus.cc:359] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.654786 23085 raft_consensus.cc:385] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.655133 23085 raft_consensus.cc:740] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.655894 23085 consensus_queue.cc:260] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.654072 23173 leader_election.cc:290] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:41.658689 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0957b803701e45648f538789ec62e507" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:41.659147 23086 tablet_bootstrap.cc:492] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:41.659317 23087 tablet_bootstrap.cc:492] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:41.659503 22968 raft_consensus.cc:2393] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 in current term 1: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:41.659770 23087 ts_tablet_manager.cc:1403] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.004s
I20260709 13:21:41.659929 23086 ts_tablet_manager.cc:1403] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.053s	user 0.036s	sys 0.017s
I20260709 13:21:41.659821 23085 ts_tablet_manager.cc:1434] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.004s
I20260709 13:21:41.660856 23008 leader_election.cc:304] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:41.662642 23085 tablet_bootstrap.cc:492] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:41.662847 23173 raft_consensus.cc:3060] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.663810 23086 raft_consensus.cc:359] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.664539 23086 raft_consensus.cc:385] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.664817 23086 raft_consensus.cc:740] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.665511 23086 consensus_queue.cc:260] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
W20260709 13:21:41.666770 22766 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:41.667500 23086 ts_tablet_manager.cc:1434] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:21:41.668043 23084 tablet_bootstrap.cc:492] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:41.668522 23086 tablet_bootstrap.cc:492] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:41.668586 23084 ts_tablet_manager.cc:1403] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.100s	user 0.010s	sys 0.005s
I20260709 13:21:41.672631 23085 tablet_bootstrap.cc:654] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.674703 23086 tablet_bootstrap.cc:654] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.674788 23137 consensus_queue.cc:1048] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:41.662776 23087 raft_consensus.cc:359] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.675665 23087 raft_consensus.cc:385] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.675966 23087 raft_consensus.cc:740] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.676843 23087 consensus_queue.cc:260] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.679107 23125 raft_consensus.cc:493] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.679720 23125 raft_consensus.cc:515] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.683876 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a064be2ec640a4a8283957d0b9b98c" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:41.679170 23087 ts_tablet_manager.cc:1434] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.000s
I20260709 13:21:41.685369 23087 tablet_bootstrap.cc:492] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
W20260709 13:21:41.685505 23008 leader_election.cc:343] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.685977 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a064be2ec640a4a8283957d0b9b98c" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:41.686724 22895 raft_consensus.cc:2468] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 0.
I20260709 13:21:41.684062 23125 leader_election.cc:290] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:41.688802 23008 leader_election.cc:304] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:41.691700 22716 catalog_manager.cc:5697] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 1 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:41.695399 23091 raft_consensus.cc:493] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.672644 23084 raft_consensus.cc:359] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.695986 23084 raft_consensus.cc:385] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.695927 23091 raft_consensus.cc:515] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:41.696431 23084 raft_consensus.cc:740] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.697399 23084 consensus_queue.cc:260] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.699687 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:41.701396 22968 raft_consensus.cc:2468] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
I20260709 13:21:41.701737 23087 tablet_bootstrap.cc:654] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:41.705134 22860 consensus_peers.cc:597] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Couldn't send request to peer 8ef3887fb3874f8b81079e542d252944. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:41.700590 23173 raft_consensus.cc:2749] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:41.700274 23091 leader_election.cc:290] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:41.702529 22785 leader_election.cc:304] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:41.708369 23130 raft_consensus.cc:2804] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:41.708778 23130 raft_consensus.cc:493] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:41.709116 23130 raft_consensus.cc:3060] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.710569 23125 consensus_queue.cc:1048] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:41.702534 23094 raft_consensus.cc:2804] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:41.711787 23094 raft_consensus.cc:493] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:41.712169 23094 raft_consensus.cc:3060] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.715696 23138 raft_consensus.cc:493] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.716270 23138 raft_consensus.cc:515] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:41.718351 23138 leader_election.cc:290] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:41.719553 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd78248be3954373ad4fd60b8ae7593d" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:41.720304 23043 raft_consensus.cc:2468] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 0.
I20260709 13:21:41.721804 23084 ts_tablet_manager.cc:1434] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.052s	user 0.005s	sys 0.000s
I20260709 13:21:41.722839 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:41.723129 22932 leader_election.cc:304] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:41.745571 23138 raft_consensus.cc:2804] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:41.746019 23138 raft_consensus.cc:493] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:41.746361 23138 raft_consensus.cc:3060] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.747249 23084 tablet_bootstrap.cc:492] T 086a7123d3ad43d79101a7c5de5158fe P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
W20260709 13:21:41.749526 22783 leader_election.cc:343] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.727751 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd78248be3954373ad4fd60b8ae7593d" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:41.751928 22820 raft_consensus.cc:2468] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 0.
I20260709 13:21:41.753446 23084 tablet_bootstrap.cc:654] T 086a7123d3ad43d79101a7c5de5158fe P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.753732 23125 raft_consensus.cc:493] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.754271 23125 raft_consensus.cc:515] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.731026 23085 tablet_bootstrap.cc:492] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:41.735630 23086 tablet_bootstrap.cc:492] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:41.757258 23086 ts_tablet_manager.cc:1403] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.089s	user 0.016s	sys 0.035s
I20260709 13:21:41.757318 23125 leader_election.cc:290] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:41.760059 23086 raft_consensus.cc:359] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.760787 23086 raft_consensus.cc:385] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.760619 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94800e302e4641bb857f2d160f1f60ea" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:41.761211 23086 raft_consensus.cc:740] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.761696 22895 raft_consensus.cc:2468] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 0.
I20260709 13:21:41.762096 23086 consensus_queue.cc:260] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.764214 23086 ts_tablet_manager.cc:1434] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:21:41.735860 23087 tablet_bootstrap.cc:492] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:41.737455 23094 raft_consensus.cc:515] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.764809 23087 ts_tablet_manager.cc:1403] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.080s	user 0.009s	sys 0.003s
I20260709 13:21:41.766304 23094 leader_election.cc:290] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:41.742484 23173 consensus_queue.cc:1048] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:41.726733 23130 raft_consensus.cc:515] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:41.766925 23086 tablet_bootstrap.cc:492] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:41.767704 23087 raft_consensus.cc:359] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.768378 23087 raft_consensus.cc:385] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.768667 23087 raft_consensus.cc:740] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.768390 23130 leader_election.cc:290] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:41.770201 23085 ts_tablet_manager.cc:1403] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.108s	user 0.015s	sys 0.000s
I20260709 13:21:41.771008 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a064be2ec640a4a8283957d0b9b98c" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:41.775949 22818 raft_consensus.cc:3060] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.778680 23086 tablet_bootstrap.cc:654] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.780181 23008 leader_election.cc:304] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:41.773020 23085 raft_consensus.cc:359] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.781018 23085 raft_consensus.cc:385] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.781379 23085 raft_consensus.cc:740] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.782305 23085 consensus_queue.cc:260] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.784458 23085 ts_tablet_manager.cc:1434] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.004s
I20260709 13:21:41.785504 23130 raft_consensus.cc:493] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.774323 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a064be2ec640a4a8283957d0b9b98c" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:41.780233 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:41.786162 23130 raft_consensus.cc:515] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.787055 22895 raft_consensus.cc:3060] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.789031 23130 leader_election.cc:290] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:41.784901 23125 raft_consensus.cc:2804] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:41.789700 23125 raft_consensus.cc:493] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:41.790163 23125 raft_consensus.cc:3060] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.791545 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050e9ef11f664c849a40cbb6acb25f19" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
W20260709 13:21:41.793447 22785 leader_election.cc:343] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.795444 23130 raft_consensus.cc:493] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.769380 23087 consensus_queue.cc:260] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.796005 23130 raft_consensus.cc:515] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.798996 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3139446c554b419fc28cad31653600" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:41.772432 23138 raft_consensus.cc:515] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:41.799808 22893 raft_consensus.cc:2468] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
I20260709 13:21:41.800421 23085 tablet_bootstrap.cc:492] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:41.801306 22785 leader_election.cc:304] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:41.802232 23091 raft_consensus.cc:2804] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:41.802611 23091 raft_consensus.cc:493] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:41.802947 23091 raft_consensus.cc:3060] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.809049 23085 tablet_bootstrap.cc:654] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.799046 23130 leader_election.cc:290] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:41.801561 23087 ts_tablet_manager.cc:1434] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.000s
W20260709 13:21:41.810721 22783 leader_election.cc:343] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.811146 23087 tablet_bootstrap.cc:492] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:41.813103 22895 raft_consensus.cc:2468] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
I20260709 13:21:41.816871 23125 raft_consensus.cc:515] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.817582 23087 tablet_bootstrap.cc:654] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.819265 23091 raft_consensus.cc:515] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.821355 23008 leader_election.cc:304] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:41.822746 23138 leader_election.cc:290] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:41.827375 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd78248be3954373ad4fd60b8ae7593d" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:41.834879 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:41.835633 22968 raft_consensus.cc:3060] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.830214 23173 raft_consensus.cc:2804] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:41.838295 23173 raft_consensus.cc:697] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:41.839114 23173 consensus_queue.cc:237] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:41.831413 23138 raft_consensus.cc:493] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.841739 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94800e302e4641bb857f2d160f1f60ea" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:41.842238 23138 raft_consensus.cc:515] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.842734 22894 raft_consensus.cc:3060] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.844416 23138 leader_election.cc:290] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:41.845486 22820 raft_consensus.cc:3060] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.841784 22818 raft_consensus.cc:2468] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
I20260709 13:21:41.846160 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968e16cef40146509d654274b9fd2687" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
W20260709 13:21:41.847697 22934 leader_election.cc:343] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.848699 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd78248be3954373ad4fd60b8ae7593d" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:41.849301 23043 raft_consensus.cc:3060] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.853632 22968 raft_consensus.cc:2468] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:41.854233 23091 leader_election.cc:290] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:41.856238 23182 raft_consensus.cc:493] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.856720 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3139446c554b419fc28cad31653600" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:41.857391 22895 raft_consensus.cc:3060] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.856776 23182 raft_consensus.cc:515] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.856768 23091 consensus_queue.cc:1048] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:21:41.860035 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3139446c554b419fc28cad31653600" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:41.857230 23043 raft_consensus.cc:2468] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:41.860667 22969 raft_consensus.cc:2468] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
I20260709 13:21:41.862923 23125 leader_election.cc:290] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:41.867936 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955a6d8dbbe94ef9be9c1f69b10675d8" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:41.868554 23182 leader_election.cc:290] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:41.869769 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955a6d8dbbe94ef9be9c1f69b10675d8" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
W20260709 13:21:41.871812 22934 leader_election.cc:343] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.872910 23086 tablet_bootstrap.cc:492] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:41.873420 23086 ts_tablet_manager.cc:1403] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.107s	user 0.008s	sys 0.006s
I20260709 13:21:41.874238 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050e9ef11f664c849a40cbb6acb25f19" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:41.874725 22785 leader_election.cc:304] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68
I20260709 13:21:41.874940 22968 raft_consensus.cc:2468] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
I20260709 13:21:41.875592 22967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94800e302e4641bb857f2d160f1f60ea" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:41.876202 23086 raft_consensus.cc:359] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.876533 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3139446c554b419fc28cad31653600" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:41.876937 23086 raft_consensus.cc:385] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.877287 23086 raft_consensus.cc:740] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.876247 22967 raft_consensus.cc:2468] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 0.
I20260709 13:21:41.879040 23084 tablet_bootstrap.cc:492] T 086a7123d3ad43d79101a7c5de5158fe P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:41.878667 23086 consensus_queue.cc:260] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.879549 23084 ts_tablet_manager.cc:1403] T 086a7123d3ad43d79101a7c5de5158fe P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.133s	user 0.022s	sys 0.022s
I20260709 13:21:41.879751 23130 raft_consensus.cc:2804] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:41.880268 23130 raft_consensus.cc:697] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:41.881079 23130 consensus_queue.cc:237] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
W20260709 13:21:41.881847 22934 leader_election.cc:343] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.882120 22820 raft_consensus.cc:2468] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:41.882447 22934 leader_election.cc:304] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:41.882938 22894 raft_consensus.cc:2468] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
I20260709 13:21:41.884507 23182 raft_consensus.cc:2749] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:41.877154 22969 raft_consensus.cc:3060] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.888507 23086 ts_tablet_manager.cc:1434] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.004s
I20260709 13:21:41.890256 23086 tablet_bootstrap.cc:492] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:41.890439 23084 raft_consensus.cc:359] T 086a7123d3ad43d79101a7c5de5158fe P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.891258 23084 raft_consensus.cc:385] T 086a7123d3ad43d79101a7c5de5158fe P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.901417 22785 leader_election.cc:304] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:41.902218 22934 leader_election.cc:304] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:41.903141 23182 raft_consensus.cc:2804] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:41.903215 23091 raft_consensus.cc:2804] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:41.903668 23091 raft_consensus.cc:493] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:41.904063 23091 raft_consensus.cc:3060] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.903744 23182 raft_consensus.cc:697] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 [term 1 LEADER]: Becoming Leader. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:21:41.905442 23182 consensus_queue.cc:237] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:41.901404 23165 consensus_queue.cc:1048] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:41.908660 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968e16cef40146509d654274b9fd2687" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:41.908721 22967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94800e302e4641bb857f2d160f1f60ea" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:41.912211 22967 raft_consensus.cc:3060] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.912761 22895 raft_consensus.cc:2468] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:41.899714 23084 raft_consensus.cc:740] T 086a7123d3ad43d79101a7c5de5158fe P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.919378 23084 consensus_queue.cc:260] T 086a7123d3ad43d79101a7c5de5158fe P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.900954 23008 leader_election.cc:304] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:41.926890 22785 leader_election.cc:304] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:41.930272 23091 raft_consensus.cc:515] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.931257 22969 raft_consensus.cc:2468] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:41.934785 23165 raft_consensus.cc:2804] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:21:41.920686 22932 leader_election.cc:343] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.945446 23084 ts_tablet_manager.cc:1434] T 086a7123d3ad43d79101a7c5de5158fe P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.066s	user 0.008s	sys 0.001s
I20260709 13:21:41.946230 23130 raft_consensus.cc:493] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.947458 23130 raft_consensus.cc:515] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.946555 23084 tablet_bootstrap.cc:492] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:41.948282 23085 tablet_bootstrap.cc:492] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:41.948769 23085 ts_tablet_manager.cc:1403] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.163s	user 0.017s	sys 0.029s
I20260709 13:21:41.949074 23165 raft_consensus.cc:697] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:41.950682 23121 raft_consensus.cc:493] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.957731 23121 raft_consensus.cc:515] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.958107 23130 leader_election.cc:290] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:41.958894 23087 tablet_bootstrap.cc:492] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:41.959332 23087 ts_tablet_manager.cc:1403] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.148s	user 0.007s	sys 0.009s
I20260709 13:21:41.964082 23085 raft_consensus.cc:359] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:41.964977 23085 raft_consensus.cc:385] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.965323 23085 raft_consensus.cc:740] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.966109 23085 consensus_queue.cc:260] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:41.953727 23094 raft_consensus.cc:493] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.951404 22932 leader_election.cc:304] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:41.967355 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e7b082dcf344fbf83e0ddec4b08b6de" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
W20260709 13:21:41.968138 22932 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 49159.1us
I20260709 13:21:41.969081 23184 raft_consensus.cc:2749] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:21:41.969231 22860 leader_election.cc:343] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.969421 23130 raft_consensus.cc:493] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.967358 23094 raft_consensus.cc:515] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:41.970077 23130 raft_consensus.cc:515] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:41.972076 23094 leader_election.cc:290] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:41.972898 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e7b082dcf344fbf83e0ddec4b08b6de" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:41.973730 22820 raft_consensus.cc:2468] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 0.
I20260709 13:21:41.972899 23130 leader_election.cc:290] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:41.975334 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e638dc505d4b04b802e91b8863d833" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:41.975790 23086 tablet_bootstrap.cc:654] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.976017 22818 raft_consensus.cc:2468] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 0.
I20260709 13:21:41.977522 23008 leader_election.cc:304] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:41.961822 23187 raft_consensus.cc:2804] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:41.978441 23187 raft_consensus.cc:697] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:41.978612 23173 raft_consensus.cc:2804] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:41.979058 22860 leader_election.cc:304] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:41.960119 23121 leader_election.cc:290] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:41.980893 23137 raft_consensus.cc:2804] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:41.981326 23137 raft_consensus.cc:493] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:41.981663 23137 raft_consensus.cc:3060] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.984790 23087 raft_consensus.cc:359] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.985589 23087 raft_consensus.cc:385] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.979107 23173 raft_consensus.cc:493] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:41.986541 23173 raft_consensus.cc:3060] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.986932 23091 leader_election.cc:290] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:41.950160 23165 consensus_queue.cc:237] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.989979 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e638dc505d4b04b802e91b8863d833" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:41.960882 22967 raft_consensus.cc:2468] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
W20260709 13:21:41.992610 23008 leader_election.cc:343] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.985849 23087 raft_consensus.cc:740] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.974852 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e638dc505d4b04b802e91b8863d833" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:41.994145 23087 consensus_queue.cc:260] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:41.996490 23187 consensus_queue.cc:237] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:41.999431 23084 tablet_bootstrap.cc:654] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.009605 23085 ts_tablet_manager.cc:1434] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.000s
I20260709 13:21:42.010893 23085 tablet_bootstrap.cc:492] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:42.030319 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ea5c83aada45d3a98027069e96658c" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:42.031622 23091 consensus_queue.cc:1048] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:42.032342 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e638dc505d4b04b802e91b8863d833" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:42.032761 22967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ea5c83aada45d3a98027069e96658c" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:42.032790 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050e9ef11f664c849a40cbb6acb25f19" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:42.035064 22968 raft_consensus.cc:3060] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.031949 23087 ts_tablet_manager.cc:1434] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.072s	user 0.008s	sys 0.000s
I20260709 13:21:42.035444 22717 catalog_manager.cc:5697] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 1 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:42.037942 23087 tablet_bootstrap.cc:492] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:42.038367 23121 raft_consensus.cc:493] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.039113 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050e9ef11f664c849a40cbb6acb25f19" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
W20260709 13:21:42.039906 22785 leader_election.cc:343] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.022542 23184 raft_consensus.cc:493] T 086a7123d3ad43d79101a7c5de5158fe P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.043747 23121 raft_consensus.cc:515] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.060475 23121 leader_election.cc:290] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:42.061333 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "084ee2ebbf564ed583f84c1227d29bd4" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:42.063171 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "084ee2ebbf564ed583f84c1227d29bd4" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
W20260709 13:21:42.061216 22785 leader_election.cc:343] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:42.079172 22785 leader_election.cc:343] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.079622 22785 leader_election.cc:304] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:42.080468 23195 raft_consensus.cc:2749] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:21:42.080775 22785 leader_election.cc:343] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.051734 23184 raft_consensus.cc:515] T 086a7123d3ad43d79101a7c5de5158fe P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:42.083582 23184 leader_election.cc:290] T 086a7123d3ad43d79101a7c5de5158fe P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:42.078259 22858 leader_election.cc:304] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:42.084987 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086a7123d3ad43d79101a7c5de5158fe" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:42.085216 23121 raft_consensus.cc:2749] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:42.086723 22895 raft_consensus.cc:3060] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.065963 23137 raft_consensus.cc:515] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:42.090956 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086a7123d3ad43d79101a7c5de5158fe" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:42.091950 22818 raft_consensus.cc:2393] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 in current term 1: Already voted for candidate 27745ef236cd494684f62a1269411a68 in this term.
I20260709 13:21:42.092114 23137 leader_election.cc:290] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:42.093375 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e7b082dcf344fbf83e0ddec4b08b6de" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:42.092955 22714 catalog_manager.cc:5697] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:42.094013 22818 raft_consensus.cc:3060] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.097097 22934 leader_election.cc:304] T 086a7123d3ad43d79101a7c5de5158fe P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
W20260709 13:21:42.099077 22785 consensus_peers.cc:597] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 -> Peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Couldn't send request to peer 8ef3887fb3874f8b81079e542d252944. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:42.100952 23085 tablet_bootstrap.cc:654] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.102205 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e7b082dcf344fbf83e0ddec4b08b6de" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:42.105008 23195 consensus_queue.cc:1048] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:21:42.105479 22860 leader_election.cc:343] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.106436 23187 raft_consensus.cc:493] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.107064 23187 raft_consensus.cc:515] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:42.107839 23087 tablet_bootstrap.cc:654] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.108435 23184 raft_consensus.cc:3060] T 086a7123d3ad43d79101a7c5de5158fe P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.109516 23187 leader_election.cc:290] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:42.110291 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cabd9adbf414496bbaab3946b413d313" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:42.112797 22716 catalog_manager.cc:5697] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ef3887fb3874f8b81079e542d252944 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "8ef3887fb3874f8b81079e542d252944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:42.114758 23173 raft_consensus.cc:515] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:42.115981 23042 raft_consensus.cc:2393] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 in current term 1: Already voted for candidate 27745ef236cd494684f62a1269411a68 in this term.
W20260709 13:21:42.116433 23008 leader_election.cc:343] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.112015 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cabd9adbf414496bbaab3946b413d313" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:42.118309 22969 raft_consensus.cc:2468] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 0.
I20260709 13:21:42.118602 23173 leader_election.cc:290] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:42.121031 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e638dc505d4b04b802e91b8863d833" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:42.121670 22820 raft_consensus.cc:3060] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.125061 22968 raft_consensus.cc:2468] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:42.120270 22783 leader_election.cc:304] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:42.128268 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e638dc505d4b04b802e91b8863d833" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:42.133361 23086 tablet_bootstrap.cc:492] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:42.134025 23086 ts_tablet_manager.cc:1403] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.244s	user 0.029s	sys 0.051s
W20260709 13:21:42.137050 23008 leader_election.cc:343] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.138101 22785 leader_election.cc:304] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:42.139034 23130 raft_consensus.cc:2804] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:42.139593 23130 raft_consensus.cc:697] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:42.140563 23130 consensus_queue.cc:237] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:42.142438 23084 tablet_bootstrap.cc:492] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:42.142971 23084 ts_tablet_manager.cc:1403] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.197s	user 0.015s	sys 0.016s
I20260709 13:21:42.136857 23086 raft_consensus.cc:359] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:42.144093 23086 raft_consensus.cc:385] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.144493 23086 raft_consensus.cc:740] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.146044 23086 consensus_queue.cc:260] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:42.149823 23173 raft_consensus.cc:493] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.152365 23173 raft_consensus.cc:515] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:42.153512 23084 raft_consensus.cc:359] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:42.154261 23084 raft_consensus.cc:385] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.154439 23173 leader_election.cc:290] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:42.157022 22820 raft_consensus.cc:2468] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
I20260709 13:21:42.154532 23084 raft_consensus.cc:740] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.161722 23195 raft_consensus.cc:2749] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:42.161041 22818 raft_consensus.cc:2468] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:42.162228 23084 consensus_queue.cc:260] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:42.165441 23086 ts_tablet_manager.cc:1434] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 13:21:42.165475 22860 leader_election.cc:304] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:42.167276 23086 tablet_bootstrap.cc:492] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:42.169184 23084 ts_tablet_manager.cc:1434] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.001s
I20260709 13:21:42.170444 23084 tablet_bootstrap.cc:492] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:42.169242 23137 raft_consensus.cc:2804] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:42.171453 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:42.171660 23184 raft_consensus.cc:2749] T 086a7123d3ad43d79101a7c5de5158fe P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:42.171583 23137 raft_consensus.cc:697] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:42.173357 23085 tablet_bootstrap.cc:492] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:42.173009 23137 consensus_queue.cc:237] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:42.173887 23085 ts_tablet_manager.cc:1403] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.163s	user 0.009s	sys 0.007s
I20260709 13:21:42.184955 23008 leader_election.cc:304] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:42.186837 23187 raft_consensus.cc:2804] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:42.186761 23008 leader_election.cc:304] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:42.187120 23173 consensus_queue.cc:1048] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:42.188052 23094 raft_consensus.cc:2804] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:42.188493 23094 raft_consensus.cc:493] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:42.188862 23094 raft_consensus.cc:3060] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.189266 23085 raft_consensus.cc:359] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.187460 23187 raft_consensus.cc:697] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:42.183202 23084 tablet_bootstrap.cc:654] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.190096 23085 raft_consensus.cc:385] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.190585 23085 raft_consensus.cc:740] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.191656 22717 catalog_manager.cc:5697] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 1 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:42.195618 23187 consensus_queue.cc:237] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:42.201589 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:42.205041 23087 tablet_bootstrap.cc:492] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:42.205588 23087 ts_tablet_manager.cc:1403] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.168s	user 0.011s	sys 0.004s
I20260709 13:21:42.208305 23087 raft_consensus.cc:359] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:42.209002 23087 raft_consensus.cc:385] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.209283 23087 raft_consensus.cc:740] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.194808 23085 consensus_queue.cc:260] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.216584 23086 tablet_bootstrap.cc:654] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.217298 23087 consensus_queue.cc:260] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:42.221076 23194 consensus_queue.cc:1048] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:42.222152 23125 raft_consensus.cc:493] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.222781 23125 raft_consensus.cc:515] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.216917 23094 raft_consensus.cc:515] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:42.226991 23094 leader_election.cc:290] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:42.230840 23085 ts_tablet_manager.cc:1434] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.057s	user 0.003s	sys 0.004s
I20260709 13:21:42.232200 23085 tablet_bootstrap.cc:492] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:42.231556 22718 catalog_manager.cc:5697] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:42.188264 23008 leader_election.cc:343] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.236377 22718 catalog_manager.cc:5697] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:42.238605 23125 leader_election.cc:290] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:42.240922 23087 ts_tablet_manager.cc:1434] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 13:21:42.242605 23087 tablet_bootstrap.cc:492] T cabd9adbf414496bbaab3946b413d313 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
W20260709 13:21:42.234938 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 46717.3us
I20260709 13:21:42.250425 23087 tablet_bootstrap.cc:654] T cabd9adbf414496bbaab3946b413d313 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.255707 23094 consensus_queue.cc:1048] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:42.260804 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:42.261601 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cabd9adbf414496bbaab3946b413d313" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:42.265121 22967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cabd9adbf414496bbaab3946b413d313" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:42.267473 23086 tablet_bootstrap.cc:492] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:42.269045 23086 ts_tablet_manager.cc:1403] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.102s	user 0.015s	sys 0.020s
I20260709 13:21:42.269954 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:42.271809 23086 raft_consensus.cc:359] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.272617 23086 raft_consensus.cc:385] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.272884 23086 raft_consensus.cc:740] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.273586 23086 consensus_queue.cc:260] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.275818 23086 ts_tablet_manager.cc:1434] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:21:42.276901 23086 tablet_bootstrap.cc:492] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:42.277415 22967 raft_consensus.cc:3060] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:42.286612 23008 leader_election.cc:343] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.287381 23008 leader_election.cc:304] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:42.288585 23125 raft_consensus.cc:2749] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:21:42.290064 23008 leader_election.cc:343] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.292935 23194 raft_consensus.cc:493] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.293578 23194 raft_consensus.cc:515] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:42.295647 23085 tablet_bootstrap.cc:654] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.296149 23086 tablet_bootstrap.cc:654] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.297319 23094 consensus_queue.cc:1048] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:42.298164 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955a6d8dbbe94ef9be9c1f69b10675d8" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:42.298894 22969 raft_consensus.cc:2468] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
I20260709 13:21:42.295909 23194 leader_election.cc:290] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:42.303764 22785 leader_election.cc:304] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:42.305071 23194 raft_consensus.cc:2804] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:42.305493 23194 raft_consensus.cc:493] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:42.305824 23194 raft_consensus.cc:3060] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.310408 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955a6d8dbbe94ef9be9c1f69b10675d8" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:42.311794 22895 raft_consensus.cc:2468] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
I20260709 13:21:42.312680 22967 raft_consensus.cc:2468] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
I20260709 13:21:42.325466 23184 consensus_queue.cc:1048] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:42.327641 22718 catalog_manager.cc:5697] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:42.330619 23194 raft_consensus.cc:515] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
W20260709 13:21:42.331507 23008 leader_election.cc:343] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:42.332623 23008 leader_election.cc:343] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.333037 23008 leader_election.cc:304] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:42.335556 23184 raft_consensus.cc:493] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.336114 23184 raft_consensus.cc:515] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:42.339064 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e638dc505d4b04b802e91b8863d833" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:42.341907 23187 raft_consensus.cc:2749] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:42.346653 23084 tablet_bootstrap.cc:492] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:42.347332 23084 ts_tablet_manager.cc:1403] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.177s	user 0.029s	sys 0.028s
I20260709 13:21:42.354893 23086 tablet_bootstrap.cc:492] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:42.356832 23086 ts_tablet_manager.cc:1403] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.080s	user 0.014s	sys 0.006s
I20260709 13:21:42.361303 23086 raft_consensus.cc:359] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.364816 23086 raft_consensus.cc:385] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.365098 23086 raft_consensus.cc:740] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.367403 23210 raft_consensus.cc:493] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.367928 23210 raft_consensus.cc:515] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:42.367856 23086 consensus_queue.cc:260] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.368232 22714 catalog_manager.cc:5697] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 1 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:42.370285 23210 leader_election.cc:290] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:42.370172 23091 consensus_queue.cc:1048] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:42.371465 23084 raft_consensus.cc:359] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:42.372162 23084 raft_consensus.cc:385] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.372499 23084 raft_consensus.cc:740] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.373162 23084 consensus_queue.cc:260] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:42.380333 23094 consensus_queue.cc:1048] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:42.372119 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ecf8a5022f43c787aaff1cf7679264" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:42.364320 23184 leader_election.cc:290] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:42.388111 23087 tablet_bootstrap.cc:492] T cabd9adbf414496bbaab3946b413d313 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:42.388783 23087 ts_tablet_manager.cc:1403] T cabd9adbf414496bbaab3946b413d313 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.146s	user 0.029s	sys 0.010s
W20260709 13:21:42.382766 22858 leader_election.cc:343] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.390327 23091 raft_consensus.cc:493] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.390923 23091 raft_consensus.cc:515] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.391549 23194 leader_election.cc:290] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:42.393158 23091 leader_election.cc:290] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:42.393754 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ecf8a5022f43c787aaff1cf7679264" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:42.396111 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e638dc505d4b04b802e91b8863d833" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:42.396944 22818 raft_consensus.cc:2393] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 in current term 1: Already voted for candidate 27745ef236cd494684f62a1269411a68 in this term.
I20260709 13:21:42.397956 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955a6d8dbbe94ef9be9c1f69b10675d8" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:42.398591 22968 raft_consensus.cc:3060] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.396656 23008 leader_election.cc:304] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:42.406296 23086 ts_tablet_manager.cc:1434] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.000s
I20260709 13:21:42.415449 23084 ts_tablet_manager.cc:1434] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.068s	user 0.013s	sys 0.003s
I20260709 13:21:42.416710 23084 tablet_bootstrap.cc:492] T 5e7b082dcf344fbf83e0ddec4b08b6de P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:42.423736 23086 tablet_bootstrap.cc:492] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:42.424773 23094 raft_consensus.cc:2804] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:42.428448 23094 raft_consensus.cc:697] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:42.429375 23094 consensus_queue.cc:237] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:42.430455 23085 tablet_bootstrap.cc:492] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:42.431057 23085 ts_tablet_manager.cc:1403] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.199s	user 0.023s	sys 0.018s
I20260709 13:21:42.431406 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955a6d8dbbe94ef9be9c1f69b10675d8" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:42.432142 22895 raft_consensus.cc:3060] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.434396 23085 raft_consensus.cc:359] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.435259 23085 raft_consensus.cc:385] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.435585 23085 raft_consensus.cc:740] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.436465 23085 consensus_queue.cc:260] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.438959 23085 ts_tablet_manager.cc:1434] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
W20260709 13:21:42.405478 22860 leader_election.cc:343] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.441008 23085 tablet_bootstrap.cc:492] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:42.447045 23087 raft_consensus.cc:359] T cabd9adbf414496bbaab3946b413d313 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:42.448589 23087 raft_consensus.cc:385] T cabd9adbf414496bbaab3946b413d313 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.448868 23087 raft_consensus.cc:740] T cabd9adbf414496bbaab3946b413d313 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.442405 22860 leader_election.cc:304] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
W20260709 13:21:42.450690 22783 consensus_peers.cc:597] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 -> Peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509): Couldn't send request to peer 27745ef236cd494684f62a1269411a68. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:42.450861 22860 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 46858.1us
I20260709 13:21:42.451534 23210 raft_consensus.cc:2749] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:42.439989 22934 leader_election.cc:304] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:42.453768 23084 tablet_bootstrap.cc:654] T 5e7b082dcf344fbf83e0ddec4b08b6de P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.460954 23087 consensus_queue.cc:260] T cabd9adbf414496bbaab3946b413d313 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:42.464550 23222 raft_consensus.cc:3060] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.466226 23086 tablet_bootstrap.cc:654] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.467480 23085 tablet_bootstrap.cc:654] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.464385 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b154956cc17f4ba68f906daffbcedff6" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:42.476430 23130 consensus_queue.cc:1048] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:21:42.479027 22785 leader_election.cc:343] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.445657 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b154956cc17f4ba68f906daffbcedff6" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:42.480643 23042 raft_consensus.cc:2468] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
I20260709 13:21:42.483505 22783 leader_election.cc:304] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:42.484608 23094 consensus_queue.cc:1048] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:42.489284 23220 raft_consensus.cc:493] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.484930 23130 raft_consensus.cc:2804] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:42.489858 23220 raft_consensus.cc:515] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:42.490427 23130 raft_consensus.cc:493] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:42.490918 23130 raft_consensus.cc:3060] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.492205 23220 leader_election.cc:290] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:42.494545 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505eab84b06f46ee9d336abf01353dee" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:42.501030 23220 consensus_queue.cc:1048] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:42.501330 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505eab84b06f46ee9d336abf01353dee" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:42.473762 23222 raft_consensus.cc:2749] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:42.516337 23087 ts_tablet_manager.cc:1434] T cabd9adbf414496bbaab3946b413d313 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.127s	user 0.008s	sys 0.000s
I20260709 13:21:42.505730 22969 raft_consensus.cc:3060] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.503147 22934 leader_election.cc:304] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:42.525291 23087 tablet_bootstrap.cc:492] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:42.528227 22718 catalog_manager.cc:5697] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 1 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:42.501979 22968 raft_consensus.cc:2468] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:42.501526 22895 raft_consensus.cc:2468] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:42.507084 23125 consensus_queue.cc:1048] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 13:21:42.535866 22785 leader_election.cc:304] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:42.539849 23087 tablet_bootstrap.cc:654] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.544955 23091 raft_consensus.cc:2804] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:42.551191 23091 raft_consensus.cc:697] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:42.552157 23091 consensus_queue.cc:237] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:42.544521 23130 raft_consensus.cc:515] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.595211 23130 leader_election.cc:290] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:42.598850 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b154956cc17f4ba68f906daffbcedff6" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:42.599351 23085 tablet_bootstrap.cc:492] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:42.599540 23042 raft_consensus.cc:3060] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.599861 23085 ts_tablet_manager.cc:1403] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.160s	user 0.019s	sys 0.013s
I20260709 13:21:42.601079 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b154956cc17f4ba68f906daffbcedff6" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:42.603494 23086 tablet_bootstrap.cc:492] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:42.604033 23086 ts_tablet_manager.cc:1403] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.197s	user 0.030s	sys 0.028s
W20260709 13:21:42.604488 22785 leader_election.cc:343] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.607161 23086 raft_consensus.cc:359] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:42.607834 23086 raft_consensus.cc:385] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.608048 23086 raft_consensus.cc:740] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.610740 23086 consensus_queue.cc:260] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:42.616783 23222 raft_consensus.cc:2749] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:42.618767 23086 ts_tablet_manager.cc:1434] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.002s
I20260709 13:21:42.614768 23130 consensus_queue.cc:1048] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:21:42.630571 23085 raft_consensus.cc:359] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:42.637041 23085 raft_consensus.cc:385] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.637337 23085 raft_consensus.cc:740] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.638459 23085 consensus_queue.cc:260] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:42.642321 23085 ts_tablet_manager.cc:1434] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.042s	user 0.009s	sys 0.000s
I20260709 13:21:42.643316 23085 tablet_bootstrap.cc:492] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:42.644100 23086 tablet_bootstrap.cc:492] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
W20260709 13:21:42.660454 22860 consensus_peers.cc:597] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Couldn't send request to peer 8ef3887fb3874f8b81079e542d252944. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:42.660652 23084 tablet_bootstrap.cc:492] T 5e7b082dcf344fbf83e0ddec4b08b6de P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:42.661304 23084 ts_tablet_manager.cc:1403] T 5e7b082dcf344fbf83e0ddec4b08b6de P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.245s	user 0.029s	sys 0.021s
I20260709 13:21:42.663198 23125 consensus_queue.cc:1048] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:42.664130 23084 raft_consensus.cc:359] T 5e7b082dcf344fbf83e0ddec4b08b6de P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:42.664865 23084 raft_consensus.cc:385] T 5e7b082dcf344fbf83e0ddec4b08b6de P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.665123 23084 raft_consensus.cc:740] T 5e7b082dcf344fbf83e0ddec4b08b6de P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.669127 23210 consensus_queue.cc:1048] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:42.674507 23205 consensus_queue.cc:1048] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:42.680078 23042 raft_consensus.cc:2468] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:42.698585 22783 leader_election.cc:304] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:42.699580 23091 raft_consensus.cc:2804] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:42.700096 23091 raft_consensus.cc:697] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:42.700980 23091 consensus_queue.cc:237] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.701946 23084 consensus_queue.cc:260] T 5e7b082dcf344fbf83e0ddec4b08b6de P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:42.709045 23187 consensus_queue.cc:1048] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:42.709486 23085 tablet_bootstrap.cc:654] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.711120 23084 ts_tablet_manager.cc:1434] T 5e7b082dcf344fbf83e0ddec4b08b6de P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.050s	user 0.009s	sys 0.001s
I20260709 13:21:42.712129 23084 tablet_bootstrap.cc:492] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:42.715492 23087 tablet_bootstrap.cc:492] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:42.716959 23087 ts_tablet_manager.cc:1403] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.193s	user 0.022s	sys 0.012s
I20260709 13:21:42.722565 23205 raft_consensus.cc:493] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.723114 23205 raft_consensus.cc:515] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.725279 23205 leader_election.cc:290] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:42.726486 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10066ccf8d174f4a9dd69253613857b8" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
W20260709 13:21:42.728715 22785 leader_election.cc:343] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.731623 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10066ccf8d174f4a9dd69253613857b8" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
W20260709 13:21:42.734262 22783 leader_election.cc:343] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.734819 22783 leader_election.cc:304] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:42.751001 23210 raft_consensus.cc:493] T cabd9adbf414496bbaab3946b413d313 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.751596 23210 raft_consensus.cc:515] T cabd9adbf414496bbaab3946b413d313 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:42.754122 23085 tablet_bootstrap.cc:492] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:42.755007 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cabd9adbf414496bbaab3946b413d313" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:42.755149 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cabd9adbf414496bbaab3946b413d313" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:42.755990 22966 raft_consensus.cc:2393] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 in current term 1: Already voted for candidate 27745ef236cd494684f62a1269411a68 in this term.
I20260709 13:21:42.738145 23091 raft_consensus.cc:2749] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:42.756016 23085 ts_tablet_manager.cc:1403] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.113s	user 0.010s	sys 0.007s
I20260709 13:21:42.753756 23210 leader_election.cc:290] T cabd9adbf414496bbaab3946b413d313 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:42.760262 22858 leader_election.cc:304] T cabd9adbf414496bbaab3946b413d313 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:42.761763 23086 tablet_bootstrap.cc:654] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.767666 23210 raft_consensus.cc:3060] T cabd9adbf414496bbaab3946b413d313 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.769157 23084 tablet_bootstrap.cc:654] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.774993 23210 raft_consensus.cc:2749] T cabd9adbf414496bbaab3946b413d313 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:42.769013 23085 raft_consensus.cc:359] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:42.777841 23085 raft_consensus.cc:385] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.778172 23085 raft_consensus.cc:740] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.779241 23085 consensus_queue.cc:260] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:42.782651 23085 ts_tablet_manager.cc:1434] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:21:42.783422 23085 tablet_bootstrap.cc:492] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
W20260709 13:21:42.787618 22766 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:42.792606 23087 raft_consensus.cc:359] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.794811 23087 raft_consensus.cc:385] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.795176 23087 raft_consensus.cc:740] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.796267 23087 consensus_queue.cc:260] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.800359 23087 ts_tablet_manager.cc:1434] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.083s	user 0.005s	sys 0.002s
I20260709 13:21:42.801388 23087 tablet_bootstrap.cc:492] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:42.807849 23087 tablet_bootstrap.cc:654] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.812148 23084 tablet_bootstrap.cc:492] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:42.812680 23084 ts_tablet_manager.cc:1403] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.101s	user 0.013s	sys 0.007s
I20260709 13:21:42.815456 23085 tablet_bootstrap.cc:654] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.815843 23087 tablet_bootstrap.cc:492] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:42.816396 23087 ts_tablet_manager.cc:1403] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.001s
W20260709 13:21:42.821803 22785 consensus_peers.cc:597] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 -> Peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Couldn't send request to peer 49cbf11e18ce4268a7fcca76872afb23. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:42.830534 23084 raft_consensus.cc:359] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:42.831573 23084 raft_consensus.cc:385] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.831928 23084 raft_consensus.cc:740] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.832823 23084 consensus_queue.cc:260] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:42.826815 23085 tablet_bootstrap.cc:492] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:42.835795 23086 tablet_bootstrap.cc:492] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:42.836086 23085 ts_tablet_manager.cc:1403] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.053s	user 0.016s	sys 0.000s
I20260709 13:21:42.836349 23086 ts_tablet_manager.cc:1403] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.192s	user 0.018s	sys 0.018s
I20260709 13:21:42.819448 23087 raft_consensus.cc:359] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:42.838442 23087 raft_consensus.cc:385] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.838768 23087 raft_consensus.cc:740] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.839071 23085 raft_consensus.cc:359] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:42.839584 23087 consensus_queue.cc:260] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:42.839398 22714 catalog_manager.cc:5697] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:42.840828 23086 raft_consensus.cc:359] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:42.841591 23086 raft_consensus.cc:385] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.841866 23086 raft_consensus.cc:740] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.842242 23085 raft_consensus.cc:385] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.842664 23085 raft_consensus.cc:740] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.842731 23086 consensus_queue.cc:260] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:42.842145 23087 ts_tablet_manager.cc:1434] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.001s
I20260709 13:21:42.843482 23085 consensus_queue.cc:260] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:42.844625 23087 tablet_bootstrap.cc:492] T 10066ccf8d174f4a9dd69253613857b8 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:42.845189 23086 ts_tablet_manager.cc:1434] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:21:42.846186 23086 tablet_bootstrap.cc:492] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:42.842136 22714 catalog_manager.cc:5697] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:42.847716 23084 ts_tablet_manager.cc:1434] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 13:21:42.848867 23130 raft_consensus.cc:493] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.857443 23130 raft_consensus.cc:515] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:42.859361 23130 leader_election.cc:290] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:42.849395 23085 ts_tablet_manager.cc:1434] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:21:42.860471 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:42.860651 22964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:42.855458 23086 tablet_bootstrap.cc:654] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.855937 23087 tablet_bootstrap.cc:654] T 10066ccf8d174f4a9dd69253613857b8 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:42.862284 22785 leader_election.cc:343] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.862833 23084 tablet_bootstrap.cc:492] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
W20260709 13:21:42.865933 22785 leader_election.cc:343] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.866489 22785 leader_election.cc:304] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:42.866928 23087 tablet_bootstrap.cc:492] T 10066ccf8d174f4a9dd69253613857b8 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:42.867451 23087 ts_tablet_manager.cc:1403] T 10066ccf8d174f4a9dd69253613857b8 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.000s
I20260709 13:21:42.867740 23130 raft_consensus.cc:2749] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:42.869174 23084 tablet_bootstrap.cc:654] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.870283 23087 raft_consensus.cc:359] T 10066ccf8d174f4a9dd69253613857b8 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.870990 23087 raft_consensus.cc:385] T 10066ccf8d174f4a9dd69253613857b8 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.871272 23087 raft_consensus.cc:740] T 10066ccf8d174f4a9dd69253613857b8 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.861078 23085 tablet_bootstrap.cc:492] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:42.872025 23087 consensus_queue.cc:260] T 10066ccf8d174f4a9dd69253613857b8 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.873311 23187 consensus_queue.cc:1048] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:42.874377 23087 ts_tablet_manager.cc:1434] T 10066ccf8d174f4a9dd69253613857b8 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:21:42.875399 23220 raft_consensus.cc:493] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.875901 23220 raft_consensus.cc:515] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:42.878031 23220 leader_election.cc:290] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:42.878234 23085 tablet_bootstrap.cc:654] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.879750 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0957b803701e45648f538789ec62e507" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:42.880446 23042 raft_consensus.cc:2468] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:42.881814 22932 leader_election.cc:304] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:42.884438 23222 raft_consensus.cc:493] T 5e7b082dcf344fbf83e0ddec4b08b6de P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.884966 23222 raft_consensus.cc:515] T 5e7b082dcf344fbf83e0ddec4b08b6de P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:42.887005 23222 leader_election.cc:290] T 5e7b082dcf344fbf83e0ddec4b08b6de P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:42.887553 23222 raft_consensus.cc:2804] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:42.887887 23222 raft_consensus.cc:493] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:42.888206 23222 raft_consensus.cc:3060] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:42.890142 23085 tablet_bootstrap.cc:492] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:42.890681 23085 ts_tablet_manager.cc:1403] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.000s
I20260709 13:21:42.891081 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0957b803701e45648f538789ec62e507" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:42.891767 22894 raft_consensus.cc:2468] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:42.893858 23187 consensus_queue.cc:1048] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:42.895610 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e7b082dcf344fbf83e0ddec4b08b6de" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:42.898227 23222 raft_consensus.cc:515] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:42.898430 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e7b082dcf344fbf83e0ddec4b08b6de" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:42.875384 23087 tablet_bootstrap.cc:492] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:42.897778 23084 tablet_bootstrap.cc:492] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:42.899682 23084 ts_tablet_manager.cc:1403] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.000s
I20260709 13:21:42.901418 22934 leader_election.cc:304] T 5e7b082dcf344fbf83e0ddec4b08b6de P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:42.893383 23085 raft_consensus.cc:359] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.906549 23220 raft_consensus.cc:2749] T 5e7b082dcf344fbf83e0ddec4b08b6de P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:42.907202 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0957b803701e45648f538789ec62e507" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:42.907872 22894 raft_consensus.cc:3060] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:42.908581 23084 raft_consensus.cc:359] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:42.909137 23084 raft_consensus.cc:385] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.909348 23084 raft_consensus.cc:740] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.907356 23085 raft_consensus.cc:385] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.909864 23084 consensus_queue.cc:260] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:42.910305 23085 raft_consensus.cc:740] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.912304 23084 ts_tablet_manager.cc:1434] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:21:42.911950 23085 consensus_queue.cc:260] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.913421 23084 tablet_bootstrap.cc:492] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:42.915266 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0957b803701e45648f538789ec62e507" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:42.915931 23042 raft_consensus.cc:3060] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:42.915292 23085 ts_tablet_manager.cc:1434] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 13:21:42.928609 23085 tablet_bootstrap.cc:492] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:42.935225 23084 tablet_bootstrap.cc:654] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.935674 23042 raft_consensus.cc:2468] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 2.
I20260709 13:21:42.937332 23087 tablet_bootstrap.cc:654] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.937199 22932 leader_election.cc:304] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:42.938333 23220 raft_consensus.cc:2804] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:42.938887 23220 raft_consensus.cc:697] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 2 LEADER]: Becoming Leader. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:21:42.926563 23222 leader_election.cc:290] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:42.939836 23220 consensus_queue.cc:237] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:42.944412 23086 tablet_bootstrap.cc:492] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:42.944707 23187 raft_consensus.cc:493] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.945263 23187 raft_consensus.cc:515] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:42.948308 22964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:42.948856 23187 leader_election.cc:290] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:42.951819 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:42.953909 23008 leader_election.cc:304] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:42.954628 22894 raft_consensus.cc:2468] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 2.
I20260709 13:21:42.954870 23227 raft_consensus.cc:2749] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:42.956460 23085 tablet_bootstrap.cc:654] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.944999 23086 ts_tablet_manager.cc:1403] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.099s	user 0.029s	sys 0.018s
I20260709 13:21:42.960842 23086 raft_consensus.cc:359] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:42.961534 23086 raft_consensus.cc:385] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.961826 23086 raft_consensus.cc:740] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.962610 23086 consensus_queue.cc:260] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:42.965098 23085 tablet_bootstrap.cc:492] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:42.965668 23085 ts_tablet_manager.cc:1403] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.000s
I20260709 13:21:42.968326 23085 raft_consensus.cc:359] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.969048 23085 raft_consensus.cc:385] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.969323 23085 raft_consensus.cc:740] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.970002 23085 consensus_queue.cc:260] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.974737 23085 ts_tablet_manager.cc:1434] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:21:42.975726 23085 tablet_bootstrap.cc:492] T be220d19020b428a8b1cc356f32ff5ad P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:42.976522 23087 tablet_bootstrap.cc:492] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:42.977069 23087 ts_tablet_manager.cc:1403] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.102s	user 0.011s	sys 0.000s
I20260709 13:21:42.979698 23042 raft_consensus.cc:3060] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.981225 23085 tablet_bootstrap.cc:654] T be220d19020b428a8b1cc356f32ff5ad P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.981614 23087 raft_consensus.cc:359] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.982373 23087 raft_consensus.cc:385] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.982645 23087 raft_consensus.cc:740] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.983624 22714 catalog_manager.cc:5697] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 0 to 2, leader changed from <none> to 8ef3887fb3874f8b81079e542d252944 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "8ef3887fb3874f8b81079e542d252944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:42.985319 23084 tablet_bootstrap.cc:492] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:42.985860 23084 ts_tablet_manager.cc:1403] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.073s	user 0.010s	sys 0.002s
I20260709 13:21:42.987962 23086 ts_tablet_manager.cc:1434] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.000s
I20260709 13:21:42.988515 23084 raft_consensus.cc:359] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:42.989238 23084 raft_consensus.cc:385] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.989513 23084 raft_consensus.cc:740] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.990367 23084 consensus_queue.cc:260] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:42.991406 23087 consensus_queue.cc:260] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:42.998050 23084 ts_tablet_manager.cc:1434] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:21:42.998929 23087 ts_tablet_manager.cc:1434] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:21:43.000005 23087 tablet_bootstrap.cc:492] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:43.001323 23085 tablet_bootstrap.cc:492] T be220d19020b428a8b1cc356f32ff5ad P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:43.001895 23085 ts_tablet_manager.cc:1403] T be220d19020b428a8b1cc356f32ff5ad P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.000s
I20260709 13:21:43.004412 23084 tablet_bootstrap.cc:492] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:43.005026 23085 raft_consensus.cc:359] T be220d19020b428a8b1cc356f32ff5ad P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:43.005818 23085 raft_consensus.cc:385] T be220d19020b428a8b1cc356f32ff5ad P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.009203 23085 raft_consensus.cc:740] T be220d19020b428a8b1cc356f32ff5ad P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.009884 23085 consensus_queue.cc:260] T be220d19020b428a8b1cc356f32ff5ad P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:43.012009 23085 ts_tablet_manager.cc:1434] T be220d19020b428a8b1cc356f32ff5ad P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.010s	user 0.001s	sys 0.004s
I20260709 13:21:43.012944 23085 tablet_bootstrap.cc:492] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:43.016510 23086 tablet_bootstrap.cc:492] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:43.018662 23085 tablet_bootstrap.cc:654] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.020289 23084 tablet_bootstrap.cc:654] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.022887 23086 tablet_bootstrap.cc:654] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.013774 23087 tablet_bootstrap.cc:654] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.030201 23085 tablet_bootstrap.cc:492] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:43.030843 23085 ts_tablet_manager.cc:1403] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.008s
I20260709 13:21:43.031528 23220 raft_consensus.cc:493] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.032058 23220 raft_consensus.cc:515] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.033782 23085 raft_consensus.cc:359] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:43.034582 23085 raft_consensus.cc:385] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.034898 23085 raft_consensus.cc:740] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.035655 23085 consensus_queue.cc:260] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:43.038600 23085 ts_tablet_manager.cc:1434] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:21:43.039680 23085 tablet_bootstrap.cc:492] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:43.035079 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:43.041478 23042 raft_consensus.cc:2468] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 0.
I20260709 13:21:43.042528 23130 raft_consensus.cc:493] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.043041 22932 leader_election.cc:304] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:43.043072 23130 raft_consensus.cc:515] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.044643 23220 leader_election.cc:290] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:43.045521 23220 raft_consensus.cc:2804] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:43.045584 23130 leader_election.cc:290] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:43.051003 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:43.051664 23042 raft_consensus.cc:2468] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
I20260709 13:21:43.051882 23086 tablet_bootstrap.cc:492] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:43.052408 23086 ts_tablet_manager.cc:1403] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.036s	user 0.010s	sys 0.005s
I20260709 13:21:43.045986 23220 raft_consensus.cc:493] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:43.053102 23220 raft_consensus.cc:3060] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.053027 22783 leader_election.cc:304] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:43.054368 23130 raft_consensus.cc:2804] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:43.054757 23130 raft_consensus.cc:493] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:43.054987 23086 raft_consensus.cc:359] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.055768 23086 raft_consensus.cc:385] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.056156 23086 raft_consensus.cc:740] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.055085 23130 raft_consensus.cc:3060] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.057014 23086 consensus_queue.cc:260] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.060508 23220 raft_consensus.cc:515] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.048439 23085 tablet_bootstrap.cc:654] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.048177 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:43.049451 22964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:43.062243 22964 raft_consensus.cc:2393] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 in current term 1: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:43.062778 23220 leader_election.cc:290] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:43.064677 23084 tablet_bootstrap.cc:492] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:43.067309 23084 ts_tablet_manager.cc:1403] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.063s	user 0.016s	sys 0.000s
I20260709 13:21:43.066828 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:43.066030 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:43.068917 23042 raft_consensus.cc:3060] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.069310 23087 tablet_bootstrap.cc:492] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:43.070353 23087 ts_tablet_manager.cc:1403] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.071s	user 0.014s	sys 0.001s
I20260709 13:21:43.070143 23084 raft_consensus.cc:359] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:43.070851 23084 raft_consensus.cc:385] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.071139 23084 raft_consensus.cc:740] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.071841 23084 consensus_queue.cc:260] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:43.072983 23087 raft_consensus.cc:359] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:43.073671 23087 raft_consensus.cc:385] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.073861 23084 ts_tablet_manager.cc:1434] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:21:43.073944 23087 raft_consensus.cc:740] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.074972 23087 consensus_queue.cc:260] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:43.076886 23084 tablet_bootstrap.cc:492] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:43.077286 23130 raft_consensus.cc:515] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.079458 23130 leader_election.cc:290] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:43.079711 22817 raft_consensus.cc:2393] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 in current term 1: Already voted for candidate 91f33bdbdd63438294071eb3776db679 in this term.
I20260709 13:21:43.080935 23085 tablet_bootstrap.cc:492] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:43.081473 23085 ts_tablet_manager.cc:1403] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.042s	user 0.009s	sys 0.002s
I20260709 13:21:43.083477 23087 ts_tablet_manager.cc:1434] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 13:21:43.083536 23084 tablet_bootstrap.cc:654] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.084987 23087 tablet_bootstrap.cc:492] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:43.086670 23222 raft_consensus.cc:493] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.086764 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:43.087265 23222 raft_consensus.cc:515] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:43.089376 23222 leader_election.cc:290] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:43.066681 23086 ts_tablet_manager.cc:1434] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:21:43.089859 23222 raft_consensus.cc:493] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.090355 23222 raft_consensus.cc:515] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.091183 22964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:43.092070 22964 raft_consensus.cc:2393] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 in current term 1: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:43.093142 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e230e64c6984b72a2ba543f2a017e25" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:43.094547 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
W20260709 13:21:43.094676 22932 leader_election.cc:343] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:43.097029 23222 leader_election.cc:290] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:43.101130 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:43.101524 23187 raft_consensus.cc:493] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.101871 23087 tablet_bootstrap.cc:654] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.104866 23086 tablet_bootstrap.cc:492] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:43.102042 23187 raft_consensus.cc:515] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:43.107264 23187 leader_election.cc:290] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:43.106530 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e230e64c6984b72a2ba543f2a017e25" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:43.099359 23085 raft_consensus.cc:359] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:43.119330 23085 raft_consensus.cc:385] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.120286 22820 raft_consensus.cc:2468] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 0.
I20260709 13:21:43.122655 22964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968e16cef40146509d654274b9fd2687" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:43.123505 22964 raft_consensus.cc:2468] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 0.
I20260709 13:21:43.116652 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968e16cef40146509d654274b9fd2687" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:43.119693 23085 raft_consensus.cc:740] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.127179 23085 consensus_queue.cc:260] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:43.129429 23085 ts_tablet_manager.cc:1434] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.048s	user 0.004s	sys 0.001s
I20260709 13:21:43.130575 23085 tablet_bootstrap.cc:492] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:43.126228 23008 leader_election.cc:304] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
W20260709 13:21:43.131857 23008 leader_election.cc:343] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:43.133200 23125 raft_consensus.cc:2804] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:43.133824 23125 raft_consensus.cc:493] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:43.134199 23125 raft_consensus.cc:3060] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.133319 23210 raft_consensus.cc:493] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.135500 23210 raft_consensus.cc:515] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:43.138821 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be220d19020b428a8b1cc356f32ff5ad" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:43.139398 23085 tablet_bootstrap.cc:654] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.139518 23041 raft_consensus.cc:2468] T be220d19020b428a8b1cc356f32ff5ad P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 0.
I20260709 13:21:43.141243 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be220d19020b428a8b1cc356f32ff5ad" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:43.142457 22858 leader_election.cc:304] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:43.143356 23096 raft_consensus.cc:2804] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:43.143932 23096 raft_consensus.cc:493] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:43.144296 23096 raft_consensus.cc:3060] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.139472 23210 leader_election.cc:290] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:43.159685 23086 tablet_bootstrap.cc:654] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.158820 23125 raft_consensus.cc:515] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
W20260709 13:21:43.148113 22860 leader_election.cc:343] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:43.162186 23084 tablet_bootstrap.cc:492] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:43.149307 23087 tablet_bootstrap.cc:492] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:43.143291 22785 leader_election.cc:304] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:43.163647 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968e16cef40146509d654274b9fd2687" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:43.163568 23130 raft_consensus.cc:2749] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:43.152343 23042 raft_consensus.cc:2468] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:43.162937 23087 ts_tablet_manager.cc:1403] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.078s	user 0.020s	sys 0.020s
I20260709 13:21:43.165822 22932 leader_election.cc:304] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:43.167167 23220 raft_consensus.cc:2804] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:43.167719 23220 raft_consensus.cc:697] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 1 LEADER]: Becoming Leader. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:21:43.167512 23087 raft_consensus.cc:359] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:43.168330 23087 raft_consensus.cc:385] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.168625 23087 raft_consensus.cc:740] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
W20260709 13:21:43.168869 23008 leader_election.cc:343] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:43.168684 23220 consensus_queue.cc:237] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.170017 22934 leader_election.cc:304] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:43.169330 23087 consensus_queue.cc:260] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:43.170845 23085 tablet_bootstrap.cc:492] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:43.171162 23222 raft_consensus.cc:2749] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:43.171830 23085 ts_tablet_manager.cc:1403] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.042s	user 0.016s	sys 0.018s
I20260709 13:21:43.172282 22934 leader_election.cc:304] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68
I20260709 13:21:43.173592 23222 raft_consensus.cc:2804] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:43.174047 23222 raft_consensus.cc:493] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:43.174408 23222 raft_consensus.cc:3060] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.176486 23085 raft_consensus.cc:359] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.180130 23086 tablet_bootstrap.cc:492] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:43.180943 23086 ts_tablet_manager.cc:1403] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.076s	user 0.010s	sys 0.003s
I20260709 13:21:43.167742 23084 ts_tablet_manager.cc:1403] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.091s	user 0.031s	sys 0.010s
I20260709 13:21:43.183660 22964 raft_consensus.cc:3060] T 5e7b082dcf344fbf83e0ddec4b08b6de P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.184154 23085 raft_consensus.cc:385] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.184546 23085 raft_consensus.cc:740] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.184948 23087 ts_tablet_manager.cc:1434] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:21:43.185856 23125 leader_election.cc:290] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:43.185355 23085 consensus_queue.cc:260] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.185011 23084 raft_consensus.cc:359] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:43.187450 23084 raft_consensus.cc:385] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.187736 23084 raft_consensus.cc:740] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.183828 23086 raft_consensus.cc:359] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:43.188665 23086 raft_consensus.cc:385] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.188987 23086 raft_consensus.cc:740] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.189683 23086 consensus_queue.cc:260] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:43.191251 23086 ts_tablet_manager.cc:1434] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:21:43.192970 23086 tablet_bootstrap.cc:492] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:43.193156 23084 consensus_queue.cc:260] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:43.191977 23085 ts_tablet_manager.cc:1434] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:21:43.194729 23087 tablet_bootstrap.cc:492] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:43.195194 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968e16cef40146509d654274b9fd2687" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:43.194734 23085 tablet_bootstrap.cc:492] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:43.195859 22966 raft_consensus.cc:3060] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.196895 23084 ts_tablet_manager.cc:1434] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:21:43.197956 23084 tablet_bootstrap.cc:492] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:43.206238 23086 tablet_bootstrap.cc:654] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.209663 23087 tablet_bootstrap.cc:654] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.211854 23084 tablet_bootstrap.cc:654] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.214694 23085 tablet_bootstrap.cc:654] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.217126 22966 raft_consensus.cc:2468] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
I20260709 13:21:43.218942 23008 leader_election.cc:304] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:43.219839 23125 raft_consensus.cc:493] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.220302 23227 raft_consensus.cc:2804] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:43.220317 23125 raft_consensus.cc:515] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:43.218315 23086 tablet_bootstrap.cc:492] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:43.222404 23086 ts_tablet_manager.cc:1403] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.007s
I20260709 13:21:43.223444 23125 leader_election.cc:290] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:43.225663 23091 consensus_queue.cc:1048] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:43.229277 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10066ccf8d174f4a9dd69253613857b8" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:43.229883 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10066ccf8d174f4a9dd69253613857b8" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:43.230072 22817 raft_consensus.cc:2468] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 0.
I20260709 13:21:43.230619 22894 raft_consensus.cc:2468] T 10066ccf8d174f4a9dd69253613857b8 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 0.
I20260709 13:21:43.231483 23182 raft_consensus.cc:493] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.231990 23182 raft_consensus.cc:515] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:43.233831 23210 raft_consensus.cc:493] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.234390 23210 raft_consensus.cc:515] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:43.225613 23086 raft_consensus.cc:359] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:43.236557 23086 raft_consensus.cc:385] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.236888 23086 raft_consensus.cc:740] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.237283 23085 tablet_bootstrap.cc:492] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:43.237610 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:43.237908 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:43.237800 23085 ts_tablet_manager.cc:1403] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.043s	user 0.014s	sys 0.001s
I20260709 13:21:43.238584 23041 raft_consensus.cc:2468] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 0.
I20260709 13:21:43.237663 23086 consensus_queue.cc:260] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
W20260709 13:21:43.239737 22860 leader_election.cc:343] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:43.240852 23091 raft_consensus.cc:493] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.241494 23091 raft_consensus.cc:515] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.234333 23182 leader_election.cc:290] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:43.244531 22967 raft_consensus.cc:3060] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.245127 23121 raft_consensus.cc:493] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.244686 23091 leader_election.cc:290] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:43.245718 23121 raft_consensus.cc:515] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:43.249790 23086 ts_tablet_manager.cc:1434] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.002s
I20260709 13:21:43.251587 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:43.252364 22817 raft_consensus.cc:2468] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 0.
I20260709 13:21:43.245785 23008 leader_election.cc:304] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:43.254457 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ea5c83aada45d3a98027069e96658c" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:43.252683 23086 tablet_bootstrap.cc:492] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:43.255062 23187 raft_consensus.cc:2804] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:43.255489 23187 raft_consensus.cc:493] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:43.255892 23187 raft_consensus.cc:3060] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.255198 22893 raft_consensus.cc:2468] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 0.
I20260709 13:21:43.257104 22860 leader_election.cc:304] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:43.258424 23091 consensus_queue.cc:1048] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:43.241845 23210 leader_election.cc:290] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:43.259608 23210 raft_consensus.cc:2804] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:43.259964 23210 raft_consensus.cc:493] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:43.254117 23121 leader_election.cc:290] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:43.261574 23087 tablet_bootstrap.cc:492] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:43.262131 23087 ts_tablet_manager.cc:1403] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.068s	user 0.008s	sys 0.009s
I20260709 13:21:43.260310 23210 raft_consensus.cc:3060] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.263181 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8be856e3d604f8d8a1c56a88ea18643" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:43.264845 23087 raft_consensus.cc:359] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:43.265584 23087 raft_consensus.cc:385] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.267500 23096 raft_consensus.cc:515] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:43.263284 23085 raft_consensus.cc:359] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.268939 23085 raft_consensus.cc:385] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.269273 23085 raft_consensus.cc:740] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.270035 23085 consensus_queue.cc:260] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.268007 23086 tablet_bootstrap.cc:654] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.270673 23222 raft_consensus.cc:515] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.255813 22858 leader_election.cc:304] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:43.272653 23121 raft_consensus.cc:2804] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:43.273046 23121 raft_consensus.cc:493] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:43.273376 23121 raft_consensus.cc:3060] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.273684 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e230e64c6984b72a2ba543f2a017e25" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
W20260709 13:21:43.275045 22932 leader_election.cc:343] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:43.276786 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8be856e3d604f8d8a1c56a88ea18643" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:43.277494 23041 raft_consensus.cc:2468] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
I20260709 13:21:43.278832 22934 leader_election.cc:304] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:43.279767 23182 raft_consensus.cc:2804] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:43.280151 23182 raft_consensus.cc:493] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:43.280493 23182 raft_consensus.cc:3060] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.265869 23087 raft_consensus.cc:740] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.290326 23087 consensus_queue.cc:260] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:43.269640 23096 leader_election.cc:290] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:43.297078 23087 ts_tablet_manager.cc:1434] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 13:21:43.302342 23087 tablet_bootstrap.cc:492] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:43.304001 23085 ts_tablet_manager.cc:1434] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.065s	user 0.013s	sys 0.008s
I20260709 13:21:43.305354 23085 tablet_bootstrap.cc:492] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:43.310038 23210 raft_consensus.cc:515] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
W20260709 13:21:43.316877 22860 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 59859.3us
I20260709 13:21:43.326201 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:43.314599 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be220d19020b428a8b1cc356f32ff5ad" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:43.331807 23125 raft_consensus.cc:493] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.314548 23227 raft_consensus.cc:697] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:43.285732 23187 raft_consensus.cc:515] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:43.312933 23130 consensus_queue.cc:1048] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 13:21:43.333248 23222 leader_election.cc:290] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:43.353243 23085 tablet_bootstrap.cc:654] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.341216 23125 raft_consensus.cc:515] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:43.354388 23096 raft_consensus.cc:493] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.354965 23096 raft_consensus.cc:515] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:43.340899 23041 raft_consensus.cc:3060] T be220d19020b428a8b1cc356f32ff5ad P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.346589 23275 raft_consensus.cc:493] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.358132 23275 raft_consensus.cc:515] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.360150 23275 leader_election.cc:290] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:43.361596 23096 leader_election.cc:290] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:43.355412 23125 leader_election.cc:290] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:43.337502 22783 leader_election.cc:304] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:43.347508 23121 raft_consensus.cc:515] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:43.380719 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968e16cef40146509d654274b9fd2687" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:43.382714 23125 raft_consensus.cc:493] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.394630 23125 raft_consensus.cc:515] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:43.394665 22717 catalog_manager.cc:5697] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ef3887fb3874f8b81079e542d252944 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "8ef3887fb3874f8b81079e542d252944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:43.367334 23182 raft_consensus.cc:515] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:43.393345 23086 tablet_bootstrap.cc:492] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:43.393970 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4c294dbaeb4040b157ad72102a9e56" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
W20260709 13:21:43.382484 22785 leader_election.cc:343] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:43.397102 23086 ts_tablet_manager.cc:1403] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.145s	user 0.032s	sys 0.033s
W20260709 13:21:43.387759 22783 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 55412.5us
I20260709 13:21:43.390765 23282 raft_consensus.cc:2804] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:43.414499 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be220d19020b428a8b1cc356f32ff5ad" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:43.415227 22820 raft_consensus.cc:3060] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.418215 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968e16cef40146509d654274b9fd2687" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:43.418869 23220 raft_consensus.cc:493] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.419094 22966 raft_consensus.cc:2393] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 in current term 1: Already voted for candidate 27745ef236cd494684f62a1269411a68 in this term.
I20260709 13:21:43.419400 23220 raft_consensus.cc:515] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:43.421527 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4c294dbaeb4040b157ad72102a9e56" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:43.422391 22966 raft_consensus.cc:2468] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 0.
I20260709 13:21:43.424011 22894 raft_consensus.cc:1275] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Refusing update from remote peer 8ef3887fb3874f8b81079e542d252944: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:21:43.426558 23276 consensus_queue.cc:1048] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:43.408905 23282 raft_consensus.cc:493] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:43.431660 23282 raft_consensus.cc:3060] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.405164 23041 raft_consensus.cc:2468] T be220d19020b428a8b1cc356f32ff5ad P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:43.387187 23227 consensus_queue.cc:237] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:43.451318 23084 tablet_bootstrap.cc:492] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:43.451944 23084 ts_tablet_manager.cc:1403] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.254s	user 0.016s	sys 0.020s
I20260709 13:21:43.454910 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b783c62bd54d4c8f4f1795b7409958" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:43.455919 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b783c62bd54d4c8f4f1795b7409958" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:43.455302 23086 raft_consensus.cc:359] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.458474 23086 raft_consensus.cc:385] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.458879 23086 raft_consensus.cc:740] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.459617 23086 consensus_queue.cc:260] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.446951 23125 leader_election.cc:290] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:43.463294 23086 ts_tablet_manager.cc:1434] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.062s	user 0.006s	sys 0.004s
W20260709 13:21:43.448927 22785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 112860us
I20260709 13:21:43.453092 23210 leader_election.cc:290] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:43.462842 22858 leader_election.cc:304] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:43.465153 23086 tablet_bootstrap.cc:492] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
W20260709 13:21:43.446707 22860 leader_election.cc:343] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:43.469750 23087 tablet_bootstrap.cc:654] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.458299 23182 leader_election.cc:290] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:43.472309 23085 tablet_bootstrap.cc:492] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:43.473476 23085 ts_tablet_manager.cc:1403] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.168s	user 0.013s	sys 0.002s
I20260709 13:21:43.467530 23187 leader_election.cc:290] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:43.465013 22817 raft_consensus.cc:2468] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 0.
I20260709 13:21:43.475191 23210 raft_consensus.cc:2804] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:43.481226 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ea5c83aada45d3a98027069e96658c" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:43.481490 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ea5c83aada45d3a98027069e96658c" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:43.482093 22893 raft_consensus.cc:3060] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.482052 22818 raft_consensus.cc:2468] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 0.
I20260709 13:21:43.477615 23220 leader_election.cc:290] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:43.490242 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e230e64c6984b72a2ba543f2a017e25" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:43.490904 22818 raft_consensus.cc:3060] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:43.478080 22860 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 66504.9us
I20260709 13:21:43.491818 23086 tablet_bootstrap.cc:654] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.494910 22860 leader_election.cc:304] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:43.486246 23210 raft_consensus.cc:697] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:43.496510 23210 consensus_queue.cc:237] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:43.497586 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:43.498237 23040 raft_consensus.cc:3060] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.498620 23084 raft_consensus.cc:359] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.498984 23277 raft_consensus.cc:3060] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.500682 23084 raft_consensus.cc:385] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.500993 23084 raft_consensus.cc:740] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.501368 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:43.501789 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4c294dbaeb4040b157ad72102a9e56" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:43.502017 22817 raft_consensus.cc:3060] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.502465 23041 raft_consensus.cc:2468] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 0.
I20260709 13:21:43.484369 23008 leader_election.cc:304] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
W20260709 13:21:43.503923 23008 leader_election.cc:343] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:43.504983 22932 leader_election.cc:304] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:43.506696 23084 consensus_queue.cc:260] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.511771 23187 raft_consensus.cc:2804] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:43.512244 23187 raft_consensus.cc:493] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:43.512574 23187 raft_consensus.cc:3060] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:43.514508 22785 consensus_peers.cc:597] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 -> Peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Couldn't send request to peer 49cbf11e18ce4268a7fcca76872afb23. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:43.515228 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4c294dbaeb4040b157ad72102a9e56" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:43.516641 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ea5c83aada45d3a98027069e96658c" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:43.517297 22819 raft_consensus.cc:3060] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:43.518261 22934 leader_election.cc:343] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:43.522258 22893 raft_consensus.cc:2468] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:43.509307 23125 raft_consensus.cc:493] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.532701 22934 leader_election.cc:304] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:43.533016 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:43.526763 23084 ts_tablet_manager.cc:1434] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.074s	user 0.006s	sys 0.001s
I20260709 13:21:43.532547 23125 raft_consensus.cc:515] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.536523 23084 tablet_bootstrap.cc:492] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:43.545529 23085 raft_consensus.cc:359] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.546096 23085 raft_consensus.cc:385] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.526717 23220 raft_consensus.cc:2804] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:43.514827 22820 raft_consensus.cc:2468] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:43.552268 23220 raft_consensus.cc:493] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:43.547538 23085 raft_consensus.cc:740] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.530138 23275 raft_consensus.cc:493] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.534883 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:43.552815 23220 raft_consensus.cc:3060] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.555215 23085 consensus_queue.cc:260] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.547240 23182 raft_consensus.cc:2804] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:43.566689 23085 ts_tablet_manager.cc:1434] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.093s	user 0.006s	sys 0.000s
I20260709 13:21:43.569878 23085 tablet_bootstrap.cc:492] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:43.550782 23121 leader_election.cc:290] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:43.562705 22818 raft_consensus.cc:2468] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:43.580806 23041 raft_consensus.cc:1275] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Refusing update from remote peer 8ef3887fb3874f8b81079e542d252944: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:21:43.600579 23086 tablet_bootstrap.cc:492] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:43.601032 23086 ts_tablet_manager.cc:1403] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.137s	user 0.008s	sys 0.015s
I20260709 13:21:43.563825 23275 raft_consensus.cc:515] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:43.580183 23182 raft_consensus.cc:697] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [term 1 LEADER]: Becoming Leader. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:21:43.596158 22817 raft_consensus.cc:2468] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:43.605155 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10066ccf8d174f4a9dd69253613857b8" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:43.626520 23087 tablet_bootstrap.cc:492] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:43.627053 23087 ts_tablet_manager.cc:1403] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.325s	user 0.008s	sys 0.006s
I20260709 13:21:43.622690 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
W20260709 13:21:43.616357 22785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 78271.1us
I20260709 13:21:43.625690 22820 raft_consensus.cc:3060] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.619356 23008 leader_election.cc:304] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:43.635123 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10066ccf8d174f4a9dd69253613857b8" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:43.648017 22890 raft_consensus.cc:3060] T 10066ccf8d174f4a9dd69253613857b8 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.644507 23222 consensus_queue.cc:1048] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:21:43.632400 23182 consensus_queue.cc:237] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:43.652954 23125 leader_election.cc:290] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:43.664443 23085 tablet_bootstrap.cc:654] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:43.660172 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 46861.2us
I20260709 13:21:43.668120 22934 leader_election.cc:304] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68
I20260709 13:21:43.680754 23084 tablet_bootstrap.cc:654] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.681334 23281 raft_consensus.cc:2804] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:43.669200 23210 raft_consensus.cc:493] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.683691 23087 raft_consensus.cc:359] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:43.685518 23087 raft_consensus.cc:385] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.685848 23087 raft_consensus.cc:740] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.686538 23087 consensus_queue.cc:260] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:43.688084 23087 ts_tablet_manager.cc:1434] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.061s	user 0.002s	sys 0.001s
I20260709 13:21:43.689081 23087 tablet_bootstrap.cc:492] T 1a4c294dbaeb4040b157ad72102a9e56 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:43.690040 23296 raft_consensus.cc:2804] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:43.690506 23296 raft_consensus.cc:493] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:43.690846 23296 raft_consensus.cc:3060] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.695145 23220 raft_consensus.cc:515] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:43.696161 23087 tablet_bootstrap.cc:654] T 1a4c294dbaeb4040b157ad72102a9e56 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.697436 23220 leader_election.cc:290] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:43.698734 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4c294dbaeb4040b157ad72102a9e56" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:43.655757 23282 raft_consensus.cc:515] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.683077 22819 raft_consensus.cc:2468] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:43.703478 23040 raft_consensus.cc:2468] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:43.676367 23275 leader_election.cc:290] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:43.701552 23210 raft_consensus.cc:515] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:43.796943 22858 leader_election.cc:304] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
W20260709 13:21:43.701341 23008 leader_election.cc:343] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:43.800263 22858 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 88162.1us
I20260709 13:21:43.800814 23121 raft_consensus.cc:2804] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:43.801162 22895 raft_consensus.cc:3060] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.801385 23121 raft_consensus.cc:697] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:43.801982 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b154956cc17f4ba68f906daffbcedff6" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:43.795382 23281 raft_consensus.cc:697] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 1 LEADER]: Becoming Leader. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:21:43.805658 23303 raft_consensus.cc:493] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.803090 23121 consensus_queue.cc:237] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:43.806653 23281 consensus_queue.cc:237] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.808233 23210 leader_election.cc:290] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:43.820298 23303 raft_consensus.cc:515] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.829789 23086 raft_consensus.cc:359] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:43.830744 23086 raft_consensus.cc:385] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.831084 23086 raft_consensus.cc:740] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
W20260709 13:21:43.811762 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 108746us
I20260709 13:21:43.819279 22890 raft_consensus.cc:2468] T 10066ccf8d174f4a9dd69253613857b8 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
W20260709 13:21:43.835944 22860 leader_election.cc:343] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:43.832346 23187 raft_consensus.cc:515] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:43.853710 23282 leader_election.cc:290] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:43.870592 23041 raft_consensus.cc:2393] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 in current term 1: Already voted for candidate 27745ef236cd494684f62a1269411a68 in this term.
I20260709 13:21:43.871309 23296 raft_consensus.cc:515] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.873661 23296 leader_election.cc:290] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:43.856874 23276 consensus_queue.cc:1048] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:21:43.875270 23084 tablet_bootstrap.cc:492] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:43.870311 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4c294dbaeb4040b157ad72102a9e56" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:43.877372 23084 ts_tablet_manager.cc:1403] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.341s	user 0.009s	sys 0.012s
I20260709 13:21:43.871268 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
W20260709 13:21:43.881826 23300 log.cc:927] Time spent T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:21:43.888269 23087 tablet_bootstrap.cc:492] T 1a4c294dbaeb4040b157ad72102a9e56 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:43.888863 23087 ts_tablet_manager.cc:1403] T 1a4c294dbaeb4040b157ad72102a9e56 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.200s	user 0.027s	sys 0.036s
W20260709 13:21:43.880028 23298 log.cc:927] Time spent T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:21:43.891036 23220 consensus_queue.cc:1048] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:43.893287 22820 raft_consensus.cc:2468] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
I20260709 13:21:43.882020 23277 raft_consensus.cc:2749] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:43.889711 23187 leader_election.cc:290] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:43.899461 22892 raft_consensus.cc:3060] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.891640 23087 raft_consensus.cc:359] T 1a4c294dbaeb4040b157ad72102a9e56 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:43.909368 23087 raft_consensus.cc:385] T 1a4c294dbaeb4040b157ad72102a9e56 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.909705 23087 raft_consensus.cc:740] T 1a4c294dbaeb4040b157ad72102a9e56 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.910450 23087 consensus_queue.cc:260] T 1a4c294dbaeb4040b157ad72102a9e56 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:43.901907 23086 consensus_queue.cc:260] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:43.885643 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8be856e3d604f8d8a1c56a88ea18643" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:43.914892 22717 catalog_manager.cc:5697] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:43.916522 23086 ts_tablet_manager.cc:1434] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.315s	user 0.016s	sys 0.000s
I20260709 13:21:43.918776 23086 tablet_bootstrap.cc:492] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:43.895148 23303 leader_election.cc:290] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:43.923710 23087 ts_tablet_manager.cc:1434] T 1a4c294dbaeb4040b157ad72102a9e56 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.035s	user 0.009s	sys 0.000s
I20260709 13:21:43.924854 23087 tablet_bootstrap.cc:492] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:43.883448 22860 leader_election.cc:304] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:43.926604 23085 tablet_bootstrap.cc:492] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:43.928020 23085 ts_tablet_manager.cc:1403] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.358s	user 0.014s	sys 0.014s
I20260709 13:21:43.919412 22817 raft_consensus.cc:3060] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:43.934105 23008 leader_election.cc:343] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:43.935424 23085 raft_consensus.cc:359] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:43.936110 23085 raft_consensus.cc:385] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.936368 23085 raft_consensus.cc:740] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.936935 23085 consensus_queue.cc:260] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:43.922192 22966 raft_consensus.cc:3060] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:43.935546 23008 leader_election.cc:343] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:43.944085 23085 ts_tablet_manager.cc:1434] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:21:43.948060 23085 tablet_bootstrap.cc:492] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:43.950770 23084 raft_consensus.cc:359] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:43.951366 23084 raft_consensus.cc:385] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.951628 23084 raft_consensus.cc:740] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.952263 23084 consensus_queue.cc:260] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:43.944914 23210 raft_consensus.cc:2804] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:43.953881 23084 ts_tablet_manager.cc:1434] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.076s	user 0.002s	sys 0.001s
I20260709 13:21:43.963099 23084 tablet_bootstrap.cc:492] T 50b783c62bd54d4c8f4f1795b7409958 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:43.938696 23187 raft_consensus.cc:493] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.957352 23210 raft_consensus.cc:697] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
W20260709 13:21:43.967008 22860 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 84245.3us
I20260709 13:21:43.981415 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8be856e3d604f8d8a1c56a88ea18643" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:43.982337 22819 raft_consensus.cc:2393] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 in current term 1: Already voted for candidate 91f33bdbdd63438294071eb3776db679 in this term.
I20260709 13:21:43.983074 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b154956cc17f4ba68f906daffbcedff6" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:43.948273 23008 leader_election.cc:304] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:43.988031 22860 leader_election.cc:304] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
W20260709 13:21:43.988245 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 52725.5us
I20260709 13:21:43.988801 23296 raft_consensus.cc:2749] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:21:43.975528 22934 leader_election.cc:343] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:43.989379 23008 leader_election.cc:304] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:43.991737 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8be856e3d604f8d8a1c56a88ea18643" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:43.992894 23210 consensus_queue.cc:237] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:43.993669 23277 raft_consensus.cc:2749] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:43.974057 23187 raft_consensus.cc:515] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:43.995517 23121 consensus_queue.cc:1048] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:44.001269 22934 leader_election.cc:304] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:44.006165 23187 leader_election.cc:290] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:44.002755 22966 raft_consensus.cc:2468] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:44.008154 23281 raft_consensus.cc:2749] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:43.995560 23085 tablet_bootstrap.cc:654] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.002526 23296 raft_consensus.cc:2804] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:44.010001 23296 raft_consensus.cc:697] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:44.014492 23086 tablet_bootstrap.cc:654] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.015089 22785 leader_election.cc:304] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:44.016027 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ecf8a5022f43c787aaff1cf7679264" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:44.016456 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39611d2ff3524bc0840e4fd222d96a84" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:44.016721 22968 raft_consensus.cc:2468] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 0.
W20260709 13:21:44.017815 22766 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:44.018538 23282 raft_consensus.cc:2804] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:44.019083 23282 raft_consensus.cc:697] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:44.000972 22817 raft_consensus.cc:2468] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:44.019913 23282 consensus_queue.cc:237] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:44.020237 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ecf8a5022f43c787aaff1cf7679264" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:44.020923 22893 raft_consensus.cc:2468] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 0.
I20260709 13:21:44.002279 23043 raft_consensus.cc:2468] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 0.
I20260709 13:21:44.026821 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b783c62bd54d4c8f4f1795b7409958" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:44.015975 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39611d2ff3524bc0840e4fd222d96a84" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:44.027823 23296 consensus_queue.cc:237] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:44.028599 22932 leader_election.cc:304] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:44.029469 23281 raft_consensus.cc:2764] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:21:44.032799 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b783c62bd54d4c8f4f1795b7409958" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:44.033180 23084 tablet_bootstrap.cc:654] T 50b783c62bd54d4c8f4f1795b7409958 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.032903 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4c294dbaeb4040b157ad72102a9e56" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:44.034157 22893 raft_consensus.cc:3060] T 1a4c294dbaeb4040b157ad72102a9e56 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.033453 22717 catalog_manager.cc:5697] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 1 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:44.036039 22820 raft_consensus.cc:3060] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.036267 23087 tablet_bootstrap.cc:654] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.047464 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e230e64c6984b72a2ba543f2a017e25" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:44.048342 22817 raft_consensus.cc:2393] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 in current term 1: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:44.052578 23008 leader_election.cc:304] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:44.033130 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4c294dbaeb4040b157ad72102a9e56" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:44.059131 22968 raft_consensus.cc:2393] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 in current term 1: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
W20260709 13:21:44.063793 23008 leader_election.cc:343] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:44.066279 23008 leader_election.cc:343] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:44.068874 23008 leader_election.cc:343] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:44.071326 23008 leader_election.cc:304] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:44.072749 22893 raft_consensus.cc:2468] T 1a4c294dbaeb4040b157ad72102a9e56 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
I20260709 13:21:44.073218 23283 raft_consensus.cc:2749] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:44.068377 22716 catalog_manager.cc:5697] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ef3887fb3874f8b81079e542d252944 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "8ef3887fb3874f8b81079e542d252944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:44.076391 22716 catalog_manager.cc:5697] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ef3887fb3874f8b81079e542d252944 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "8ef3887fb3874f8b81079e542d252944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:44.065176 23187 raft_consensus.cc:2804] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:44.069142 23275 consensus_queue.cc:1048] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:21:44.090183 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e230e64c6984b72a2ba543f2a017e25" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:44.099838 23085 tablet_bootstrap.cc:492] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:44.100504 23085 ts_tablet_manager.cc:1403] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.153s	user 0.015s	sys 0.004s
I20260709 13:21:44.079372 23187 raft_consensus.cc:493] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:44.106746 22820 raft_consensus.cc:2468] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
I20260709 13:21:44.109432 23085 raft_consensus.cc:359] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:44.120013 22718 catalog_manager.cc:5697] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:44.121238 23085 raft_consensus.cc:385] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.127388 23086 tablet_bootstrap.cc:492] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:44.127956 23086 ts_tablet_manager.cc:1403] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.209s	user 0.008s	sys 0.010s
I20260709 13:21:44.121778 23085 raft_consensus.cc:740] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.128831 23085 consensus_queue.cc:260] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
W20260709 13:21:44.133309 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 56799.7us
I20260709 13:21:44.139246 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8be856e3d604f8d8a1c56a88ea18643" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:44.139897 23043 raft_consensus.cc:3060] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.121373 23187 raft_consensus.cc:3060] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.146294 23085 ts_tablet_manager.cc:1434] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.046s	user 0.009s	sys 0.000s
I20260709 13:21:44.140899 23121 consensus_queue.cc:1048] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:21:44.183645 23085 tablet_bootstrap.cc:492] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:44.181869 23043 raft_consensus.cc:2468] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:44.187896 23008 leader_election.cc:304] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:44.194612 23008 leader_election.cc:304] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:44.194998 23227 consensus_queue.cc:1048] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:44.195514 23125 raft_consensus.cc:3060] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.202477 23086 raft_consensus.cc:359] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:44.203245 23086 raft_consensus.cc:385] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.203516 23086 raft_consensus.cc:740] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.194660 23296 raft_consensus.cc:2804] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:44.205456 23296 raft_consensus.cc:697] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:44.206146 23087 tablet_bootstrap.cc:492] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:44.209352 23087 ts_tablet_manager.cc:1403] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.285s	user 0.011s	sys 0.026s
I20260709 13:21:44.212224 23087 raft_consensus.cc:359] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:44.218479 23087 raft_consensus.cc:385] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.220046 23087 raft_consensus.cc:740] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.221069 23087 consensus_queue.cc:260] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:44.206895 23008 leader_election.cc:304] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:44.208019 23182 consensus_queue.cc:1048] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:44.224311 23275 raft_consensus.cc:2804] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:44.224875 23275 raft_consensus.cc:697] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:44.225705 23275 consensus_queue.cc:237] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:44.213009 23125 raft_consensus.cc:2749] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:44.213476 23296 consensus_queue.cc:237] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:44.204253 23187 raft_consensus.cc:515] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:44.241636 23187 leader_election.cc:290] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:44.243584 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ecf8a5022f43c787aaff1cf7679264" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:44.243815 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ecf8a5022f43c787aaff1cf7679264" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:44.244293 22893 raft_consensus.cc:3060] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.244583 22968 raft_consensus.cc:3060] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.249315 23087 ts_tablet_manager.cc:1434] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.000s
I20260709 13:21:44.252373 23087 tablet_bootstrap.cc:492] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:44.215596 23085 tablet_bootstrap.cc:654] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.204207 23086 consensus_queue.cc:260] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:44.214787 23084 tablet_bootstrap.cc:492] T 50b783c62bd54d4c8f4f1795b7409958 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:44.256244 23084 ts_tablet_manager.cc:1403] T 50b783c62bd54d4c8f4f1795b7409958 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.293s	user 0.007s	sys 0.018s
I20260709 13:21:44.257361 23086 ts_tablet_manager.cc:1434] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.129s	user 0.005s	sys 0.002s
I20260709 13:21:44.258806 23086 tablet_bootstrap.cc:492] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:44.261350 22893 raft_consensus.cc:2468] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
I20260709 13:21:44.262199 23084 raft_consensus.cc:359] T 50b783c62bd54d4c8f4f1795b7409958 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:44.263901 23084 raft_consensus.cc:385] T 50b783c62bd54d4c8f4f1795b7409958 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.264261 23084 raft_consensus.cc:740] T 50b783c62bd54d4c8f4f1795b7409958 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.265024 23087 tablet_bootstrap.cc:654] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.265375 23084 consensus_queue.cc:260] T 50b783c62bd54d4c8f4f1795b7409958 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:44.268105 23084 ts_tablet_manager.cc:1434] T 50b783c62bd54d4c8f4f1795b7409958 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:21:44.271579 23086 tablet_bootstrap.cc:654] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.271565 23084 tablet_bootstrap.cc:492] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:44.280795 23084 tablet_bootstrap.cc:654] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.283586 23008 leader_election.cc:304] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:44.284536 23320 raft_consensus.cc:2804] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:44.285316 23320 raft_consensus.cc:697] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:44.286149 23320 consensus_queue.cc:237] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:44.287310 23087 tablet_bootstrap.cc:492] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:44.288249 23087 ts_tablet_manager.cc:1403] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.036s	user 0.023s	sys 0.003s
I20260709 13:21:44.289748 23182 consensus_queue.cc:1048] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:44.291553 23087 raft_consensus.cc:359] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:44.292452 23087 raft_consensus.cc:385] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.292784 23087 raft_consensus.cc:740] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.293684 23087 consensus_queue.cc:260] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:44.296048 23087 ts_tablet_manager.cc:1434] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:21:44.300479 22968 raft_consensus.cc:2468] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
I20260709 13:21:44.300889 23087 tablet_bootstrap.cc:492] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:44.304293 22716 catalog_manager.cc:5697] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:44.315291 22716 catalog_manager.cc:5697] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:44.321735 23087 tablet_bootstrap.cc:654] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.324815 23086 tablet_bootstrap.cc:492] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:44.325402 23086 ts_tablet_manager.cc:1403] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.067s	user 0.013s	sys 0.002s
I20260709 13:21:44.322650 23182 consensus_queue.cc:1048] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:44.323982 23187 raft_consensus.cc:493] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:44.328089 23086 raft_consensus.cc:359] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:44.328855 23086 raft_consensus.cc:385] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.329129 23086 raft_consensus.cc:740] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.326480 23187 raft_consensus.cc:515] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:44.337960 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "745e7e3bed5b4d59949aeddcede543f3" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:44.338943 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "745e7e3bed5b4d59949aeddcede543f3" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
W20260709 13:21:44.339576 23008 leader_election.cc:343] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:44.341464 23008 leader_election.cc:343] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:44.341883 23008 leader_election.cc:304] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:44.329843 23086 consensus_queue.cc:260] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:44.342768 23125 raft_consensus.cc:2749] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:44.344583 23086 ts_tablet_manager.cc:1434] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:21:44.345721 23086 tablet_bootstrap.cc:492] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:44.351691 23281 consensus_queue.cc:1048] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:44.336784 23187 leader_election.cc:290] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:44.361707 23210 consensus_queue.cc:1048] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:44.366331 23210 consensus_queue.cc:1048] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:44.363930 23085 tablet_bootstrap.cc:492] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:44.371331 23085 ts_tablet_manager.cc:1403] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.188s	user 0.014s	sys 0.014s
I20260709 13:21:44.375795 23084 tablet_bootstrap.cc:492] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:44.378252 23084 ts_tablet_manager.cc:1403] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.107s	user 0.020s	sys 0.019s
I20260709 13:21:44.378512 23087 tablet_bootstrap.cc:492] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:44.379045 23087 ts_tablet_manager.cc:1403] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.078s	user 0.011s	sys 0.008s
I20260709 13:21:44.382369 23087 raft_consensus.cc:359] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:44.384266 23087 raft_consensus.cc:385] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.384663 23087 raft_consensus.cc:740] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.383451 23084 raft_consensus.cc:359] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:44.385560 23084 raft_consensus.cc:385] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.385910 23084 raft_consensus.cc:740] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.388594 23084 consensus_queue.cc:260] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:44.392349 23084 ts_tablet_manager.cc:1434] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 13:21:44.394076 23084 tablet_bootstrap.cc:492] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:44.399801 23084 tablet_bootstrap.cc:654] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.389508 23087 consensus_queue.cc:260] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:44.401644 23087 ts_tablet_manager.cc:1434] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.002s
I20260709 13:21:44.402604 23087 tablet_bootstrap.cc:492] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:44.383451 23085 raft_consensus.cc:359] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:44.406227 23085 raft_consensus.cc:385] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.406545 23085 raft_consensus.cc:740] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.407230 23085 consensus_queue.cc:260] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:44.409245 23085 ts_tablet_manager.cc:1434] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.028s	user 0.004s	sys 0.000s
I20260709 13:21:44.412657 23085 tablet_bootstrap.cc:492] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:44.412767 23187 consensus_queue.cc:1048] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:44.419385 23087 tablet_bootstrap.cc:654] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.424710 23086 tablet_bootstrap.cc:654] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.428725 23085 tablet_bootstrap.cc:654] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.439358 23084 tablet_bootstrap.cc:492] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:44.439855 23084 ts_tablet_manager.cc:1403] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.046s	user 0.010s	sys 0.000s
I20260709 13:21:44.441876 23320 consensus_queue.cc:1048] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:44.442605 23084 raft_consensus.cc:359] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:44.443346 23084 raft_consensus.cc:385] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.443608 23084 raft_consensus.cc:740] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.444294 23084 consensus_queue.cc:260] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:44.445125 23085 tablet_bootstrap.cc:492] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:44.445667 23085 ts_tablet_manager.cc:1403] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.003s
I20260709 13:21:44.447389 23087 tablet_bootstrap.cc:492] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:44.447835 22968 raft_consensus.cc:3060] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.448437 23085 raft_consensus.cc:359] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:44.448971 23086 tablet_bootstrap.cc:492] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:44.449152 23085 raft_consensus.cc:385] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.447909 23087 ts_tablet_manager.cc:1403] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.046s	user 0.010s	sys 0.000s
I20260709 13:21:44.449461 23086 ts_tablet_manager.cc:1403] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.104s	user 0.012s	sys 0.000s
I20260709 13:21:44.449468 23085 raft_consensus.cc:740] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.452136 23085 consensus_queue.cc:260] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:44.454375 23085 ts_tablet_manager.cc:1434] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:21:44.455374 23085 tablet_bootstrap.cc:492] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:44.456691 23281 raft_consensus.cc:493] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:44.457221 23281 raft_consensus.cc:515] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:44.459398 23087 raft_consensus.cc:359] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:44.460191 23087 raft_consensus.cc:385] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.460536 23087 raft_consensus.cc:740] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.460987 23210 consensus_queue.cc:1048] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:44.461299 23087 consensus_queue.cc:260] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:44.460191 23086 raft_consensus.cc:359] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:44.464010 23086 raft_consensus.cc:385] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.464274 23085 tablet_bootstrap.cc:654] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.464319 23086 raft_consensus.cc:740] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.463624 23087 ts_tablet_manager.cc:1434] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:21:44.465837 23087 tablet_bootstrap.cc:492] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:44.465444 23086 consensus_queue.cc:260] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:44.468047 23086 ts_tablet_manager.cc:1434] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:21:44.469039 23086 tablet_bootstrap.cc:492] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:44.472424 23087 tablet_bootstrap.cc:654] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.474587 23086 tablet_bootstrap.cc:654] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.464810 23084 ts_tablet_manager.cc:1434] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:21:44.476648 23084 tablet_bootstrap.cc:492] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:44.479628 23281 leader_election.cc:290] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:44.482790 23084 tablet_bootstrap.cc:654] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.488168 23087 tablet_bootstrap.cc:492] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:44.488736 23087 ts_tablet_manager.cc:1403] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.001s
I20260709 13:21:44.491777 23087 raft_consensus.cc:359] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:44.492540 23087 raft_consensus.cc:385] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.539602 22716 catalog_manager.cc:5697] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 1 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } }
I20260709 13:21:44.545266 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:44.546837 22820 raft_consensus.cc:2468] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:44.554303 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:44.556840 22934 leader_election.cc:304] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:44.492817 23087 raft_consensus.cc:740] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.567894 23096 raft_consensus.cc:493] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:44.567754 23087 consensus_queue.cc:260] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:44.568915 23096 raft_consensus.cc:515] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:44.570575 23085 tablet_bootstrap.cc:492] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:44.571870 23085 ts_tablet_manager.cc:1403] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.117s	user 0.020s	sys 0.016s
I20260709 13:21:44.572733 23096 leader_election.cc:290] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:44.573253 23087 ts_tablet_manager.cc:1434] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.084s	user 0.009s	sys 0.000s
I20260709 13:21:44.574656 23096 raft_consensus.cc:493] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:44.575455 23087 tablet_bootstrap.cc:492] T d1934129c8764c018495eff97d43a3b5 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:44.576364 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171526d2fe264ee8b1168192bec31c7a" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:44.577474 23040 raft_consensus.cc:2468] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 0.
I20260709 13:21:44.580703 23125 raft_consensus.cc:493] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:44.581513 23125 raft_consensus.cc:515] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:44.586311 22858 leader_election.cc:304] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:44.586889 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171526d2fe264ee8b1168192bec31c7a" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:44.587575 23121 raft_consensus.cc:2804] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:44.588132 23121 raft_consensus.cc:493] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:44.588261 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171526d2fe264ee8b1168192bec31c7a" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
W20260709 13:21:44.590471 23008 leader_election.cc:343] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:44.588482 23121 raft_consensus.cc:3060] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.595494 23281 raft_consensus.cc:493] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:44.601737 23303 raft_consensus.cc:2804] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:44.602463 23303 raft_consensus.cc:493] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:44.602911 23303 raft_consensus.cc:3060] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:44.607141 23086 tablet_bootstrap.cc:492] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:44.608137 23086 ts_tablet_manager.cc:1403] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.139s	user 0.016s	sys 0.008s
I20260709 13:21:44.596282 23281 raft_consensus.cc:515] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:44.610280 23084 tablet_bootstrap.cc:492] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:44.611328 23084 ts_tablet_manager.cc:1403] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.135s	user 0.013s	sys 0.001s
I20260709 13:21:44.613831 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "745e7e3bed5b4d59949aeddcede543f3" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:44.614816 23040 raft_consensus.cc:2468] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 0.
I20260709 13:21:44.586329 23125 leader_election.cc:290] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:44.616967 23281 leader_election.cc:290] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:44.621665 22932 leader_election.cc:304] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:44.575681 23096 raft_consensus.cc:515] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:44.629223 23277 raft_consensus.cc:493] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:44.629690 23087 tablet_bootstrap.cc:654] T d1934129c8764c018495eff97d43a3b5 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.630172 23277 raft_consensus.cc:515] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:44.632863 23277 leader_election.cc:290] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:44.638054 23353 consensus_queue.cc:1048] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:21:44.647464 23220 raft_consensus.cc:493] T 50b783c62bd54d4c8f4f1795b7409958 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:44.648146 23277 raft_consensus.cc:493] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:44.648381 23220 raft_consensus.cc:515] T 50b783c62bd54d4c8f4f1795b7409958 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:44.648967 23277 raft_consensus.cc:515] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:44.651165 23220 leader_election.cc:290] T 50b783c62bd54d4c8f4f1795b7409958 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:44.656726 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "745e7e3bed5b4d59949aeddcede543f3" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:44.658056 23085 raft_consensus.cc:359] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:44.654089 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b783c62bd54d4c8f4f1795b7409958" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:44.654320 23297 consensus_queue.cc:1048] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:44.661862 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b783c62bd54d4c8f4f1795b7409958" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:44.669642 22817 raft_consensus.cc:2393] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 in current term 1: Already voted for candidate 27745ef236cd494684f62a1269411a68 in this term.
I20260709 13:21:44.672117 23210 raft_consensus.cc:493] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:44.672703 23210 raft_consensus.cc:515] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:44.674962 23210 leader_election.cc:290] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:44.652133 22717 catalog_manager.cc:5697] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 1 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:44.680949 23297 raft_consensus.cc:493] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:44.656688 23277 leader_election.cc:290] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:44.665278 23085 raft_consensus.cc:385] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.637971 23084 raft_consensus.cc:359] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:44.667145 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171526d2fe264ee8b1168192bec31c7a" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:44.682276 23297 raft_consensus.cc:515] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:44.683702 23084 raft_consensus.cc:385] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.684062 23084 raft_consensus.cc:740] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.684543 23353 raft_consensus.cc:493] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:44.685051 23084 consensus_queue.cc:260] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:44.685309 23353 raft_consensus.cc:515] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:44.687139 23084 ts_tablet_manager.cc:1434] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.075s	user 0.006s	sys 0.002s
I20260709 13:21:44.688812 23084 tablet_bootstrap.cc:492] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:44.689558 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:44.690773 23085 raft_consensus.cc:740] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.691057 23041 raft_consensus.cc:2468] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 0.
I20260709 13:21:44.691808 23085 consensus_queue.cc:260] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:44.692507 22858 leader_election.cc:304] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:44.697938 23085 ts_tablet_manager.cc:1434] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.126s	user 0.006s	sys 0.003s
I20260709 13:21:44.699679 23085 tablet_bootstrap.cc:492] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:44.676023 23121 raft_consensus.cc:515] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:44.701036 22895 raft_consensus.cc:2468] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 0.
I20260709 13:21:44.701979 22893 raft_consensus.cc:2393] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 in current term 1: Already voted for candidate 49cbf11e18ce4268a7fcca76872afb23 in this term.
I20260709 13:21:44.706439 23096 leader_election.cc:290] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:44.707218 23096 raft_consensus.cc:2804] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:44.731444 23096 raft_consensus.cc:493] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:44.714389 22932 leader_election.cc:304] T 50b783c62bd54d4c8f4f1795b7409958 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:44.715396 23121 leader_election.cc:290] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:44.716794 23086 raft_consensus.cc:359] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:44.717972 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "745e7e3bed5b4d59949aeddcede543f3" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:44.720114 23008 leader_election.cc:304] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:44.721006 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171526d2fe264ee8b1168192bec31c7a" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:44.722533 23297 leader_election.cc:290] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:44.725323 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:44.725772 23353 leader_election.cc:290] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:44.725323 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:44.728037 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:44.625288 23281 raft_consensus.cc:2804] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:44.735500 23086 raft_consensus.cc:385] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.734721 23096 raft_consensus.cc:3060] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.741086 23281 raft_consensus.cc:493] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:44.743026 23281 raft_consensus.cc:3060] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.739553 23220 raft_consensus.cc:3060] T 50b783c62bd54d4c8f4f1795b7409958 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.737733 23041 raft_consensus.cc:2468] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 0.
I20260709 13:21:44.737782 23040 raft_consensus.cc:3060] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.747267 22858 leader_election.cc:304] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:44.748368 23354 raft_consensus.cc:2804] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:44.748842 23354 raft_consensus.cc:493] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:44.749178 23354 raft_consensus.cc:3060] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.742839 22717 catalog_manager.cc:5697] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 1 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } }
W20260709 13:21:44.750414 22860 leader_election.cc:343] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:44.741525 22893 raft_consensus.cc:2468] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
I20260709 13:21:44.753083 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:44.754431 22820 raft_consensus.cc:2468] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 0.
I20260709 13:21:44.755287 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:44.756069 23043 raft_consensus.cc:2468] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
I20260709 13:21:44.760071 22783 leader_election.cc:304] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:44.761760 23301 raft_consensus.cc:2804] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:44.762178 23301 raft_consensus.cc:493] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:44.759828 23354 raft_consensus.cc:515] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:44.762511 23301 raft_consensus.cc:3060] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.765352 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "745e7e3bed5b4d59949aeddcede543f3" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:44.766031 23043 raft_consensus.cc:3060] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.740803 23086 raft_consensus.cc:740] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.753051 22717 catalog_manager.cc:5697] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 1 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:44.773320 23220 raft_consensus.cc:2749] T 50b783c62bd54d4c8f4f1795b7409958 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:44.773862 23187 consensus_queue.cc:1048] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:44.780447 23040 raft_consensus.cc:2468] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:44.781234 23296 raft_consensus.cc:2749] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:44.768586 23354 leader_election.cc:290] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:44.770612 23297 raft_consensus.cc:493] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:44.790716 23297 raft_consensus.cc:515] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:44.792287 23296 consensus_queue.cc:1048] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:44.793495 23297 leader_election.cc:290] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:44.806406 23086 consensus_queue.cc:260] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:44.810770 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:44.811772 23042 raft_consensus.cc:2468] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
I20260709 13:21:44.812723 23086 ts_tablet_manager.cc:1434] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.204s	user 0.006s	sys 0.003s
I20260709 13:21:44.801595 23296 consensus_queue.cc:1048] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:21:44.814254 23086 tablet_bootstrap.cc:492] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:44.815860 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:44.821103 22783 leader_election.cc:304] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:44.822337 22785 leader_election.cc:304] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:44.823462 23282 raft_consensus.cc:2804] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:21:44.823652 22785 leader_election.cc:343] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:44.823982 23282 raft_consensus.cc:493] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:44.824410 23282 raft_consensus.cc:3060] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.825838 23353 raft_consensus.cc:2804] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:44.826296 23353 raft_consensus.cc:493] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:44.826459 22858 leader_election.cc:304] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:44.827220 23353 raft_consensus.cc:3060] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.828250 23354 raft_consensus.cc:2804] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:44.803750 23297 raft_consensus.cc:493] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:44.829691 23354 raft_consensus.cc:697] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:44.838912 23187 raft_consensus.cc:493] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:44.864849 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:44.867933 23296 raft_consensus.cc:493] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:44.868520 23296 raft_consensus.cc:515] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:44.869752 23086 tablet_bootstrap.cc:654] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.848431 23301 raft_consensus.cc:515] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:44.870669 23296 leader_election.cc:290] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:44.872025 23301 leader_election.cc:290] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:44.874431 23354 consensus_queue.cc:237] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
W20260709 13:21:44.879453 22785 leader_election.cc:343] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:44.893620 23087 tablet_bootstrap.cc:492] T d1934129c8764c018495eff97d43a3b5 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:44.894186 23087 ts_tablet_manager.cc:1403] T d1934129c8764c018495eff97d43a3b5 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.319s	user 0.041s	sys 0.072s
I20260709 13:21:44.897010 22965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "745e7e3bed5b4d59949aeddcede543f3" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:44.862418 23125 consensus_queue.cc:1048] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 13:21:44.904877 22967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171526d2fe264ee8b1168192bec31c7a" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:44.907161 23303 raft_consensus.cc:515] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:44.918229 23084 tablet_bootstrap.cc:654] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.922235 23357 consensus_queue.cc:1048] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:44.882164 23297 raft_consensus.cc:515] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:44.931624 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:44.932468 22817 raft_consensus.cc:3060] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:44.912086 23303 leader_election.cc:290] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:44.911865 23043 raft_consensus.cc:2468] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:44.892001 23187 raft_consensus.cc:515] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:44.946298 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "087d64074b5943f0a7d789ead06f4783" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:44.951035 23086 tablet_bootstrap.cc:492] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:44.951269 23087 raft_consensus.cc:359] T d1934129c8764c018495eff97d43a3b5 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:44.952018 23087 raft_consensus.cc:385] T d1934129c8764c018495eff97d43a3b5 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.952328 23087 raft_consensus.cc:740] T d1934129c8764c018495eff97d43a3b5 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.953061 23087 consensus_queue.cc:260] T d1934129c8764c018495eff97d43a3b5 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:44.954017 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:44.956060 23086 ts_tablet_manager.cc:1403] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.142s	user 0.013s	sys 0.009s
I20260709 13:21:44.937759 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
W20260709 13:21:44.918959 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 79386.4us
I20260709 13:21:44.990216 23296 consensus_queue.cc:1048] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:44.991338 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "087d64074b5943f0a7d789ead06f4783" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:44.992219 22819 raft_consensus.cc:2468] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 0.
I20260709 13:21:44.950368 22860 leader_election.cc:304] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:44.998448 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad9b8807851437d9a22c0a33f567c15" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:44.998173 22858 leader_election.cc:304] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:45.000589 22967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad9b8807851437d9a22c0a33f567c15" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:44.979592 23282 raft_consensus.cc:515] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:45.003961 23282 leader_election.cc:290] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:44.999912 23087 ts_tablet_manager.cc:1434] T d1934129c8764c018495eff97d43a3b5 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.105s	user 0.004s	sys 0.004s
I20260709 13:21:45.004576 23121 raft_consensus.cc:2804] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:45.013193 23125 consensus_queue.cc:1048] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:45.015024 23087 tablet_bootstrap.cc:492] T 21e1e18beffe450f87ff940a4e9165ea P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:45.010702 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:45.015636 23040 raft_consensus.cc:3060] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:45.015818 23187 leader_election.cc:290] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:45.013542 23121 raft_consensus.cc:493] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:45.022276 23121 raft_consensus.cc:3060] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:45.004769 23210 raft_consensus.cc:2804] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:44.990216 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:45.024004 23210 raft_consensus.cc:697] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:45.024489 22816 raft_consensus.cc:2393] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 in current term 1: Already voted for candidate 91f33bdbdd63438294071eb3776db679 in this term.
W20260709 13:21:45.025554 23358 log.cc:927] Time spent T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:21:45.025068 23210 consensus_queue.cc:237] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
W20260709 13:21:45.005388 22860 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 60983.4us
I20260709 13:21:45.044381 23096 raft_consensus.cc:515] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:45.045883 22892 raft_consensus.cc:2393] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 in current term 1: Already voted for candidate 49cbf11e18ce4268a7fcca76872afb23 in this term.
I20260709 13:21:45.046582 23297 leader_election.cc:290] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:45.048571 23281 raft_consensus.cc:515] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:45.049998 22965 raft_consensus.cc:2393] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 in current term 1: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:45.053136 23281 leader_election.cc:290] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:45.069777 23086 raft_consensus.cc:359] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:45.083952 23086 raft_consensus.cc:385] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:45.084307 23086 raft_consensus.cc:740] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:45.090204 23087 tablet_bootstrap.cc:654] T 21e1e18beffe450f87ff940a4e9165ea P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:45.072052 23121 raft_consensus.cc:515] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:45.074553 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:45.067852 23096 leader_election.cc:290] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
W20260709 13:21:45.077658 22934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 65213.1us
I20260709 13:21:45.091641 23220 raft_consensus.cc:493] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:45.064113 22817 raft_consensus.cc:2468] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 2.
I20260709 13:21:45.069895 23040 raft_consensus.cc:2468] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:45.102509 23086 consensus_queue.cc:260] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
W20260709 13:21:45.067660 22860 leader_election.cc:343] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:45.112134 22785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 168826us
I20260709 13:21:45.116112 23353 raft_consensus.cc:515] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:45.116653 23039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "745e7e3bed5b4d59949aeddcede543f3" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:45.104194 23220 raft_consensus.cc:515] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:45.129351 23220 leader_election.cc:290] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:45.130968 22967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39611d2ff3524bc0840e4fd222d96a84" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:45.131673 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:45.131714 22967 raft_consensus.cc:2468] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
I20260709 13:21:45.136730 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39611d2ff3524bc0840e4fd222d96a84" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:45.137513 23041 raft_consensus.cc:2468] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 0.
I20260709 13:21:45.137707 23121 leader_election.cc:290] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:45.137903 23353 leader_election.cc:290] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:45.139343 22932 leader_election.cc:304] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
W20260709 13:21:45.147614 22785 leader_election.cc:343] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:45.148502 22785 leader_election.cc:304] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 
W20260709 13:21:45.150571 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 156360us
I20260709 13:21:45.155308 23371 raft_consensus.cc:2804] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:45.155929 23371 raft_consensus.cc:493] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:45.156278 23371 raft_consensus.cc:3060] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:45.158608 22967 raft_consensus.cc:1240] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Rejecting Update request from peer 49cbf11e18ce4268a7fcca76872afb23 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:45.162468 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "745e7e3bed5b4d59949aeddcede543f3" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:45.163434 22966 raft_consensus.cc:2393] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 in current term 1: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
W20260709 13:21:45.146073 22860 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 59123.1us
I20260709 13:21:45.169416 23039 raft_consensus.cc:2393] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 in current term 1: Already voted for candidate 49cbf11e18ce4268a7fcca76872afb23 in this term.
I20260709 13:21:45.170562 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:45.172418 23085 tablet_bootstrap.cc:654] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:45.172829 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "745e7e3bed5b4d59949aeddcede543f3" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
W20260709 13:21:45.174782 22860 leader_election.cc:343] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:45.175921 22860 leader_election.cc:343] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:45.179396 23361 raft_consensus.cc:2804] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:45.179930 23361 raft_consensus.cc:493] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:45.180387 23361 raft_consensus.cc:3060] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:45.174706 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1934129c8764c018495eff97d43a3b5" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:45.184623 22816 raft_consensus.cc:1240] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Rejecting Update request from peer 49cbf11e18ce4268a7fcca76872afb23 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:21:45.184298 23371 raft_consensus.cc:515] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:45.186527 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:45.187000 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:45.187397 22892 raft_consensus.cc:2393] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 in current term 1: Already voted for candidate 49cbf11e18ce4268a7fcca76872afb23 in this term.
I20260709 13:21:45.187789 22820 raft_consensus.cc:2393] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 in current term 1: Already voted for candidate 91f33bdbdd63438294071eb3776db679 in this term.
I20260709 13:21:45.188747 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:45.189734 22819 raft_consensus.cc:2393] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 in current term 1: Already voted for candidate 91f33bdbdd63438294071eb3776db679 in this term.
I20260709 13:21:45.184489 22891 raft_consensus.cc:2468] T d1934129c8764c018495eff97d43a3b5 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 0.
I20260709 13:21:45.191179 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39611d2ff3524bc0840e4fd222d96a84" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:45.191872 23041 raft_consensus.cc:3060] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:45.187500 22860 leader_election.cc:304] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:45.201633 23121 raft_consensus.cc:2804] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:45.202095 23121 raft_consensus.cc:493] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:45.202423 23121 raft_consensus.cc:3060] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:45.203882 22934 leader_election.cc:304] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:45.204978 23220 raft_consensus.cc:2749] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:45.214344 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:45.214874 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39611d2ff3524bc0840e4fd222d96a84" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:45.217768 23121 raft_consensus.cc:515] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:45.218118 23041 raft_consensus.cc:2468] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
W20260709 13:21:45.224081 23008 leader_election.cc:343] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:45.225895 22785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 71141.4us
I20260709 13:21:45.214246 23039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:45.234057 23086 ts_tablet_manager.cc:1434] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.278s	user 0.007s	sys 0.003s
I20260709 13:21:45.236603 23084 tablet_bootstrap.cc:492] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:45.230235 23008 leader_election.cc:304] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:45.237247 23084 ts_tablet_manager.cc:1403] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.549s	user 0.011s	sys 0.016s
W20260709 13:21:45.248103 23008 leader_election.cc:343] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:45.248695 23008 leader_election.cc:304] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:45.258163 23085 tablet_bootstrap.cc:492] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:45.258726 23085 ts_tablet_manager.cc:1403] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.559s	user 0.018s	sys 0.024s
I20260709 13:21:45.257691 23363 raft_consensus.cc:2749] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:45.260279 23375 raft_consensus.cc:2804] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:21:45.236691 22783 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 62456.1us
I20260709 13:21:45.250581 23039 raft_consensus.cc:3060] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:45.270356 23375 raft_consensus.cc:493] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:45.269848 23371 leader_election.cc:290] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:45.276300 23375 raft_consensus.cc:3060] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:45.301265 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1934129c8764c018495eff97d43a3b5" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:45.302054 22819 raft_consensus.cc:2468] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 0.
I20260709 13:21:45.306422 23357 consensus_queue.cc:1059] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:45.306638 22860 leader_election.cc:304] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:45.307403 23357 raft_consensus.cc:3055] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:45.307758 23357 raft_consensus.cc:740] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:45.308339 23354 raft_consensus.cc:2749] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:45.308669 23357 consensus_queue.cc:260] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:45.312367 23087 tablet_bootstrap.cc:492] T 21e1e18beffe450f87ff940a4e9165ea P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:45.312896 23087 ts_tablet_manager.cc:1403] T 21e1e18beffe450f87ff940a4e9165ea P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.298s	user 0.026s	sys 0.025s
I20260709 13:21:45.314924 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39611d2ff3524bc0840e4fd222d96a84" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:45.317565 23087 raft_consensus.cc:359] T 21e1e18beffe450f87ff940a4e9165ea P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:45.318343 23087 raft_consensus.cc:385] T 21e1e18beffe450f87ff940a4e9165ea P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:45.318651 23084 raft_consensus.cc:359] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:45.319365 23084 raft_consensus.cc:385] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:45.319610 23084 raft_consensus.cc:740] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:45.320233 23084 consensus_queue.cc:260] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:45.262686 22932 leader_election.cc:304] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:45.321471 23357 raft_consensus.cc:3060] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:45.327226 23084 ts_tablet_manager.cc:1434] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.090s	user 0.006s	sys 0.001s
I20260709 13:21:45.328493 23084 tablet_bootstrap.cc:492] T 171526d2fe264ee8b1168192bec31c7a P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:45.318643 23087 raft_consensus.cc:740] T 21e1e18beffe450f87ff940a4e9165ea P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:45.329241 23086 tablet_bootstrap.cc:492] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:45.330803 22934 leader_election.cc:304] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:45.340032 23087 consensus_queue.cc:260] T 21e1e18beffe450f87ff940a4e9165ea P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:45.346556 22783 leader_election.cc:304] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:45.353467 23361 raft_consensus.cc:515] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:45.342705 23281 raft_consensus.cc:2804] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:45.341198 23371 raft_consensus.cc:2804] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:45.362614 23087 ts_tablet_manager.cc:1434] T 21e1e18beffe450f87ff940a4e9165ea P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.049s	user 0.009s	sys 0.000s
I20260709 13:21:45.355060 22816 raft_consensus.cc:2393] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 in current term 1: Already voted for candidate 91f33bdbdd63438294071eb3776db679 in this term.
W20260709 13:21:45.341440 22932 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 85869.6us
W20260709 13:21:45.369184 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 48797.9us
I20260709 13:21:45.362727 23282 raft_consensus.cc:2804] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:45.375442 23085 raft_consensus.cc:359] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:45.376188 23085 raft_consensus.cc:385] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:45.376430 23085 raft_consensus.cc:740] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:45.368788 23371 raft_consensus.cc:697] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 2 LEADER]: Becoming Leader. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:21:45.374058 23281 raft_consensus.cc:697] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 [term 1 LEADER]: Becoming Leader. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:21:45.384524 23085 consensus_queue.cc:260] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:45.375419 23121 leader_election.cc:290] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:45.386099 23087 tablet_bootstrap.cc:492] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:45.378320 23282 raft_consensus.cc:697] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:45.394626 23085 ts_tablet_manager.cc:1434] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.136s	user 0.007s	sys 0.001s
W20260709 13:21:45.386071 22860 leader_election.cc:343] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:45.399420 22860 leader_election.cc:304] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:45.405961 23354 raft_consensus.cc:2749] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:45.407948 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "087d64074b5943f0a7d789ead06f4783" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:45.411522 23087 tablet_bootstrap.cc:654] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:45.412904 22785 leader_election.cc:343] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:45.413757 22785 leader_election.cc:304] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:45.420993 23369 raft_consensus.cc:2749] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:45.402222 23375 raft_consensus.cc:515] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:45.423666 23375 leader_election.cc:290] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:45.424854 23086 tablet_bootstrap.cc:654] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:45.417263 23371 consensus_queue.cc:237] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:45.431455 23039 raft_consensus.cc:2468] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:45.412729 22818 raft_consensus.cc:3060] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:45.439834 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 63734.9us
I20260709 13:21:45.440523 23085 tablet_bootstrap.cc:492] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:45.418511 23361 leader_election.cc:290] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:45.410005 23281 consensus_queue.cc:237] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:45.463665 22814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1934129c8764c018495eff97d43a3b5" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:45.464401 22814 raft_consensus.cc:3060] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:45.466205 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1934129c8764c018495eff97d43a3b5" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:45.466884 22891 raft_consensus.cc:3060] T d1934129c8764c018495eff97d43a3b5 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:45.447491 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "087d64074b5943f0a7d789ead06f4783" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:45.433282 23282 consensus_queue.cc:237] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
W20260709 13:21:45.488454 22785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 69819.7us
I20260709 13:21:45.490067 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39611d2ff3524bc0840e4fd222d96a84" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:45.492007 23086 tablet_bootstrap.cc:492] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:45.492755 23086 ts_tablet_manager.cc:1403] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.230s	user 0.015s	sys 0.008s
I20260709 13:21:45.496163 23086 raft_consensus.cc:359] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:45.497155 23086 raft_consensus.cc:385] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:45.497517 23086 raft_consensus.cc:740] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:45.486986 23121 raft_consensus.cc:493] T 21e1e18beffe450f87ff940a4e9165ea P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:45.507511 23086 consensus_queue.cc:260] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:45.512260 23121 raft_consensus.cc:515] T 21e1e18beffe450f87ff940a4e9165ea P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:45.516006 23121 leader_election.cc:290] T 21e1e18beffe450f87ff940a4e9165ea P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:45.527527 23085 tablet_bootstrap.cc:654] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:45.529636 23086 ts_tablet_manager.cc:1434] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.036s	user 0.004s	sys 0.003s
I20260709 13:21:45.531320 23086 tablet_bootstrap.cc:492] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:45.532193 23084 tablet_bootstrap.cc:654] T 171526d2fe264ee8b1168192bec31c7a P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:45.531232 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39611d2ff3524bc0840e4fd222d96a84" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:45.537566 23041 raft_consensus.cc:2393] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 in current term 1: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:45.539300 23086 tablet_bootstrap.cc:654] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:45.529608 22783 leader_election.cc:304] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:45.547807 23362 raft_consensus.cc:2804] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:45.548357 23182 raft_consensus.cc:493] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:45.548619 22860 leader_election.cc:343] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:45.548359 23362 raft_consensus.cc:697] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:45.561779 23087 tablet_bootstrap.cc:492] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:45.562451 23087 ts_tablet_manager.cc:1403] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.184s	user 0.024s	sys 0.041s
I20260709 13:21:45.563922 23182 raft_consensus.cc:515] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:45.511075 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39611d2ff3524bc0840e4fd222d96a84" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:45.551169 22814 raft_consensus.cc:2468] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
W20260709 13:21:45.566248 22766 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:45.566370 22818 raft_consensus.cc:2468] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:45.595782 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1e18beffe450f87ff940a4e9165ea" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:45.596817 22815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1e18beffe450f87ff940a4e9165ea" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:45.597025 23008 leader_election.cc:304] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:45.600045 23357 consensus_queue.cc:1048] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:45.604599 23087 raft_consensus.cc:359] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:45.614985 23087 raft_consensus.cc:385] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:45.615324 23087 raft_consensus.cc:740] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:45.615828 23387 raft_consensus.cc:2804] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:45.616452 23387 raft_consensus.cc:697] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:45.606958 23354 consensus_queue.cc:1048] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:45.617105 22717 catalog_manager.cc:5697] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 1 to 2, leader changed from 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66) to 8ef3887fb3874f8b81079e542d252944 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "8ef3887fb3874f8b81079e542d252944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:45.618676 22785 leader_election.cc:304] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:45.619474 23282 raft_consensus.cc:2749] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:45.620828 23182 leader_election.cc:290] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:45.616016 23087 consensus_queue.cc:260] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:45.619024 22717 catalog_manager.cc:5697] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ef3887fb3874f8b81079e542d252944 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "8ef3887fb3874f8b81079e542d252944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:45.624001 23087 ts_tablet_manager.cc:1434] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.000s
I20260709 13:21:45.611994 22891 raft_consensus.cc:2468] T d1934129c8764c018495eff97d43a3b5 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
I20260709 13:21:45.612496 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:45.617441 23387 consensus_queue.cc:237] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:45.629302 23354 consensus_queue.cc:1048] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:45.642269 22816 raft_consensus.cc:2393] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 in current term 1: Already voted for candidate 91f33bdbdd63438294071eb3776db679 in this term.
I20260709 13:21:45.551265 23362 consensus_queue.cc:237] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:45.630216 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:45.644150 22893 raft_consensus.cc:2393] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 in current term 1: Already voted for candidate 49cbf11e18ce4268a7fcca76872afb23 in this term.
I20260709 13:21:45.631676 23087 tablet_bootstrap.cc:492] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:45.640918 23085 tablet_bootstrap.cc:492] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:45.641351 23084 tablet_bootstrap.cc:492] T 171526d2fe264ee8b1168192bec31c7a P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:45.646962 23085 ts_tablet_manager.cc:1403] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.248s	user 0.011s	sys 0.007s
I20260709 13:21:45.647097 23084 ts_tablet_manager.cc:1403] T 171526d2fe264ee8b1168192bec31c7a P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.319s	user 0.015s	sys 0.020s
W20260709 13:21:45.621486 22860 consensus_peers.cc:597] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Couldn't send request to peer 8ef3887fb3874f8b81079e542d252944. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:45.650406 22934 leader_election.cc:304] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
W20260709 13:21:45.650137 22932 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 82623.2us
I20260709 13:21:45.651767 23182 raft_consensus.cc:3060] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:45.654148 22860 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 47473.5us
W20260709 13:21:45.660030 22860 leader_election.cc:343] T 21e1e18beffe450f87ff940a4e9165ea P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:45.667857 23084 raft_consensus.cc:359] T 171526d2fe264ee8b1168192bec31c7a P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:45.668484 23084 raft_consensus.cc:385] T 171526d2fe264ee8b1168192bec31c7a P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:45.668692 23084 raft_consensus.cc:740] T 171526d2fe264ee8b1168192bec31c7a P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:45.671270 22860 leader_election.cc:304] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
W20260709 13:21:45.672660 22860 leader_election.cc:343] T 21e1e18beffe450f87ff940a4e9165ea P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:45.673123 22860 leader_election.cc:304] T 21e1e18beffe450f87ff940a4e9165ea P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:45.673731 23357 raft_consensus.cc:2804] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:45.674369 23357 raft_consensus.cc:697] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:45.675228 23357 consensus_queue.cc:237] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:45.678529 23277 raft_consensus.cc:2749] T 21e1e18beffe450f87ff940a4e9165ea P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:45.689291 23084 consensus_queue.cc:260] T 171526d2fe264ee8b1168192bec31c7a P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:45.700740 23087 tablet_bootstrap.cc:654] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:45.703316 23086 tablet_bootstrap.cc:492] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:45.703845 23086 ts_tablet_manager.cc:1403] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.173s	user 0.017s	sys 0.010s
I20260709 13:21:45.712069 23182 raft_consensus.cc:2749] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:45.712831 23364 raft_consensus.cc:493] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:45.713344 23364 raft_consensus.cc:515] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:45.716151 23364 leader_election.cc:290] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:45.717293 22815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7802e4d0fd394596b42b661c9eb4c7ce" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:45.720033 22714 catalog_manager.cc:5697] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:45.723016 23085 raft_consensus.cc:359] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:45.731853 23085 raft_consensus.cc:385] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:45.723134 23086 raft_consensus.cc:359] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:45.732863 23086 raft_consensus.cc:385] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:45.732298 23085 raft_consensus.cc:740] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:45.733687 23086 raft_consensus.cc:740] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:45.734557 23085 consensus_queue.cc:260] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:45.735303 23086 consensus_queue.cc:260] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:45.747448 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7802e4d0fd394596b42b661c9eb4c7ce" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:45.735451 22815 raft_consensus.cc:2468] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 0.
I20260709 13:21:45.735918 22714 catalog_manager.cc:5697] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:45.753724 23084 ts_tablet_manager.cc:1434] T 171526d2fe264ee8b1168192bec31c7a P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.106s	user 0.004s	sys 0.002s
I20260709 13:21:45.752337 22716 catalog_manager.cc:5697] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 1 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:45.764844 23008 leader_election.cc:304] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:45.769430 23296 raft_consensus.cc:2804] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:45.769868 23296 raft_consensus.cc:493] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:45.770215 23296 raft_consensus.cc:3060] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:45.782263 23008 leader_election.cc:343] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:45.790310 23084 tablet_bootstrap.cc:492] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:45.791512 23296 raft_consensus.cc:515] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:45.793471 23296 leader_election.cc:290] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:45.795980 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7802e4d0fd394596b42b661c9eb4c7ce" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
W20260709 13:21:45.797549 23008 leader_election.cc:343] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:45.798738 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7802e4d0fd394596b42b661c9eb4c7ce" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:45.799429 22816 raft_consensus.cc:3060] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:45.809834 23084 tablet_bootstrap.cc:654] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:45.813742 23085 ts_tablet_manager.cc:1434] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.167s	user 0.005s	sys 0.003s
I20260709 13:21:45.815075 23085 tablet_bootstrap.cc:492] T 78234c90229d4615ab9e2c2a811bd0f8 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:45.812484 23086 ts_tablet_manager.cc:1434] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.108s	user 0.005s	sys 0.006s
I20260709 13:21:45.819913 23087 tablet_bootstrap.cc:492] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:45.820495 23087 ts_tablet_manager.cc:1403] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.189s	user 0.014s	sys 0.023s
I20260709 13:21:45.826910 23086 tablet_bootstrap.cc:492] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:45.809962 22816 raft_consensus.cc:2468] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
I20260709 13:21:45.828855 23008 leader_election.cc:304] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:45.830168 23364 raft_consensus.cc:2804] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:45.831189 23364 raft_consensus.cc:697] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:45.831980 23087 raft_consensus.cc:359] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:45.832196 23364 consensus_queue.cc:237] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:45.832792 23087 raft_consensus.cc:385] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:45.836222 23087 raft_consensus.cc:740] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:45.837631 23087 consensus_queue.cc:260] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:45.838887 23085 tablet_bootstrap.cc:654] T 78234c90229d4615ab9e2c2a811bd0f8 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:45.842651 23087 ts_tablet_manager.cc:1434] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 13:21:45.844132 23087 tablet_bootstrap.cc:492] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:45.842669 23086 tablet_bootstrap.cc:654] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:45.849301 23371 raft_consensus.cc:493] T 171526d2fe264ee8b1168192bec31c7a P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:45.849354 23084 tablet_bootstrap.cc:492] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:45.850034 23371 raft_consensus.cc:515] T 171526d2fe264ee8b1168192bec31c7a P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:45.851105 23087 tablet_bootstrap.cc:654] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:45.852403 23084 ts_tablet_manager.cc:1403] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.062s	user 0.007s	sys 0.012s
I20260709 13:21:45.854149 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171526d2fe264ee8b1168192bec31c7a" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:45.854434 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171526d2fe264ee8b1168192bec31c7a" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:45.852380 23371 leader_election.cc:290] T 171526d2fe264ee8b1168192bec31c7a P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:45.856643 23084 raft_consensus.cc:359] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:45.857462 23084 raft_consensus.cc:385] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:45.857745 23084 raft_consensus.cc:740] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:45.858666 23084 consensus_queue.cc:260] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:45.861052 22934 leader_election.cc:304] T 171526d2fe264ee8b1168192bec31c7a P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:45.861394 23084 ts_tablet_manager.cc:1434] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 13:21:45.862622 23084 tablet_bootstrap.cc:492] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:45.863512 23087 tablet_bootstrap.cc:492] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:45.863521 23371 raft_consensus.cc:2749] T 171526d2fe264ee8b1168192bec31c7a P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:45.864686 23301 raft_consensus.cc:493] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:45.865288 23301 raft_consensus.cc:515] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:45.868466 23084 tablet_bootstrap.cc:654] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:45.869431 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b752c38e1b47a489a3d720801661d1" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:45.870172 23041 raft_consensus.cc:2468] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
I20260709 13:21:45.871127 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b752c38e1b47a489a3d720801661d1" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:45.871410 22783 leader_election.cc:304] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
W20260709 13:21:45.872741 22785 leader_election.cc:343] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:45.870836 23301 leader_election.cc:290] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:45.874284 23087 ts_tablet_manager.cc:1403] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.008s
I20260709 13:21:45.874428 23371 consensus_queue.cc:1048] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:45.873109 23085 tablet_bootstrap.cc:492] T 78234c90229d4615ab9e2c2a811bd0f8 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:45.875563 23085 ts_tablet_manager.cc:1403] T 78234c90229d4615ab9e2c2a811bd0f8 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.061s	user 0.013s	sys 0.000s
I20260709 13:21:45.873402 23282 raft_consensus.cc:2804] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:45.877101 23282 raft_consensus.cc:493] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:45.877460 23282 raft_consensus.cc:3060] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:45.877312 23087 raft_consensus.cc:359] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:45.878072 23087 raft_consensus.cc:385] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:45.878372 23087 raft_consensus.cc:740] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:45.879222 23087 consensus_queue.cc:260] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:45.881870 23087 ts_tablet_manager.cc:1434] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:21:45.882884 23087 tablet_bootstrap.cc:492] T 7802e4d0fd394596b42b661c9eb4c7ce P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:45.884013 22816 raft_consensus.cc:1275] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Refusing update from remote peer 8ef3887fb3874f8b81079e542d252944: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:45.888124 23282 raft_consensus.cc:515] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:45.889130 23301 consensus_queue.cc:1048] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:45.889961 23087 tablet_bootstrap.cc:654] T 7802e4d0fd394596b42b661c9eb4c7ce P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:45.891170 23282 leader_election.cc:290] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:45.891762 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b752c38e1b47a489a3d720801661d1" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:45.892385 23041 raft_consensus.cc:3060] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:45.892798 23371 consensus_queue.cc:1048] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:45.892488 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b752c38e1b47a489a3d720801661d1" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
W20260709 13:21:45.899071 22785 leader_election.cc:343] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:45.899852 23041 raft_consensus.cc:2468] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:45.901193 22783 leader_election.cc:304] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:45.902778 23282 raft_consensus.cc:2804] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:45.903615 23282 raft_consensus.cc:697] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:45.904603 23282 consensus_queue.cc:237] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:45.906096 23357 raft_consensus.cc:493] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:45.906721 23357 raft_consensus.cc:515] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:45.914557 23357 leader_election.cc:290] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:45.916304 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:45.916567 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:45.924371 22895 raft_consensus.cc:1275] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Refusing update from remote peer 8ef3887fb3874f8b81079e542d252944: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:21:45.921984 22860 leader_election.cc:343] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:45.927462 23085 raft_consensus.cc:359] T 78234c90229d4615ab9e2c2a811bd0f8 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
W20260709 13:21:45.931483 22860 leader_election.cc:343] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:45.935055 22860 leader_election.cc:304] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:45.936137 23397 consensus_queue.cc:1048] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:21:45.938572 23084 tablet_bootstrap.cc:492] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:45.939378 23084 ts_tablet_manager.cc:1403] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.077s	user 0.005s	sys 0.022s
I20260709 13:21:45.937345 23357 raft_consensus.cc:2749] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:45.930809 23085 raft_consensus.cc:385] T 78234c90229d4615ab9e2c2a811bd0f8 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:45.944669 23085 raft_consensus.cc:740] T 78234c90229d4615ab9e2c2a811bd0f8 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:45.948637 23085 consensus_queue.cc:260] T 78234c90229d4615ab9e2c2a811bd0f8 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:45.968281 23085 ts_tablet_manager.cc:1434] T 78234c90229d4615ab9e2c2a811bd0f8 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.092s	user 0.006s	sys 0.002s
I20260709 13:21:45.978861 23085 tablet_bootstrap.cc:492] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:45.979702 23087 tablet_bootstrap.cc:492] T 7802e4d0fd394596b42b661c9eb4c7ce P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:45.980244 23087 ts_tablet_manager.cc:1403] T 7802e4d0fd394596b42b661c9eb4c7ce P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.098s	user 0.013s	sys 0.006s
I20260709 13:21:45.984989 23084 raft_consensus.cc:359] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:45.985798 23084 raft_consensus.cc:385] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:45.986109 23084 raft_consensus.cc:740] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:45.986822 23084 consensus_queue.cc:260] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:45.983593 23096 raft_consensus.cc:493] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:45.990429 23096 raft_consensus.cc:515] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:45.995215 23182 consensus_queue.cc:1048] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:45.989984 22718 catalog_manager.cc:5697] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:46.003993 22718 catalog_manager.cc:5697] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:46.006556 23086 tablet_bootstrap.cc:492] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:46.007179 23086 ts_tablet_manager.cc:1403] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.181s	user 0.022s	sys 0.032s
I20260709 13:21:46.010824 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78234c90229d4615ab9e2c2a811bd0f8" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:46.011762 23277 raft_consensus.cc:493] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:46.012019 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78234c90229d4615ab9e2c2a811bd0f8" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:46.012244 23087 raft_consensus.cc:359] T 7802e4d0fd394596b42b661c9eb4c7ce P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:46.012753 23040 raft_consensus.cc:2468] T 78234c90229d4615ab9e2c2a811bd0f8 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 0.
I20260709 13:21:46.014412 22858 leader_election.cc:304] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:46.015266 23354 raft_consensus.cc:2804] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:46.015658 23354 raft_consensus.cc:493] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:46.015969 23354 raft_consensus.cc:3060] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:46.017642 23087 raft_consensus.cc:385] T 7802e4d0fd394596b42b661c9eb4c7ce P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:46.017987 23087 raft_consensus.cc:740] T 7802e4d0fd394596b42b661c9eb4c7ce P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:46.018780 23087 consensus_queue.cc:260] T 7802e4d0fd394596b42b661c9eb4c7ce P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:46.021200 23087 ts_tablet_manager.cc:1434] T 7802e4d0fd394596b42b661c9eb4c7ce P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.041s	user 0.004s	sys 0.003s
I20260709 13:21:46.022292 23087 tablet_bootstrap.cc:492] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:46.022121 23277 raft_consensus.cc:515] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:45.995283 23096 leader_election.cc:290] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:46.023679 23354 raft_consensus.cc:515] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:46.024523 23277 leader_election.cc:290] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:46.025233 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44841295bfd54159a48e285582824bad" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:46.025846 23354 leader_election.cc:290] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
W20260709 13:21:46.029868 22860 leader_election.cc:343] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:46.031632 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78234c90229d4615ab9e2c2a811bd0f8" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:46.032282 23041 raft_consensus.cc:3060] T 78234c90229d4615ab9e2c2a811bd0f8 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:46.034006 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78234c90229d4615ab9e2c2a811bd0f8" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
W20260709 13:21:46.030625 22860 leader_election.cc:343] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:46.034593 22963 raft_consensus.cc:3060] T 171526d2fe264ee8b1168192bec31c7a P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:46.038395 23084 ts_tablet_manager.cc:1434] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.099s	user 0.002s	sys 0.006s
W20260709 13:21:46.041359 22860 leader_election.cc:343] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:46.042236 23353 consensus_queue.cc:1048] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:21:46.050382 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44841295bfd54159a48e285582824bad" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:46.052666 23086 raft_consensus.cc:359] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:46.053493 23086 raft_consensus.cc:385] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:46.053778 23086 raft_consensus.cc:740] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:46.055203 23086 consensus_queue.cc:260] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:46.057518 23086 ts_tablet_manager.cc:1434] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.050s	user 0.006s	sys 0.000s
W20260709 13:21:46.058391 22858 leader_election.cc:343] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:46.058934 22858 leader_election.cc:304] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:46.062703 23085 tablet_bootstrap.cc:654] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:46.069474 22714 catalog_manager.cc:5697] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 1 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:46.070020 23296 consensus_queue.cc:1048] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:46.072739 23086 tablet_bootstrap.cc:492] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:46.077670 23353 consensus_queue.cc:1048] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:46.080421 23096 raft_consensus.cc:2749] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:46.080793 23084 tablet_bootstrap.cc:492] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:46.094218 23087 tablet_bootstrap.cc:654] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:46.097488 23397 raft_consensus.cc:493] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:46.108057 23397 raft_consensus.cc:515] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:46.113844 23086 tablet_bootstrap.cc:654] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:46.130549 23038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:46.131780 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:46.136803 22932 leader_election.cc:304] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:46.142925 23276 raft_consensus.cc:2749] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:46.143329 23364 consensus_queue.cc:1048] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:46.144688 22715 catalog_manager.cc:5697] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:46.120060 23397 leader_election.cc:290] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:46.163715 22969 raft_consensus.cc:3060] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:46.164497 23282 raft_consensus.cc:493] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:46.165302 23282 raft_consensus.cc:515] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:46.173827 23086 tablet_bootstrap.cc:492] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:46.174425 23086 ts_tablet_manager.cc:1403] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.102s	user 0.011s	sys 0.009s
I20260709 13:21:46.174865 23084 tablet_bootstrap.cc:654] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:46.175907 23085 tablet_bootstrap.cc:492] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:46.176434 23085 ts_tablet_manager.cc:1403] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.198s	user 0.016s	sys 0.051s
I20260709 13:21:46.177114 23086 raft_consensus.cc:359] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:46.177834 23086 raft_consensus.cc:385] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:46.178169 23086 raft_consensus.cc:740] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:46.181777 23086 consensus_queue.cc:260] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:46.187162 23086 ts_tablet_manager.cc:1434] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 13:21:46.188861 23086 tablet_bootstrap.cc:492] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:46.190817 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78234c90229d4615ab9e2c2a811bd0f8" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:46.191727 22891 raft_consensus.cc:2393] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 in current term 1: Already voted for candidate 49cbf11e18ce4268a7fcca76872afb23 in this term.
I20260709 13:21:46.187207 23041 raft_consensus.cc:2468] T 78234c90229d4615ab9e2c2a811bd0f8 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:46.188799 23282 leader_election.cc:290] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:46.211621 23085 raft_consensus.cc:359] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:46.213829 23085 raft_consensus.cc:385] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:46.214174 23085 raft_consensus.cc:740] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:46.214969 23085 consensus_queue.cc:260] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
W20260709 13:21:46.216473 23425 log.cc:927] Time spent T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:21:46.218309 23039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78234c90229d4615ab9e2c2a811bd0f8" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:46.219343 23039 raft_consensus.cc:2393] T 78234c90229d4615ab9e2c2a811bd0f8 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 in current term 1: Already voted for candidate 49cbf11e18ce4268a7fcca76872afb23 in this term.
I20260709 13:21:46.222564 23277 consensus_queue.cc:1048] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:46.226127 23375 raft_consensus.cc:493] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:46.226716 23375 raft_consensus.cc:515] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:46.208628 23362 consensus_queue.cc:1048] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:21:46.232685 23087 tablet_bootstrap.cc:492] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:46.233273 23087 ts_tablet_manager.cc:1403] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.211s	user 0.009s	sys 0.009s
I20260709 13:21:46.235818 22969 raft_consensus.cc:3060] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:46.237493 22718 catalog_manager.cc:5697] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:46.245901 22858 leader_election.cc:304] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:46.248132 23087 raft_consensus.cc:359] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:46.249037 23087 raft_consensus.cc:385] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:46.249248 23087 raft_consensus.cc:740] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:46.255640 23087 consensus_queue.cc:260] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:46.263281 23085 ts_tablet_manager.cc:1434] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.087s	user 0.009s	sys 0.001s
I20260709 13:21:46.255772 23277 raft_consensus.cc:2804] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:46.265964 23277 raft_consensus.cc:697] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:46.266888 23277 consensus_queue.cc:237] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:46.270406 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44841295bfd54159a48e285582824bad" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:46.272205 23085 tablet_bootstrap.cc:492] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
W20260709 13:21:46.272253 23008 leader_election.cc:343] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:46.272943 23375 leader_election.cc:290] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:46.278515 22783 leader_election.cc:304] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:46.279642 23282 raft_consensus.cc:3060] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:46.285188 23085 tablet_bootstrap.cc:654] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:46.288097 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44841295bfd54159a48e285582824bad" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:46.288955 22892 raft_consensus.cc:2468] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 0.
I20260709 13:21:46.255470 23096 consensus_queue.cc:1048] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.048s
I20260709 13:21:46.313598 23008 leader_election.cc:304] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:46.331354 23084 tablet_bootstrap.cc:492] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:46.331948 23084 ts_tablet_manager.cc:1403] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.272s	user 0.030s	sys 0.021s
I20260709 13:21:46.337239 23363 raft_consensus.cc:2804] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:46.337774 23363 raft_consensus.cc:493] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:46.338164 23363 raft_consensus.cc:3060] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:46.343907 23362 raft_consensus.cc:493] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:46.344599 23362 raft_consensus.cc:515] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:46.349311 23362 leader_election.cc:290] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:46.353590 23084 raft_consensus.cc:359] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:46.354579 23084 raft_consensus.cc:385] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:46.354856 23084 raft_consensus.cc:740] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:46.355559 23084 consensus_queue.cc:260] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:46.358108 23084 ts_tablet_manager.cc:1434] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:21:46.359280 23084 tablet_bootstrap.cc:492] T 44841295bfd54159a48e285582824bad P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:46.356128 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:46.360644 22893 raft_consensus.cc:2468] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
I20260709 13:21:46.349644 23086 tablet_bootstrap.cc:654] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:46.372324 22785 leader_election.cc:304] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:46.373415 23353 raft_consensus.cc:2804] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:46.373840 23353 raft_consensus.cc:493] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:46.374229 23353 raft_consensus.cc:3060] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:46.389760 23296 consensus_queue.cc:1048] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:46.392782 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:46.393077 23087 ts_tablet_manager.cc:1434] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.160s	user 0.007s	sys 0.002s
I20260709 13:21:46.365888 23282 raft_consensus.cc:2749] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:46.405371 23353 raft_consensus.cc:515] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
W20260709 13:21:46.445963 22785 leader_election.cc:343] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:46.447016 23087 tablet_bootstrap.cc:492] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:46.457244 22893 raft_consensus.cc:3060] T 7802e4d0fd394596b42b661c9eb4c7ce P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:46.474170 23397 raft_consensus.cc:493] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:46.474797 23397 raft_consensus.cc:515] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:46.476840 23397 leader_election.cc:290] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:46.478047 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1e18beffe450f87ff940a4e9165ea" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:46.478374 22714 catalog_manager.cc:5697] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } }
W20260709 13:21:46.479357 22934 leader_election.cc:343] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:46.479498 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1e18beffe450f87ff940a4e9165ea" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:46.494619 23085 tablet_bootstrap.cc:492] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:46.495380 23085 ts_tablet_manager.cc:1403] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.223s	user 0.023s	sys 0.041s
I20260709 13:21:46.489001 22891 raft_consensus.cc:2468] T 21e1e18beffe450f87ff940a4e9165ea P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 0.
I20260709 13:21:46.498018 23086 tablet_bootstrap.cc:492] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:46.497907 22934 leader_election.cc:304] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:46.498544 23086 ts_tablet_manager.cc:1403] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.310s	user 0.016s	sys 0.029s
I20260709 13:21:46.503309 23397 raft_consensus.cc:2804] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:46.489586 23363 raft_consensus.cc:515] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:46.503809 23397 raft_consensus.cc:493] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:46.512661 23397 raft_consensus.cc:3060] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:46.514274 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44841295bfd54159a48e285582824bad" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:46.514950 22892 raft_consensus.cc:3060] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:46.521831 23353 leader_election.cc:290] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:46.529755 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44841295bfd54159a48e285582824bad" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
W20260709 13:21:46.531280 23008 leader_election.cc:343] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:46.531543 23084 tablet_bootstrap.cc:654] T 44841295bfd54159a48e285582824bad P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:46.534430 23435 log.cc:927] Time spent T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:21:46.535933 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:46.536909 22891 raft_consensus.cc:3060] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:46.536096 23363 leader_election.cc:290] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:46.539351 23096 raft_consensus.cc:493] T 7802e4d0fd394596b42b661c9eb4c7ce P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:46.539881 23096 raft_consensus.cc:515] T 7802e4d0fd394596b42b661c9eb4c7ce P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:46.546828 23086 raft_consensus.cc:359] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
W20260709 13:21:46.559598 22783 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 61199us
I20260709 13:21:46.548935 23085 raft_consensus.cc:359] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:46.553151 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:46.554080 23296 consensus_queue.cc:1048] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:21:46.554620 22785 consensus_peers.cc:597] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 -> Peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Couldn't send request to peer 8ef3887fb3874f8b81079e542d252944. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:46.536057 23397 raft_consensus.cc:515] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:46.560020 23086 raft_consensus.cc:385] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:46.567569 23086 raft_consensus.cc:740] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
W20260709 13:21:46.568269 22785 leader_election.cc:343] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:46.560325 23085 raft_consensus.cc:385] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:46.568864 23085 raft_consensus.cc:740] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:46.558533 23087 tablet_bootstrap.cc:654] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:46.568334 22815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1e18beffe450f87ff940a4e9165ea" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:46.569626 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1e18beffe450f87ff940a4e9165ea" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:46.564339 23362 consensus_queue.cc:1048] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:21:46.571017 22895 raft_consensus.cc:3060] T 21e1e18beffe450f87ff940a4e9165ea P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:46.568400 23086 consensus_queue.cc:260] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:46.534526 22892 raft_consensus.cc:2468] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
I20260709 13:21:46.569653 23085 consensus_queue.cc:260] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
W20260709 13:21:46.574899 22934 leader_election.cc:343] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:46.577271 23086 ts_tablet_manager.cc:1434] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.078s	user 0.007s	sys 0.000s
I20260709 13:21:46.579149 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7802e4d0fd394596b42b661c9eb4c7ce" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:46.582252 23085 ts_tablet_manager.cc:1434] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.086s	user 0.005s	sys 0.002s
I20260709 13:21:46.597383 23084 tablet_bootstrap.cc:492] T 44841295bfd54159a48e285582824bad P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:46.604847 23084 ts_tablet_manager.cc:1403] T 44841295bfd54159a48e285582824bad P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.246s	user 0.008s	sys 0.008s
I20260709 13:21:46.612998 22895 raft_consensus.cc:2468] T 21e1e18beffe450f87ff940a4e9165ea P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:46.614838 22934 leader_election.cc:304] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:46.615533 23182 raft_consensus.cc:2804] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:46.616046 23182 raft_consensus.cc:697] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 [term 1 LEADER]: Becoming Leader. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:21:46.616816 23182 consensus_queue.cc:237] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:46.618880 22891 raft_consensus.cc:2468] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:46.620095 22785 leader_election.cc:304] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:46.620863 23301 raft_consensus.cc:2804] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:46.621330 23301 raft_consensus.cc:697] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:46.622160 23301 consensus_queue.cc:237] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:46.628532 23008 leader_election.cc:304] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:46.632962 23383 raft_consensus.cc:2804] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:46.633487 23383 raft_consensus.cc:697] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:46.634332 23383 consensus_queue.cc:237] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:46.566388 23397 leader_election.cc:290] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:46.650079 23086 tablet_bootstrap.cc:492] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:46.654035 23096 leader_election.cc:290] T 7802e4d0fd394596b42b661c9eb4c7ce P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:46.662948 23087 tablet_bootstrap.cc:492] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:46.663511 23087 ts_tablet_manager.cc:1403] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.230s	user 0.025s	sys 0.010s
I20260709 13:21:46.667142 23086 tablet_bootstrap.cc:654] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:46.669176 23087 raft_consensus.cc:359] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:46.671264 23087 raft_consensus.cc:385] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:46.671510 23087 raft_consensus.cc:740] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:46.674480 23087 consensus_queue.cc:260] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:46.670301 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7802e4d0fd394596b42b661c9eb4c7ce" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:46.680940 23096 raft_consensus.cc:493] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:46.681674 23087 ts_tablet_manager.cc:1434] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.002s
I20260709 13:21:46.693238 23096 raft_consensus.cc:515] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:46.694028 23087 tablet_bootstrap.cc:492] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:46.695384 23096 leader_election.cc:290] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:46.699277 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6a9c5a5fdc42e78dbbe6b411416c1a" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:46.700152 23087 tablet_bootstrap.cc:654] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:46.702991 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6a9c5a5fdc42e78dbbe6b411416c1a" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:46.706985 22860 leader_election.cc:304] T 7802e4d0fd394596b42b661c9eb4c7ce P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:46.707960 23096 raft_consensus.cc:2749] T 7802e4d0fd394596b42b661c9eb4c7ce P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:21:46.709137 22860 leader_election.cc:343] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:46.709589 22858 leader_election.cc:343] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:46.710151 22858 leader_election.cc:304] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:46.713665 23357 raft_consensus.cc:2749] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:46.723834 23085 tablet_bootstrap.cc:492] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:46.739639 23084 raft_consensus.cc:359] T 44841295bfd54159a48e285582824bad P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:46.750569 23357 consensus_queue.cc:1048] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:46.758810 23084 raft_consensus.cc:385] T 44841295bfd54159a48e285582824bad P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:46.759589 23084 raft_consensus.cc:740] T 44841295bfd54159a48e285582824bad P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:46.760833 22718 catalog_manager.cc:5697] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ef3887fb3874f8b81079e542d252944 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "8ef3887fb3874f8b81079e542d252944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:46.764236 23084 consensus_queue.cc:260] T 44841295bfd54159a48e285582824bad P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:46.768427 23084 ts_tablet_manager.cc:1434] T 44841295bfd54159a48e285582824bad P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.163s	user 0.004s	sys 0.003s
W20260709 13:21:46.779954 22766 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:46.785353 22717 catalog_manager.cc:5697] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 1 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:46.767808 23357 consensus_queue.cc:1048] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 13:21:46.827000 23296 raft_consensus.cc:493] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:46.834239 23296 raft_consensus.cc:515] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:46.836292 23296 leader_election.cc:290] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:46.832840 22714 catalog_manager.cc:5697] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:46.839905 23084 tablet_bootstrap.cc:492] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:46.840992 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e55b4e727be4089ba7454b662d03b9c" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
W20260709 13:21:46.842298 23008 leader_election.cc:343] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:46.842753 23085 tablet_bootstrap.cc:654] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:46.851192 23086 tablet_bootstrap.cc:492] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:46.851959 23086 ts_tablet_manager.cc:1403] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.206s	user 0.022s	sys 0.017s
I20260709 13:21:46.857970 23087 tablet_bootstrap.cc:492] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:46.858527 23087 ts_tablet_manager.cc:1403] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.165s	user 0.020s	sys 0.006s
I20260709 13:21:46.859653 23086 raft_consensus.cc:359] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:46.860412 23086 raft_consensus.cc:385] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:46.860677 23086 raft_consensus.cc:740] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:46.861210 23087 raft_consensus.cc:359] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:46.861949 23087 raft_consensus.cc:385] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:46.862219 23087 raft_consensus.cc:740] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:46.862872 23087 consensus_queue.cc:260] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:46.876971 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e55b4e727be4089ba7454b662d03b9c" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:46.880479 23354 raft_consensus.cc:493] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:46.881058 23354 raft_consensus.cc:515] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:46.883057 23354 leader_election.cc:290] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:46.884764 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74a5704470044c1bb9cd7c18e3b6000" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
W20260709 13:21:46.886442 22858 leader_election.cc:343] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:46.887405 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74a5704470044c1bb9cd7c18e3b6000" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
W20260709 13:21:46.889830 23008 leader_election.cc:343] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:46.902798 23008 leader_election.cc:304] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:46.903538 23187 raft_consensus.cc:2749] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:46.869740 23087 ts_tablet_manager.cc:1434] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:21:46.896355 23086 consensus_queue.cc:260] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:46.901345 23084 tablet_bootstrap.cc:654] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:46.906266 23087 tablet_bootstrap.cc:492] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
W20260709 13:21:46.904865 22860 leader_election.cc:343] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:46.906795 23085 tablet_bootstrap.cc:492] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:46.906744 22860 leader_election.cc:304] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:46.907307 23085 ts_tablet_manager.cc:1403] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.184s	user 0.016s	sys 0.007s
I20260709 13:21:46.907661 23357 raft_consensus.cc:2749] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:46.909413 23085 raft_consensus.cc:359] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:46.910173 23085 raft_consensus.cc:385] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:46.910449 23085 raft_consensus.cc:740] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:46.906837 23086 ts_tablet_manager.cc:1434] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.054s	user 0.005s	sys 0.001s
I20260709 13:21:46.912456 23086 tablet_bootstrap.cc:492] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:46.912710 23087 tablet_bootstrap.cc:654] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:46.915069 23085 consensus_queue.cc:260] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:46.926883 23085 ts_tablet_manager.cc:1434] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:21:46.928040 23085 tablet_bootstrap.cc:492] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:46.928534 23086 tablet_bootstrap.cc:654] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:46.935176 23085 tablet_bootstrap.cc:654] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:46.945418 23084 tablet_bootstrap.cc:492] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:46.946509 23084 ts_tablet_manager.cc:1403] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.107s	user 0.014s	sys 0.001s
I20260709 13:21:46.953308 23087 tablet_bootstrap.cc:492] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:46.954093 23086 tablet_bootstrap.cc:492] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:46.954084 23087 ts_tablet_manager.cc:1403] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.048s	user 0.017s	sys 0.018s
I20260709 13:21:46.954799 23086 ts_tablet_manager.cc:1403] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.043s	user 0.014s	sys 0.000s
I20260709 13:21:46.955919 23084 raft_consensus.cc:359] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:46.956655 23084 raft_consensus.cc:385] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:46.956938 23084 raft_consensus.cc:740] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:46.957597 23084 consensus_queue.cc:260] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:46.958314 23087 raft_consensus.cc:359] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:46.959007 23087 raft_consensus.cc:385] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:46.962810 23085 tablet_bootstrap.cc:492] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:46.962899 23087 raft_consensus.cc:740] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:46.963409 23085 ts_tablet_manager.cc:1403] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.036s	user 0.012s	sys 0.002s
I20260709 13:21:46.962037 23084 ts_tablet_manager.cc:1434] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:21:46.965121 23084 tablet_bootstrap.cc:492] T 8ad9b8807851437d9a22c0a33f567c15 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:46.966264 23085 raft_consensus.cc:359] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:46.966977 23085 raft_consensus.cc:385] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:46.967247 23085 raft_consensus.cc:740] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:46.958418 23086 raft_consensus.cc:359] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:46.968307 23086 raft_consensus.cc:385] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:46.968106 23085 consensus_queue.cc:260] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:46.968644 23086 raft_consensus.cc:740] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:46.969641 23086 consensus_queue.cc:260] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:46.971359 23084 tablet_bootstrap.cc:654] T 8ad9b8807851437d9a22c0a33f567c15 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:46.973801 23087 consensus_queue.cc:260] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:46.975975 23087 ts_tablet_manager.cc:1434] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 13:21:46.977066 23087 tablet_bootstrap.cc:492] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:46.977210 23086 ts_tablet_manager.cc:1434] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:21:46.978461 23086 tablet_bootstrap.cc:492] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:46.995031 23357 raft_consensus.cc:493] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:46.995675 23357 raft_consensus.cc:515] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:46.998037 23357 leader_election.cc:290] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:46.999585 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630a5e60917545b3a8072fc8358207c5" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
W20260709 13:21:47.001516 22858 leader_election.cc:343] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:46.999786 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630a5e60917545b3a8072fc8358207c5" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:47.002441 22818 raft_consensus.cc:2468] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 0.
I20260709 13:21:47.005224 22860 leader_election.cc:304] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68
I20260709 13:21:47.006019 23357 raft_consensus.cc:2804] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:47.006410 23357 raft_consensus.cc:493] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:47.006832 23357 raft_consensus.cc:3060] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.008019 23087 tablet_bootstrap.cc:654] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.006374 23085 ts_tablet_manager.cc:1434] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.043s	user 0.003s	sys 0.004s
I20260709 13:21:47.008011 23086 tablet_bootstrap.cc:654] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.010865 23085 tablet_bootstrap.cc:492] T 630a5e60917545b3a8072fc8358207c5 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:47.011107 23084 tablet_bootstrap.cc:492] T 8ad9b8807851437d9a22c0a33f567c15 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:47.011680 23084 ts_tablet_manager.cc:1403] T 8ad9b8807851437d9a22c0a33f567c15 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.047s	user 0.029s	sys 0.004s
I20260709 13:21:47.017001 23442 consensus_queue.cc:1048] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:47.022225 23085 tablet_bootstrap.cc:654] T 630a5e60917545b3a8072fc8358207c5 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.026034 22969 raft_consensus.cc:3060] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.026377 23084 raft_consensus.cc:359] T 8ad9b8807851437d9a22c0a33f567c15 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.027129 23084 raft_consensus.cc:385] T 8ad9b8807851437d9a22c0a33f567c15 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.027438 23084 raft_consensus.cc:740] T 8ad9b8807851437d9a22c0a33f567c15 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:47.030462 23084 consensus_queue.cc:260] T 8ad9b8807851437d9a22c0a33f567c15 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.034152 23087 tablet_bootstrap.cc:492] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:47.034703 23087 ts_tablet_manager.cc:1403] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.058s	user 0.008s	sys 0.011s
I20260709 13:21:47.037425 23087 raft_consensus.cc:359] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:47.038103 23087 raft_consensus.cc:385] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.038372 23087 raft_consensus.cc:740] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:47.044883 23087 consensus_queue.cc:260] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:47.047950 23087 ts_tablet_manager.cc:1434] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:21:47.023108 23397 raft_consensus.cc:493] T 44841295bfd54159a48e285582824bad P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.048995 23087 tablet_bootstrap.cc:492] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:47.050110 23084 ts_tablet_manager.cc:1434] T 8ad9b8807851437d9a22c0a33f567c15 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 13:21:47.051301 23397 raft_consensus.cc:515] T 44841295bfd54159a48e285582824bad P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.058372 23087 tablet_bootstrap.cc:654] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.053560 23084 tablet_bootstrap.cc:492] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:47.060416 23086 tablet_bootstrap.cc:492] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:47.060938 23086 ts_tablet_manager.cc:1403] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.083s	user 0.008s	sys 0.016s
I20260709 13:21:47.065165 23353 raft_consensus.cc:493] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.065718 23353 raft_consensus.cc:515] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:47.067193 23084 tablet_bootstrap.cc:654] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.068280 23277 raft_consensus.cc:493] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.068817 23277 raft_consensus.cc:515] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:47.071126 23277 leader_election.cc:290] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:47.069377 23353 leader_election.cc:290] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:47.071295 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abacd5d0d1e54db38a397e3c35c0cb81" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:47.073032 23041 raft_consensus.cc:2468] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
I20260709 13:21:47.073654 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "989cd1a042864c419d8c10665cb8afd0" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:47.074987 22783 leader_election.cc:304] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:47.076159 23301 raft_consensus.cc:2804] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:47.076555 23301 raft_consensus.cc:493] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:47.076901 23301 raft_consensus.cc:3060] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:47.077431 22860 leader_election.cc:343] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:47.078683 23187 raft_consensus.cc:493] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.079555 22963 raft_consensus.cc:3060] T 44841295bfd54159a48e285582824bad P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.084412 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "989cd1a042864c419d8c10665cb8afd0" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:47.085666 23301 raft_consensus.cc:515] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:47.086541 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abacd5d0d1e54db38a397e3c35c0cb81" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
W20260709 13:21:47.088016 22860 leader_election.cc:343] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:47.088294 23301 leader_election.cc:290] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:47.088575 23354 raft_consensus.cc:493] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.089046 23354 raft_consensus.cc:515] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.091245 23354 leader_election.cc:290] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
W20260709 13:21:47.093374 22785 leader_election.cc:343] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:47.099344 22860 leader_election.cc:304] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:47.093856 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abacd5d0d1e54db38a397e3c35c0cb81" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:47.101697 23041 raft_consensus.cc:3060] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.102332 23277 raft_consensus.cc:2749] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:47.101948 22965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad9b8807851437d9a22c0a33f567c15" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:47.104377 22965 raft_consensus.cc:2468] T 8ad9b8807851437d9a22c0a33f567c15 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 0.
I20260709 13:21:47.095572 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad9b8807851437d9a22c0a33f567c15" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:47.106334 22860 leader_election.cc:304] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:47.096998 23086 raft_consensus.cc:359] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:47.098073 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abacd5d0d1e54db38a397e3c35c0cb81" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:47.108953 23383 consensus_queue.cc:1048] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:47.111034 23354 raft_consensus.cc:2804] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:47.111471 23354 raft_consensus.cc:493] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:47.111824 23354 raft_consensus.cc:3060] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.109498 23086 raft_consensus.cc:385] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.113124 23086 raft_consensus.cc:740] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
W20260709 13:21:47.113658 22785 leader_election.cc:343] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:47.113936 23086 consensus_queue.cc:260] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:47.079262 23187 raft_consensus.cc:515] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.115769 23040 raft_consensus.cc:2468] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 0.
I20260709 13:21:47.117028 23086 ts_tablet_manager.cc:1434] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.056s	user 0.009s	sys 0.000s
I20260709 13:21:47.121474 23187 leader_election.cc:290] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:47.128445 23354 raft_consensus.cc:515] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.118382 23086 tablet_bootstrap.cc:492] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:47.129005 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44841295bfd54159a48e285582824bad" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:47.171007 23276 raft_consensus.cc:493] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.171655 23276 raft_consensus.cc:515] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:47.174119 23086 tablet_bootstrap.cc:654] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.129326 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44841295bfd54159a48e285582824bad" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:47.133057 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad9b8807851437d9a22c0a33f567c15" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:47.134142 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad9b8807851437d9a22c0a33f567c15" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:47.175962 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:47.135226 23087 tablet_bootstrap.cc:492] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:47.176900 23087 ts_tablet_manager.cc:1403] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.128s	user 0.011s	sys 0.001s
I20260709 13:21:47.176118 22966 raft_consensus.cc:2468] T 8ad9b8807851437d9a22c0a33f567c15 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 0.
I20260709 13:21:47.137379 23383 raft_consensus.cc:493] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.178226 23383 raft_consensus.cc:515] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:47.179950 22818 raft_consensus.cc:3060] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.180277 22932 leader_election.cc:304] T 44841295bfd54159a48e285582824bad P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:47.181002 23276 leader_election.cc:290] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:47.182778 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:47.183109 23383 leader_election.cc:290] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:47.155898 23397 leader_election.cc:290] T 44841295bfd54159a48e285582824bad P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:47.181416 22891 raft_consensus.cc:2393] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 in current term 1: Already voted for candidate 49cbf11e18ce4268a7fcca76872afb23 in this term.
I20260709 13:21:47.183549 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad9b8807851437d9a22c0a33f567c15" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:47.184159 22966 raft_consensus.cc:3060] T 8ad9b8807851437d9a22c0a33f567c15 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.190266 22934 leader_election.cc:304] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:47.191447 23397 raft_consensus.cc:2749] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:47.192931 23301 consensus_queue.cc:1048] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:47.192535 23087 raft_consensus.cc:359] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:47.196143 23087 raft_consensus.cc:385] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.196425 23087 raft_consensus.cc:740] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:47.197118 23087 consensus_queue.cc:260] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:47.198750 23182 raft_consensus.cc:2749] T 44841295bfd54159a48e285582824bad P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:47.205312 23354 leader_election.cc:290] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:47.206830 22966 raft_consensus.cc:2468] T 8ad9b8807851437d9a22c0a33f567c15 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:47.208102 22860 leader_election.cc:304] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:47.208710 23397 consensus_queue.cc:1048] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:47.208966 23354 raft_consensus.cc:2804] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:47.211288 23354 raft_consensus.cc:697] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:47.212148 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad9b8807851437d9a22c0a33f567c15" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:47.212806 23087 ts_tablet_manager.cc:1434] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
I20260709 13:21:47.212954 23043 raft_consensus.cc:3060] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.213991 23087 tablet_bootstrap.cc:492] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:47.051378 23357 raft_consensus.cc:515] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:47.230856 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630a5e60917545b3a8072fc8358207c5" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
W20260709 13:21:47.232470 22858 leader_election.cc:343] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:47.233451 23041 raft_consensus.cc:2468] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:47.238765 23357 leader_election.cc:290] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:47.239876 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630a5e60917545b3a8072fc8358207c5" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:47.241496 23085 tablet_bootstrap.cc:492] T 630a5e60917545b3a8072fc8358207c5 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:47.242452 22818 raft_consensus.cc:3060] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.246821 23086 tablet_bootstrap.cc:492] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:47.247609 23086 ts_tablet_manager.cc:1403] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.130s	user 0.015s	sys 0.001s
I20260709 13:21:47.244575 23085 ts_tablet_manager.cc:1403] T 630a5e60917545b3a8072fc8358207c5 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.234s	user 0.078s	sys 0.077s
I20260709 13:21:47.252794 23087 tablet_bootstrap.cc:654] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.253712 23084 tablet_bootstrap.cc:492] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:47.254314 23084 ts_tablet_manager.cc:1403] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.201s	user 0.035s	sys 0.033s
I20260709 13:21:47.257015 23086 raft_consensus.cc:359] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:47.257802 23086 raft_consensus.cc:385] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.258122 23086 raft_consensus.cc:740] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:47.259613 23085 raft_consensus.cc:359] T 630a5e60917545b3a8072fc8358207c5 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:47.257539 23397 consensus_queue.cc:1048] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:21:47.260327 23085 raft_consensus.cc:385] T 630a5e60917545b3a8072fc8358207c5 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.260603 23085 raft_consensus.cc:740] T 630a5e60917545b3a8072fc8358207c5 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:47.261104 22965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8b119f5c5844d1be439423b333ec60" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:47.261380 23085 consensus_queue.cc:260] T 630a5e60917545b3a8072fc8358207c5 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:47.263618 23085 ts_tablet_manager.cc:1434] T 630a5e60917545b3a8072fc8358207c5 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:21:47.263815 22815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8b119f5c5844d1be439423b333ec60" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:47.264601 23085 tablet_bootstrap.cc:492] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:47.258783 23086 consensus_queue.cc:260] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:47.265830 23084 raft_consensus.cc:359] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:47.266541 23084 raft_consensus.cc:385] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.266791 23084 raft_consensus.cc:740] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:47.267040 23086 ts_tablet_manager.cc:1434] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:21:47.212177 23354 consensus_queue.cc:237] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.267490 23084 consensus_queue.cc:260] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:47.268038 23086 tablet_bootstrap.cc:492] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:47.268985 22783 leader_election.cc:304] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:47.270416 23084 ts_tablet_manager.cc:1434] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:21:47.271477 23084 tablet_bootstrap.cc:492] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:47.272245 23301 raft_consensus.cc:2804] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:47.272358 23043 raft_consensus.cc:2468] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:47.281195 23301 raft_consensus.cc:697] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:47.279582 23008 leader_election.cc:304] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:47.284897 23086 tablet_bootstrap.cc:654] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.287014 23084 tablet_bootstrap.cc:654] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.286728 23301 consensus_queue.cc:237] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:47.287886 23085 tablet_bootstrap.cc:654] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.284189 23087 tablet_bootstrap.cc:492] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:47.289994 23296 raft_consensus.cc:2764] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:21:47.290644 23087 ts_tablet_manager.cc:1403] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.077s	user 0.014s	sys 0.001s
I20260709 13:21:47.293783 23296 consensus_queue.cc:1048] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:47.295308 23084 tablet_bootstrap.cc:492] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:47.295871 23084 ts_tablet_manager.cc:1403] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.001s
I20260709 13:21:47.293365 23087 raft_consensus.cc:359] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.296496 23087 raft_consensus.cc:385] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.296787 23087 raft_consensus.cc:740] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:47.298527 23087 consensus_queue.cc:260] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.298817 23084 raft_consensus.cc:359] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.299616 23084 raft_consensus.cc:385] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.299891 23084 raft_consensus.cc:740] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:47.300665 23087 ts_tablet_manager.cc:1434] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 13:21:47.300807 23084 consensus_queue.cc:260] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.302764 23086 tablet_bootstrap.cc:492] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:47.303354 23084 ts_tablet_manager.cc:1434] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:21:47.303293 23086 ts_tablet_manager.cc:1403] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.002s
I20260709 13:21:47.304461 23084 tablet_bootstrap.cc:492] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:47.306712 23086 raft_consensus.cc:359] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:47.307396 23086 raft_consensus.cc:385] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.307658 23086 raft_consensus.cc:740] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:47.310626 23085 tablet_bootstrap.cc:492] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:47.310827 23084 tablet_bootstrap.cc:654] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.311192 23085 ts_tablet_manager.cc:1403] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.047s	user 0.011s	sys 0.005s
I20260709 13:21:47.313091 23086 consensus_queue.cc:260] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:47.315363 23086 ts_tablet_manager.cc:1434] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:21:47.316289 23086 tablet_bootstrap.cc:492] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:47.322664 23086 tablet_bootstrap.cc:654] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.321935 22717 catalog_manager.cc:5697] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:47.330370 23301 raft_consensus.cc:493] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.343655 23301 raft_consensus.cc:515] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:47.331601 23085 raft_consensus.cc:359] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.346561 22965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e55b4e727be4089ba7454b662d03b9c" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:47.345842 23301 leader_election.cc:290] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:47.347978 23085 raft_consensus.cc:385] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.348438 23085 raft_consensus.cc:740] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
W20260709 13:21:47.324024 23008 leader_election.cc:343] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:47.349417 23085 consensus_queue.cc:260] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.351606 23085 ts_tablet_manager.cc:1434] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.000s
I20260709 13:21:47.352766 23085 tablet_bootstrap.cc:492] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
W20260709 13:21:47.348477 22785 leader_election.cc:343] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:47.354595 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e55b4e727be4089ba7454b662d03b9c" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:47.355690 23041 raft_consensus.cc:2468] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
I20260709 13:21:47.356629 23296 raft_consensus.cc:493] T 630a5e60917545b3a8072fc8358207c5 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.357458 23296 raft_consensus.cc:515] T 630a5e60917545b3a8072fc8358207c5 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:47.359592 23296 leader_election.cc:290] T 630a5e60917545b3a8072fc8358207c5 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:47.361985 23085 tablet_bootstrap.cc:654] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.368675 23087 tablet_bootstrap.cc:492] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:47.393324 22783 leader_election.cc:304] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:47.394739 23301 raft_consensus.cc:2804] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:47.395262 23301 raft_consensus.cc:493] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:47.395695 23301 raft_consensus.cc:3060] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.394629 23354 raft_consensus.cc:493] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.398801 23354 raft_consensus.cc:515] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:47.401831 22815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac141e204e3f4d3c84d6651893bdfc6f" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:47.402534 22815 raft_consensus.cc:2468] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 0.
I20260709 13:21:47.403769 22860 leader_election.cc:304] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:47.405347 23357 raft_consensus.cc:2804] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:47.405150 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630a5e60917545b3a8072fc8358207c5" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:47.406196 22893 raft_consensus.cc:2393] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 in current term 1: Already voted for candidate 49cbf11e18ce4268a7fcca76872afb23 in this term.
I20260709 13:21:47.405160 22815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630a5e60917545b3a8072fc8358207c5" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:47.407450 22965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac141e204e3f4d3c84d6651893bdfc6f" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:47.407995 23354 leader_election.cc:290] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
W20260709 13:21:47.409104 22860 leader_election.cc:343] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:47.405745 23357 raft_consensus.cc:493] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:47.426891 23357 raft_consensus.cc:3060] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.433620 23301 raft_consensus.cc:515] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:47.435295 22818 raft_consensus.cc:2468] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
W20260709 13:21:47.437649 23008 leader_election.cc:343] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:47.440212 23008 leader_election.cc:304] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:47.440932 22860 leader_election.cc:304] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68
I20260709 13:21:47.441268 23383 raft_consensus.cc:2749] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:47.441781 23096 raft_consensus.cc:2804] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:47.442292 23096 raft_consensus.cc:697] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:47.443473 23087 tablet_bootstrap.cc:654] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.443168 23096 consensus_queue.cc:237] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:47.444641 22715 catalog_manager.cc:5697] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:47.448908 22965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e55b4e727be4089ba7454b662d03b9c" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:47.451311 23086 tablet_bootstrap.cc:492] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
W20260709 13:21:47.456575 22785 leader_election.cc:343] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:47.452131 23301 leader_election.cc:290] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:47.455137 23085 tablet_bootstrap.cc:492] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:47.447329 23357 raft_consensus.cc:515] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:47.456657 23086 ts_tablet_manager.cc:1403] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.141s	user 0.059s	sys 0.049s
I20260709 13:21:47.458112 23085 ts_tablet_manager.cc:1403] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.106s	user 0.005s	sys 0.007s
I20260709 13:21:47.461102 23085 raft_consensus.cc:359] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.461179 22965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac141e204e3f4d3c84d6651893bdfc6f" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:47.462828 23085 raft_consensus.cc:385] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.463415 23182 raft_consensus.cc:493] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.462996 23086 raft_consensus.cc:359] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:47.463663 23085 raft_consensus.cc:740] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:47.454878 23084 tablet_bootstrap.cc:492] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:47.464447 23084 ts_tablet_manager.cc:1403] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.160s	user 0.036s	sys 0.041s
I20260709 13:21:47.463961 23182 raft_consensus.cc:515] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:47.463953 23086 raft_consensus.cc:385] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.464956 23086 raft_consensus.cc:740] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:47.465754 23086 consensus_queue.cc:260] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:47.467098 23084 raft_consensus.cc:359] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:47.467840 23182 leader_election.cc:290] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:47.468284 23085 consensus_queue.cc:260] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.468380 23086 ts_tablet_manager.cc:1434] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:21:47.469723 22815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b752c38e1b47a489a3d720801661d1" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:47.470260 23086 tablet_bootstrap.cc:492] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:47.453182 23008 leader_election.cc:304] T 630a5e60917545b3a8072fc8358207c5 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:47.467815 23084 raft_consensus.cc:385] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.470839 23084 raft_consensus.cc:740] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:47.471487 23084 consensus_queue.cc:260] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:47.467135 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b752c38e1b47a489a3d720801661d1" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:47.473706 23084 ts_tablet_manager.cc:1434] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.000s
I20260709 13:21:47.474650 23084 tablet_bootstrap.cc:492] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:47.476449 23296 raft_consensus.cc:3060] T 630a5e60917545b3a8072fc8358207c5 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.477700 22932 leader_election.cc:304] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:47.478998 23182 raft_consensus.cc:2749] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:47.477670 23086 tablet_bootstrap.cc:654] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.480310 23084 tablet_bootstrap.cc:654] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.470368 23085 ts_tablet_manager.cc:1434] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:21:47.482942 23085 tablet_bootstrap.cc:492] T b74a5704470044c1bb9cd7c18e3b6000 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:47.488783 23085 tablet_bootstrap.cc:654] T b74a5704470044c1bb9cd7c18e3b6000 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.490979 23086 tablet_bootstrap.cc:492] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:47.491506 23086 ts_tablet_manager.cc:1403] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.003s
I20260709 13:21:47.494231 23086 raft_consensus.cc:359] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:47.494956 23086 raft_consensus.cc:385] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.495217 23086 raft_consensus.cc:740] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:47.497720 23296 raft_consensus.cc:2749] T 630a5e60917545b3a8072fc8358207c5 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:47.497905 23086 consensus_queue.cc:260] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:47.498879 23084 tablet_bootstrap.cc:492] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:47.499343 23084 ts_tablet_manager.cc:1403] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.003s
I20260709 13:21:47.499598 23086 ts_tablet_manager.cc:1434] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:21:47.500620 23086 tablet_bootstrap.cc:492] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:47.504876 23085 tablet_bootstrap.cc:492] T b74a5704470044c1bb9cd7c18e3b6000 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:47.505364 23182 raft_consensus.cc:493] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.505508 23085 ts_tablet_manager.cc:1403] T b74a5704470044c1bb9cd7c18e3b6000 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.004s
I20260709 13:21:47.505952 23182 raft_consensus.cc:515] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.506949 23086 tablet_bootstrap.cc:654] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.508713 23182 leader_election.cc:290] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:47.508495 23085 raft_consensus.cc:359] T b74a5704470044c1bb9cd7c18e3b6000 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.509258 23085 raft_consensus.cc:385] T b74a5704470044c1bb9cd7c18e3b6000 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.509543 23085 raft_consensus.cc:740] T b74a5704470044c1bb9cd7c18e3b6000 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:47.501897 23084 raft_consensus.cc:359] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.510543 23084 raft_consensus.cc:385] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.511345 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74a5704470044c1bb9cd7c18e3b6000" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:47.511539 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74a5704470044c1bb9cd7c18e3b6000" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:47.511999 22891 raft_consensus.cc:2468] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 0.
W20260709 13:21:47.512140 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 59050.5us
I20260709 13:21:47.513264 22934 leader_election.cc:304] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:47.513154 23085 consensus_queue.cc:260] T b74a5704470044c1bb9cd7c18e3b6000 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.514087 23041 raft_consensus.cc:2468] T b74a5704470044c1bb9cd7c18e3b6000 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 0.
I20260709 13:21:47.510784 23084 raft_consensus.cc:740] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:47.516225 23084 consensus_queue.cc:260] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.518476 23085 ts_tablet_manager.cc:1434] T b74a5704470044c1bb9cd7c18e3b6000 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.002s
W20260709 13:21:47.462814 22860 leader_election.cc:343] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:47.519479 23086 tablet_bootstrap.cc:492] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:47.519932 23397 raft_consensus.cc:2804] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:47.520220 23086 ts_tablet_manager.cc:1403] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.020s	user 0.008s	sys 0.009s
I20260709 13:21:47.520396 23397 raft_consensus.cc:493] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:47.520852 23397 raft_consensus.cc:3060] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.521888 23084 ts_tablet_manager.cc:1434] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:21:47.522951 23084 tablet_bootstrap.cc:492] T ac141e204e3f4d3c84d6651893bdfc6f P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:47.523588 23086 raft_consensus.cc:359] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:47.524379 23086 raft_consensus.cc:385] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.524739 23086 raft_consensus.cc:740] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:47.525586 23086 consensus_queue.cc:260] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:47.527490 23086 ts_tablet_manager.cc:1434] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:21:47.529340 23084 tablet_bootstrap.cc:654] T ac141e204e3f4d3c84d6651893bdfc6f P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.519554 23085 tablet_bootstrap.cc:492] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:47.533269 23096 raft_consensus.cc:493] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.533874 23096 raft_consensus.cc:515] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:47.536165 23096 leader_election.cc:290] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
W20260709 13:21:47.520045 22860 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 57273.1us
I20260709 13:21:47.537278 23397 raft_consensus.cc:515] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.536381 23085 tablet_bootstrap.cc:654] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.539891 23397 leader_election.cc:290] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:47.540469 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74a5704470044c1bb9cd7c18e3b6000" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:47.541190 22893 raft_consensus.cc:3060] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.542261 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:47.542955 22816 raft_consensus.cc:2468] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:47.520381 23357 leader_election.cc:290] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:47.462155 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac141e204e3f4d3c84d6651893bdfc6f" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:47.544485 22818 raft_consensus.cc:3060] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.540504 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74a5704470044c1bb9cd7c18e3b6000" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:47.546689 23041 raft_consensus.cc:3060] T b74a5704470044c1bb9cd7c18e3b6000 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.544215 22860 leader_election.cc:304] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:47.557709 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e55b4e727be4089ba7454b662d03b9c" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:47.558575 23040 raft_consensus.cc:3060] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.557703 22965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:47.562448 22965 raft_consensus.cc:2468] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:47.566921 22818 raft_consensus.cc:2468] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:47.560714 23096 raft_consensus.cc:2804] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:47.568012 23096 raft_consensus.cc:493] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:47.568364 23096 raft_consensus.cc:3060] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:47.571086 23040 raft_consensus.cc:2468] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:47.571753 23086 tablet_bootstrap.cc:492] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:47.576509 22893 raft_consensus.cc:2468] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:47.591579 22934 leader_election.cc:304] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:47.593235 22783 leader_election.cc:304] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:47.594633 23301 raft_consensus.cc:2804] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:47.595173 23301 raft_consensus.cc:697] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:47.595921 23041 raft_consensus.cc:2468] T b74a5704470044c1bb9cd7c18e3b6000 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:47.596060 23301 consensus_queue.cc:237] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:47.597136 23397 raft_consensus.cc:2804] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:47.610677 23397 raft_consensus.cc:697] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944 [term 1 LEADER]: Becoming Leader. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:21:47.611642 23397 consensus_queue.cc:237] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.597553 23085 tablet_bootstrap.cc:492] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:47.601969 22966 raft_consensus.cc:3060] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.614697 23096 raft_consensus.cc:515] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:47.620532 23096 leader_election.cc:290] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:47.609643 23084 tablet_bootstrap.cc:492] T ac141e204e3f4d3c84d6651893bdfc6f P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:47.602453 23087 tablet_bootstrap.cc:492] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:47.597709 23353 raft_consensus.cc:493] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.621848 23085 ts_tablet_manager.cc:1403] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.103s	user 0.009s	sys 0.002s
I20260709 13:21:47.621820 23353 raft_consensus.cc:515] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:47.624118 23353 leader_election.cc:290] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:47.621487 23087 ts_tablet_manager.cc:1403] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.256s	user 0.010s	sys 0.002s
I20260709 13:21:47.625384 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014651e80f9f44e599d2d8e4d308d2a6" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:47.627442 23086 tablet_bootstrap.cc:654] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.627765 23087 raft_consensus.cc:359] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:47.628433 23087 raft_consensus.cc:385] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.628726 23087 raft_consensus.cc:740] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:47.630661 23296 raft_consensus.cc:493] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.632074 23087 consensus_queue.cc:260] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:47.631295 23296 raft_consensus.cc:515] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.634713 23296 leader_election.cc:290] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:47.598557 22860 leader_election.cc:304] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:47.637530 23084 ts_tablet_manager.cc:1403] T ac141e204e3f4d3c84d6651893bdfc6f P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.115s	user 0.019s	sys 0.007s
I20260709 13:21:47.638453 23354 raft_consensus.cc:2804] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:47.639055 23354 raft_consensus.cc:697] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:47.639953 23354 consensus_queue.cc:237] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:47.646061 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:47.646844 22816 raft_consensus.cc:3060] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:47.646871 22965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:47.647583 22965 raft_consensus.cc:3060] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:47.649140 23357 raft_consensus.cc:493] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.649745 23357 raft_consensus.cc:515] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.652107 23357 leader_election.cc:290] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:47.655933 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d217f95123f542e886aa049a608a88d4" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:47.656704 23040 raft_consensus.cc:2468] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 0.
I20260709 13:21:47.660501 23353 raft_consensus.cc:493] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.661101 23353 raft_consensus.cc:515] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:47.663581 23353 leader_election.cc:290] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:47.664791 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "989cd1a042864c419d8c10665cb8afd0" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:47.666263 23087 ts_tablet_manager.cc:1434] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.000s
W20260709 13:21:47.666519 22785 leader_election.cc:343] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:47.667814 23087 tablet_bootstrap.cc:492] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:47.655941 22816 raft_consensus.cc:2468] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2.
I20260709 13:21:47.672072 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "989cd1a042864c419d8c10665cb8afd0" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:47.672786 22858 leader_election.cc:304] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:47.673051 22891 raft_consensus.cc:2468] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
I20260709 13:21:47.673105 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d217f95123f542e886aa049a608a88d4" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:47.673964 23096 raft_consensus.cc:2804] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:47.674713 22895 raft_consensus.cc:2468] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 0.
I20260709 13:21:47.675654 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d217f95123f542e886aa049a608a88d4" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:47.678261 23096 raft_consensus.cc:493] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:47.678651 22893 raft_consensus.cc:2468] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
I20260709 13:21:47.679685 22860 leader_election.cc:304] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:47.679968 22785 leader_election.cc:304] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:47.680650 23357 raft_consensus.cc:2804] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:47.682029 22785 leader_election.cc:304] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:47.682865 23362 raft_consensus.cc:2804] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:47.683249 23362 raft_consensus.cc:493] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:47.683590 23362 raft_consensus.cc:3060] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.688436 22714 catalog_manager.cc:5697] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:47.690488 23442 raft_consensus.cc:2804] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:47.690927 23442 raft_consensus.cc:493] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:47.691269 23442 raft_consensus.cc:3060] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.693791 23362 raft_consensus.cc:515] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:47.678754 23096 raft_consensus.cc:3060] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.714322 23362 leader_election.cc:290] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:47.715560 23442 raft_consensus.cc:515] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:47.722433 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "989cd1a042864c419d8c10665cb8afd0" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
W20260709 13:21:47.724637 22785 leader_election.cc:343] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:47.725159 23087 tablet_bootstrap.cc:654] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.625476 23085 raft_consensus.cc:359] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:47.726022 23085 raft_consensus.cc:385] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.726361 23085 raft_consensus.cc:740] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:47.727092 23085 consensus_queue.cc:260] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:47.701777 22715 catalog_manager.cc:5697] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:47.729362 23085 ts_tablet_manager.cc:1434] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.107s	user 0.004s	sys 0.000s
I20260709 13:21:47.742928 23084 raft_consensus.cc:359] T ac141e204e3f4d3c84d6651893bdfc6f P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:47.743810 23084 raft_consensus.cc:385] T ac141e204e3f4d3c84d6651893bdfc6f P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.744089 23084 raft_consensus.cc:740] T ac141e204e3f4d3c84d6651893bdfc6f P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:47.748437 23096 raft_consensus.cc:515] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.752454 23096 leader_election.cc:290] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:47.755306 23087 tablet_bootstrap.cc:492] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:47.755944 23087 ts_tablet_manager.cc:1403] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.088s	user 0.011s	sys 0.000s
I20260709 13:21:47.758536 23442 leader_election.cc:290] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
W20260709 13:21:47.761355 22860 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 80622.8us
I20260709 13:21:47.769840 23087 raft_consensus.cc:359] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:47.773869 23087 raft_consensus.cc:385] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.774241 23087 raft_consensus.cc:740] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:47.774922 23087 consensus_queue.cc:260] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
W20260709 13:21:47.784030 22860 leader_election.cc:343] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:47.787379 23087 ts_tablet_manager.cc:1434] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.031s	user 0.002s	sys 0.005s
I20260709 13:21:47.788036 23084 consensus_queue.cc:260] T ac141e204e3f4d3c84d6651893bdfc6f P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:47.797180 23084 ts_tablet_manager.cc:1434] T ac141e204e3f4d3c84d6651893bdfc6f P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.159s	user 0.007s	sys 0.000s
I20260709 13:21:47.789649 23039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014651e80f9f44e599d2d8e4d308d2a6" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:47.806767 22714 catalog_manager.cc:5697] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ef3887fb3874f8b81079e542d252944 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "8ef3887fb3874f8b81079e542d252944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:47.807917 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d217f95123f542e886aa049a608a88d4" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:47.798820 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d217f95123f542e886aa049a608a88d4" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:47.791383 23357 raft_consensus.cc:697] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:47.822387 23085 tablet_bootstrap.cc:492] T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:47.825376 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "989cd1a042864c419d8c10665cb8afd0" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:47.826090 22891 raft_consensus.cc:3060] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.826764 23383 raft_consensus.cc:493] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.827289 23383 raft_consensus.cc:515] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.828929 22965 raft_consensus.cc:2468] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2.
I20260709 13:21:47.829335 23383 leader_election.cc:290] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
W20260709 13:21:47.832533 22860 leader_election.cc:343] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:47.833315 23008 leader_election.cc:304] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:47.856487 23084 tablet_bootstrap.cc:492] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:47.831175 23039 raft_consensus.cc:3060] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.858048 23187 raft_consensus.cc:2804] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:47.861666 23187 raft_consensus.cc:493] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:47.862109 23187 raft_consensus.cc:3060] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.863473 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014651e80f9f44e599d2d8e4d308d2a6" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:47.864087 22893 raft_consensus.cc:3060] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.887398 23087 tablet_bootstrap.cc:492] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23: Bootstrap starting.
I20260709 13:21:47.882866 23301 raft_consensus.cc:493] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.882668 23357 consensus_queue.cc:237] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:47.893687 23086 tablet_bootstrap.cc:492] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:47.895682 22893 raft_consensus.cc:2468] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:47.896694 23086 ts_tablet_manager.cc:1403] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.331s	user 0.071s	sys 0.090s
I20260709 13:21:47.899592 23086 raft_consensus.cc:359] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:47.900306 23086 raft_consensus.cc:385] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.900570 23086 raft_consensus.cc:740] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:47.901654 23086 consensus_queue.cc:260] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:47.904135 23086 ts_tablet_manager.cc:1434] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:21:47.905138 23085 tablet_bootstrap.cc:654] T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.905670 23086 tablet_bootstrap.cc:492] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:47.910543 23301 raft_consensus.cc:515] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:47.912868 23301 leader_election.cc:290] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:47.913255 23086 tablet_bootstrap.cc:654] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.913244 23354 consensus_queue.cc:1048] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:47.918249 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d217f95123f542e886aa049a608a88d4" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:47.919164 22891 raft_consensus.cc:2468] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:47.920925 23187 raft_consensus.cc:515] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.933866 23086 tablet_bootstrap.cc:492] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679: No bootstrap required, opened a new log
I20260709 13:21:47.934437 23086 ts_tablet_manager.cc:1403] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.014s
I20260709 13:21:47.934513 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014651e80f9f44e599d2d8e4d308d2a6" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:47.938374 23084 tablet_bootstrap.cc:654] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:47.932559 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 77177.5us
I20260709 13:21:47.937426 22785 leader_election.cc:304] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:47.943205 22785 leader_election.cc:304] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
W20260709 13:21:47.955144 22766 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:47.945567 23042 raft_consensus.cc:2393] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 in current term 1: Already voted for candidate 27745ef236cd494684f62a1269411a68 in this term.
I20260709 13:21:47.955942 22963 raft_consensus.cc:3060] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.942847 23353 raft_consensus.cc:493] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.963101 22964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8b119f5c5844d1be439423b333ec60" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:47.963729 23353 raft_consensus.cc:515] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:47.954972 23276 raft_consensus.cc:493] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.946084 23301 raft_consensus.cc:2804] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:47.966948 23087 tablet_bootstrap.cc:654] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:47.974015 22858 leader_election.cc:304] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:47.967332 23086 raft_consensus.cc:359] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:47.991093 23086 raft_consensus.cc:385] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:47.991457 23086 raft_consensus.cc:740] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:47.993860 23086 consensus_queue.cc:260] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:47.992867 23096 raft_consensus.cc:2749] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:21:47.999717 23008 leader_election.cc:343] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:47.969198 23301 raft_consensus.cc:697] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:47.954960 23442 raft_consensus.cc:2804] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:47.965895 23353 leader_election.cc:290] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:47.970227 23039 raft_consensus.cc:2468] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:47.968580 23276 raft_consensus.cc:515] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:47.973263 23277 consensus_queue.cc:1048] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.031s
W20260709 13:21:48.011267 22934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 73372us
I20260709 13:21:48.015002 23096 consensus_queue.cc:1048] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:48.014662 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8b119f5c5844d1be439423b333ec60" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:48.027486 23041 raft_consensus.cc:2468] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 0.
I20260709 13:21:48.014050 23187 leader_election.cc:290] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:48.033340 23086 ts_tablet_manager.cc:1434] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.099s	user 0.007s	sys 0.000s
I20260709 13:21:48.019353 23499 consensus_queue.cc:1048] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:21:48.042851 23442 raft_consensus.cc:697] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:48.045428 23442 consensus_queue.cc:237] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:48.050709 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0abfe80c1fa642ec8a0d075f75264816" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:48.051354 23353 consensus_queue.cc:1048] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:21:48.051743 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 44451.2us
I20260709 13:21:48.038507 23301 consensus_queue.cc:237] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:48.053304 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6a9c5a5fdc42e78dbbe6b411416c1a" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:48.054021 22894 raft_consensus.cc:2468] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 0.
I20260709 13:21:48.057212 22964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6a9c5a5fdc42e78dbbe6b411416c1a" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:48.057613 23085 tablet_bootstrap.cc:492] T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:48.034240 23507 raft_consensus.cc:493] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:48.059957 23085 ts_tablet_manager.cc:1403] T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 0.264s	user 0.017s	sys 0.016s
I20260709 13:21:48.045310 23276 leader_election.cc:290] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:48.078969 22934 leader_election.cc:304] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 
W20260709 13:21:48.142158 22934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 63297us
W20260709 13:21:48.142112 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 77910.2us
I20260709 13:21:48.181504 23096 raft_consensus.cc:493] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:48.182212 23096 raft_consensus.cc:515] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:48.188078 23084 tablet_bootstrap.cc:492] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:48.188787 23084 ts_tablet_manager.cc:1403] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.341s	user 0.012s	sys 0.002s
I20260709 13:21:48.192524 23096 leader_election.cc:290] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:48.157119 23504 raft_consensus.cc:493] T ac141e204e3f4d3c84d6651893bdfc6f P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:48.141062 22964 raft_consensus.cc:2468] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 0.
I20260709 13:21:48.215473 23085 raft_consensus.cc:359] T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:48.216254 23085 raft_consensus.cc:385] T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:48.216528 23085 raft_consensus.cc:740] T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:48.218835 22961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d217f95123f542e886aa049a608a88d4" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:48.219445 22783 leader_election.cc:304] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:48.221890 23497 raft_consensus.cc:2804] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:48.222306 23497 raft_consensus.cc:493] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:48.222397 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1394978547a410199b4f798a10edb2d" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:48.222712 23497 raft_consensus.cc:3060] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:48.223973 23510 log.cc:927] Time spent T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23: Append to log took a long time: real 0.183s	user 0.000s	sys 0.000s
I20260709 13:21:48.230590 23182 raft_consensus.cc:2804] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:48.141184 23507 raft_consensus.cc:515] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:48.238667 23507 leader_election.cc:290] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:48.242704 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1394978547a410199b4f798a10edb2d" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:48.243590 22816 raft_consensus.cc:2468] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 0.
I20260709 13:21:48.246778 23353 raft_consensus.cc:493] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:48.247298 23353 raft_consensus.cc:515] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:48.249348 23353 leader_election.cc:290] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:48.211738 23504 raft_consensus.cc:515] T ac141e204e3f4d3c84d6651893bdfc6f P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:48.217157 23085 consensus_queue.cc:260] T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:48.249221 22860 leader_election.cc:304] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
W20260709 13:21:48.262157 22860 leader_election.cc:343] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:48.273435 23085 ts_tablet_manager.cc:1434] T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.210s	user 0.006s	sys 0.000s
I20260709 13:21:48.271219 23354 raft_consensus.cc:2804] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:48.270768 23276 consensus_queue.cc:1048] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
W20260709 13:21:48.290123 23008 leader_election.cc:343] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:48.285938 23354 raft_consensus.cc:493] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:48.297739 23035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0abfe80c1fa642ec8a0d075f75264816" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:48.301779 22964 raft_consensus.cc:3060] T ac141e204e3f4d3c84d6651893bdfc6f P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:48.302525 23504 leader_election.cc:290] T ac141e204e3f4d3c84d6651893bdfc6f P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:48.304387 23182 raft_consensus.cc:493] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:48.304889 23182 raft_consensus.cc:3060] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:48.304454 23087 tablet_bootstrap.cc:492] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23: No bootstrap required, opened a new log
I20260709 13:21:48.306136 23087 ts_tablet_manager.cc:1403] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23: Time spent bootstrapping tablet: real 0.421s	user 0.032s	sys 0.028s
I20260709 13:21:48.306416 23084 raft_consensus.cc:359] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:48.307142 23084 raft_consensus.cc:385] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:48.307394 23084 raft_consensus.cc:740] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:48.307796 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6a9c5a5fdc42e78dbbe6b411416c1a" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:48.308293 22893 raft_consensus.cc:2468] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 0.
W20260709 13:21:48.290663 22783 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 71286.2us
I20260709 13:21:48.294842 23354 raft_consensus.cc:3060] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:48.314069 23084 consensus_queue.cc:260] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:48.319532 23500 consensus_queue.cc:1048] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:48.320298 23085 tablet_bootstrap.cc:492] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68: Bootstrap starting.
I20260709 13:21:48.318936 23497 raft_consensus.cc:515] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:48.321885 22889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac141e204e3f4d3c84d6651893bdfc6f" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:48.318884 22715 catalog_manager.cc:5697] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:48.324875 23087 raft_consensus.cc:359] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:48.328261 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6a9c5a5fdc42e78dbbe6b411416c1a" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:48.341945 23505 consensus_queue.cc:1048] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:48.342325 23041 raft_consensus.cc:2468] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 0.
I20260709 13:21:48.336820 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d217f95123f542e886aa049a608a88d4" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:48.343777 22893 raft_consensus.cc:2393] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 in current term 1: Already voted for candidate 49cbf11e18ce4268a7fcca76872afb23 in this term.
I20260709 13:21:48.344903 23087 raft_consensus.cc:385] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:48.345435 23087 raft_consensus.cc:740] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Initialized, Role: FOLLOWER
I20260709 13:21:48.346208 23087 consensus_queue.cc:260] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:48.347612 23084 ts_tablet_manager.cc:1434] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.159s	user 0.008s	sys 0.000s
I20260709 13:21:48.350435 22969 raft_consensus.cc:1275] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:21:48.341614 22715 catalog_manager.cc:5697] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 0 to 2, leader changed from <none> to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:48.354012 23182 raft_consensus.cc:515] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:48.356974 23096 raft_consensus.cc:493] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:48.357496 23096 raft_consensus.cc:515] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:48.358374 23084 tablet_bootstrap.cc:492] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:48.357077 23036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abacd5d0d1e54db38a397e3c35c0cb81" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:48.359404 23096 leader_election.cc:290] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:48.360615 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6a9c5a5fdc42e78dbbe6b411416c1a" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:48.361315 22893 raft_consensus.cc:3060] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:48.363062 23504 raft_consensus.cc:493] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:48.363600 23504 raft_consensus.cc:515] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:48.364564 23354 raft_consensus.cc:515] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:48.363380 23085 tablet_bootstrap.cc:654] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:48.368032 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52d63f523d24195bc8127113598ff91" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:48.368722 23041 raft_consensus.cc:2468] T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 0.
I20260709 13:21:48.369437 23096 consensus_queue.cc:1048] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:48.371363 23008 leader_election.cc:304] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:48.378832 22893 raft_consensus.cc:2468] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:48.379865 22934 leader_election.cc:304] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:48.381197 23084 tablet_bootstrap.cc:654] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:48.383571 23087 ts_tablet_manager.cc:1434] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23: Time spent starting tablet: real 0.077s	user 0.007s	sys 0.000s
I20260709 13:21:48.368340 23497 leader_election.cc:290] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:48.365810 23504 leader_election.cc:290] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:48.390656 22815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52d63f523d24195bc8127113598ff91" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:48.391230 23506 raft_consensus.cc:2804] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:48.391466 22815 raft_consensus.cc:2468] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 0.
I20260709 13:21:48.397138 22858 leader_election.cc:304] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:48.398528 23096 raft_consensus.cc:2804] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:48.398923 23096 raft_consensus.cc:493] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:48.399286 23096 raft_consensus.cc:3060] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:48.385460 23397 raft_consensus.cc:2804] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:48.397857 23506 raft_consensus.cc:493] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:48.411557 23506 raft_consensus.cc:3060] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:48.410220 23397 raft_consensus.cc:697] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [term 1 LEADER]: Becoming Leader. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:21:48.425215 23036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8b119f5c5844d1be439423b333ec60" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:48.440160 23036 raft_consensus.cc:3060] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:48.443395 23008 leader_election.cc:304] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
W20260709 13:21:48.422382 22783 leader_election.cc:343] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:48.715201 23533 log.cc:927] Time spent T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
W20260709 13:21:50.805001 22783 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 2.38253e+06us
I20260709 13:21:49.526329 23539 raft_consensus.cc:493] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:48.459677 23036 raft_consensus.cc:2468] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:48.458618 23037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6a9c5a5fdc42e78dbbe6b411416c1a" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:48.446276 23038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e55b4e727be4089ba7454b662d03b9c" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.366717 23609 raft_consensus.cc:493] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:51.366950 23038 raft_consensus.cc:2393] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 in current term 1: Already voted for candidate 91f33bdbdd63438294071eb3776db679 in this term.
W20260709 13:21:51.367643 23038 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36850 (request call id 86) took 2955 ms (client timeout 1590 ms). Trace:
W20260709 13:21:49.214107 23084 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.010s	sys 0.003s
W20260709 13:21:51.370464 23084 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 2.921s	user 0.014s	sys 0.003s
W20260709 13:21:50.148211 23085 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 1.780s	user 0.090s	sys 0.125s
W20260709 13:21:51.371510 23085 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 3.003s	user 0.090s	sys 0.125s
I20260709 13:21:48.721236 23096 raft_consensus.cc:515] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:48.413048 23182 leader_election.cc:290] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:48.450098 23354 leader_election.cc:290] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:48.423084 23397 consensus_queue.cc:237] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:48.963599 23504 raft_consensus.cc:493] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:48.469326 23506 raft_consensus.cc:515] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:51.413777 23037 raft_consensus.cc:2393] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 in current term 1: Already voted for candidate 27745ef236cd494684f62a1269411a68 in this term.
I20260709 13:21:51.414640 23506 leader_election.cc:290] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:49.210561 23515 raft_consensus.cc:493] T 78234c90229d4615ab9e2c2a811bd0f8 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:48.446916 23525 raft_consensus.cc:2749] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:21:48.499468 23530 log.cc:927] Time spent T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:21:50.746691 23543 raft_consensus.cc:493] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:51.424672 23543 raft_consensus.cc:515] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:50.754142 23541 raft_consensus.cc:493] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.425812 23541 raft_consensus.cc:515] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:51.426285 22894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43418 (request call id 60) took 3126 ms. Trace:
I20260709 13:21:51.427083 23543 leader_election.cc:290] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:50.755677 23544 raft_consensus.cc:493] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:50.755653 23545 raft_consensus.cc:493] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
W20260709 13:21:50.773505 22715 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.255s	user 0.004s	sys 0.006s
I20260709 13:21:50.775144 23546 raft_consensus.cc:493] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:51.427839 23544 raft_consensus.cc:515] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:50.777433 23540 raft_consensus.cc:493] T be220d19020b428a8b1cc356f32ff5ad P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:50.798509 22818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49352 (request call id 93) took 2446 ms. Trace:
I20260709 13:21:51.428624 23546 raft_consensus.cc:515] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:50.776726 23548 raft_consensus.cc:493] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:50.806547 23539 raft_consensus.cc:515] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.428885 23540 raft_consensus.cc:515] T be220d19020b428a8b1cc356f32ff5ad P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.430143 23544 leader_election.cc:290] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:51.428822 22818 rpcz_store.cc:276] 0709 13:21:48.351816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:48.351946 (+   130us) service_pool.cc:224] Handling call
0709 13:21:48.352327 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:48.352741 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:48.352773 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:48.353266 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 13:21:48.410085 (+ 56819us) raft_consensus.cc:1645] Marking committed up to 0
0709 13:21:48.412122 (+  2037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:48.433780 (+ 21658us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:50.798147 (+2364367us) raft_consensus.cc:1685] finished
0709 13:21:50.798183 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:50.798407 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1159,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":176,"thread_start_us":49447,"threads_started":1,"wal-append.queue_time_us":87715,"wal-append.run_cpu_time_us":2760,"wal-append.run_wall_time_us":1600886}
I20260709 13:21:50.825301 23554 raft_consensus.cc:493] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:51.431041 23539 leader_election.cc:290] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:50.824636 23553 raft_consensus.cc:493] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:51.431264 23540 leader_election.cc:290] T be220d19020b428a8b1cc356f32ff5ad P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:50.829347 23555 raft_consensus.cc:493] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:50.832369 23559 raft_consensus.cc:493] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:51.431674 23553 raft_consensus.cc:515] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:50.833770 23561 raft_consensus.cc:493] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:50.836730 23563 raft_consensus.cc:493] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.431134 23554 raft_consensus.cc:515] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.432960 23563 raft_consensus.cc:515] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:50.831465 23558 raft_consensus.cc:493] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.434193 23558 raft_consensus.cc:515] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.435137 23563 leader_election.cc:290] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:50.847594 22814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1394978547a410199b4f798a10edb2d" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:51.436013 22814 raft_consensus.cc:3060] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:51.436373 23558 leader_election.cc:290] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:51.426762 22894 rpcz_store.cc:276] 0709 13:21:48.299713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:48.300064 (+   351us) service_pool.cc:224] Handling call
0709 13:21:48.301552 (+  1488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:48.316696 (+ 15144us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:48.316731 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:48.317194 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 13:21:48.390045 (+ 72851us) raft_consensus.cc:1645] Marking committed up to 0
0709 13:21:48.397311 (+  7266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:48.401001 (+  3690us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:51.426003 (+3025002us) raft_consensus.cc:1685] finished
0709 13:21:51.426027 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:51.426203 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14740,"raft.queue_time_us":16587,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":123,"thread_start_us":64341,"threads_started":1,"wal-append.queue_time_us":92529}
I20260709 13:21:51.438712 23548 raft_consensus.cc:515] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.440965 23548 leader_election.cc:290] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:50.855839 23568 raft_consensus.cc:493] T 7802e4d0fd394596b42b661c9eb4c7ce P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:51.442229 23568 raft_consensus.cc:515] T 7802e4d0fd394596b42b661c9eb4c7ce P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.443200 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "084ee2ebbf564ed583f84c1227d29bd4" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.427894 23545 raft_consensus.cc:515] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.445163 23555 raft_consensus.cc:515] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:50.874756 23572 raft_consensus.cc:493] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:51.446197 23572 raft_consensus.cc:515] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:51.447327 23555 leader_election.cc:290] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:51.435050 23554 leader_election.cc:290] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:51.432127 23559 raft_consensus.cc:515] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.448342 23572 leader_election.cc:290] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:51.449841 23559 leader_election.cc:290] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:50.894076 23575 raft_consensus.cc:493] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.451025 23575 raft_consensus.cc:515] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.453174 23575 leader_election.cc:290] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
W20260709 13:21:50.895306 23009 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.684s	user 0.000s	sys 0.012s
I20260709 13:21:50.896720 23576 raft_consensus.cc:493] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.454182 23576 raft_consensus.cc:515] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.455435 22814 raft_consensus.cc:2468] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:51.456265 23576 leader_election.cc:290] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:50.898406 23573 raft_consensus.cc:493] T 10066ccf8d174f4a9dd69253613857b8 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:50.900406 23574 raft_consensus.cc:493] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:50.903240 23579 raft_consensus.cc:493] T 50b783c62bd54d4c8f4f1795b7409958 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:51.457000 23573 raft_consensus.cc:515] T 10066ccf8d174f4a9dd69253613857b8 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.457137 23574 raft_consensus.cc:515] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
W20260709 13:21:50.912250 22934 connection.cc:256] RPC call timeout handler was delayed by -1.2162476959999822s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1547s for a response
W20260709 13:21:51.458173 22934 connection.cc:256] RPC call timeout handler was delayed by -1.0615827100000388s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15900000000000003s for a response
I20260709 13:21:51.459308 23573 leader_election.cc:290] T 10066ccf8d174f4a9dd69253613857b8 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:50.899413 23569 raft_consensus.cc:493] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:50.905328 23577 raft_consensus.cc:493] T cabd9adbf414496bbaab3946b413d313 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:51.459481 23574 leader_election.cc:290] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
W20260709 13:21:51.010716 22783 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.23.68:44509 timed out after 1.886s (SENT) [suppressed 240 similar messages]
I20260709 13:21:51.133389 23581 raft_consensus.cc:493] T 8ad9b8807851437d9a22c0a33f567c15 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:51.459980 23577 raft_consensus.cc:515] T cabd9adbf414496bbaab3946b413d313 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:51.175508 23584 raft_consensus.cc:493] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.460348 23581 raft_consensus.cc:515] T 8ad9b8807851437d9a22c0a33f567c15 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
W20260709 13:21:51.176096 22932 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 50832.7us
W20260709 13:21:51.453775 23009 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.243s	user 0.000s	sys 0.013s
I20260709 13:21:51.462532 23581 leader_election.cc:290] T 8ad9b8807851437d9a22c0a33f567c15 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:51.168867 23582 raft_consensus.cc:493] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:51.189335 23583 raft_consensus.cc:493] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
W20260709 13:21:51.428267 22715 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.910s	user 0.004s	sys 0.006s
I20260709 13:21:51.198827 23585 raft_consensus.cc:493] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:51.464066 23585 raft_consensus.cc:515] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.179714 22969 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52366 (request call id 103) took 2806 ms. Trace:
I20260709 13:21:51.465358 23546 leader_election.cc:290] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:51.256647 23586 raft_consensus.cc:493] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.272177 23588 consensus_queue.cc:1048] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:21:51.211313 23007 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.697s	user 0.012s	sys 0.005s
W20260709 13:21:51.466449 23007 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.952s	user 0.013s	sys 0.006s
I20260709 13:21:51.466168 23586 raft_consensus.cc:515] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.250923 23587 raft_consensus.cc:493] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:51.468286 23586 leader_election.cc:290] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:51.460929 23584 raft_consensus.cc:515] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:51.464915 22969 rpcz_store.cc:276] 0709 13:21:48.373525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:48.374241 (+   716us) service_pool.cc:224] Handling call
0709 13:21:48.374646 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:48.375075 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:48.375126 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:48.396438 (+ 21312us) log.cc:844] Serialized 26 byte log entry
0709 13:21:48.441423 (+ 44985us) raft_consensus.cc:1645] Marking committed up to 0
0709 13:21:48.441469 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:48.441548 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:51.179365 (+2737817us) raft_consensus.cc:1685] finished
0709 13:21:51.179389 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:51.179594 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5583,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":4447,"thread_start_us":25656,"threads_started":1,"wal-append.queue_time_us":32293}
I20260709 13:21:51.292835 23552 raft_consensus.cc:493] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:51.470060 23552 raft_consensus.cc:515] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
W20260709 13:21:51.463160 22932 connection.cc:256] RPC call timeout handler was delayed by -1.6610047269998631s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15900000000000003s for a response
W20260709 13:21:51.471858 22932 connection.cc:256] RPC call timeout handler was delayed by -1.3424497440000778s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18910000000000002s for a response
I20260709 13:21:51.468305 23587 raft_consensus.cc:515] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:51.472299 23552 leader_election.cc:290] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:51.472925 23506 raft_consensus.cc:493] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.473444 23506 raft_consensus.cc:515] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:51.474165 23397 raft_consensus.cc:493] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:51.474661 23397 raft_consensus.cc:515] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:51.475292 23506 leader_election.cc:290] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:51.475623 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a064be2ec640a4a8283957d0b9b98c" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.475845 23506 raft_consensus.cc:493] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.476593 23397 leader_election.cc:290] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:51.295055 23592 raft_consensus.cc:493] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.477443 23503 raft_consensus.cc:493] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.477195 23592 raft_consensus.cc:515] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.477962 23503 raft_consensus.cc:515] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.479436 23592 leader_election.cc:290] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:51.480057 23539 raft_consensus.cc:493] T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:51.473905 23576 raft_consensus.cc:493] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.480649 23526 raft_consensus.cc:493] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:51.294045 23565 raft_consensus.cc:493] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:51.480814 23576 raft_consensus.cc:515] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:51.300494 23596 raft_consensus.cc:493] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:51.305028 23594 raft_consensus.cc:493] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:51.306758 23597 raft_consensus.cc:493] T 086a7123d3ad43d79101a7c5de5158fe P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:51.308764 23599 raft_consensus.cc:493] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:51.433802 23553 leader_election.cc:290] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:51.310673 23598 raft_consensus.cc:493] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.481865 23596 raft_consensus.cc:515] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:51.482296 23599 raft_consensus.cc:515] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.482683 23598 raft_consensus.cc:515] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.480275 23538 raft_consensus.cc:493] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:51.484285 23596 leader_election.cc:290] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:51.484771 23598 leader_election.cc:290] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
W20260709 13:21:51.313282 23006 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.103s	user 0.005s	sys 0.015s
I20260709 13:21:51.481978 23597 raft_consensus.cc:515] T 086a7123d3ad43d79101a7c5de5158fe P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
W20260709 13:21:51.485462 23006 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.275s	user 0.006s	sys 0.015s
I20260709 13:21:51.463183 23582 raft_consensus.cc:515] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.315177 23600 raft_consensus.cc:493] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:51.485368 23594 raft_consensus.cc:515] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.486182 23600 raft_consensus.cc:515] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.319072 23602 raft_consensus.cc:493] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:51.320968 23601 raft_consensus.cc:493] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:51.463426 23583 raft_consensus.cc:515] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:51.488058 22815 raft_consensus.cc:1275] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:21:51.330463 23604 raft_consensus.cc:493] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:51.488222 23600 leader_election.cc:290] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:51.315703 22816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49352 (request call id 90) took 3037 ms. Trace:
I20260709 13:21:51.332719 23606 raft_consensus.cc:493] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:51.333549 23605 raft_consensus.cc:493] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.323875 23603 raft_consensus.cc:493] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:51.489116 23605 raft_consensus.cc:515] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.489696 23583 leader_election.cc:290] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:51.481535 23565 raft_consensus.cc:515] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:51.491235 23605 leader_election.cc:290] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:51.339079 23557 raft_consensus.cc:493] T 8f631a34c02c4de8ae5bd70760fba972 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:51.491405 23603 raft_consensus.cc:515] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:51.491811 23557 raft_consensus.cc:515] T 8f631a34c02c4de8ae5bd70760fba972 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.492609 23538 raft_consensus.cc:515] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.493521 23603 leader_election.cc:290] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
W20260709 13:21:51.342093 23534 log.cc:927] Time spent T b74a5704470044c1bb9cd7c18e3b6000 P 27745ef236cd494684f62a1269411a68: Append to log took a long time: real 2.090s	user 0.008s	sys 0.014s
I20260709 13:21:51.494606 23538 leader_election.cc:290] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:51.357832 23608 raft_consensus.cc:493] T 5e7b082dcf344fbf83e0ddec4b08b6de P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
W20260709 13:21:51.364054 23036 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36860 (request call id 111) took 2940 ms (client timeout 1886 ms). Trace:
I20260709 13:21:51.477094 23296 raft_consensus.cc:493] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:51.367349 23609 raft_consensus.cc:515] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.495728 23296 raft_consensus.cc:515] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:51.488832 22816 rpcz_store.cc:276] 0709 13:21:48.287069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:48.288754 (+  1685us) service_pool.cc:224] Handling call
0709 13:21:48.296760 (+  8006us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:48.309970 (+ 13210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:48.310014 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:48.310518 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 13:21:48.384246 (+ 73728us) raft_consensus.cc:1645] Marking committed up to 0
0709 13:21:48.385269 (+  1023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:48.388658 (+  3389us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:51.310651 (+2921993us) spinlock_profiling.cc:243] Waited 571 ms on lock 0x7b44002ce0f0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1953d3c 00007f5ce1954197 00007f5ce45d3035 00007f5ce45d3769 00007f5ce45d396a 00007f5ce45d3b3d 00007f5ce45ab398 00007f5ce45a676a 00007f5ce45a5bb6 00007f5ce45a57d8
0709 13:21:51.315321 (+  4670us) raft_consensus.cc:1685] finished
0709 13:21:51.315354 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:51.315633 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9696,"raft.queue_time_us":12942,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"spinlock_wait_cycles":1313661184,"thread_start_us":61822,"threads_started":1,"wal-append.queue_time_us":102300}
W20260709 13:21:49.215097 23043 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.008s	sys 0.012s
I20260709 13:21:51.471565 23584 leader_election.cc:290] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:51.500185 23040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36850 (request call id 83) took 3149 ms. Trace:
I20260709 13:21:51.367940 23610 raft_consensus.cc:493] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
W20260709 13:21:51.368394 23038 rpcz_store.cc:269] 0709 13:21:48.440501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:48.445958 (+  5457us) service_pool.cc:224] Handling call
0709 13:21:51.367538 (+2921580us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:51.501245 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968e16cef40146509d654274b9fd2687" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.374063 23085 tablet_bootstrap.cc:492] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68: No bootstrap required, opened a new log
I20260709 13:21:51.501513 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.501832 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3139446c554b419fc28cad31653600" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.374552 23084 tablet_bootstrap.cc:492] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:51.502238 22895 raft_consensus.cc:2468] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:51.502686 23084 ts_tablet_manager.cc:1403] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 3.145s	user 0.025s	sys 0.004s
I20260709 13:21:51.375685 23611 raft_consensus.cc:493] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.382294 23589 raft_consensus.cc:493] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:51.506232 23611 raft_consensus.cc:515] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.506341 23589 raft_consensus.cc:515] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:51.382743 23556 raft_consensus.cc:493] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:51.377986 23607 raft_consensus.cc:493] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.509084 23556 raft_consensus.cc:515] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:51.509184 23607 raft_consensus.cc:515] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.385727 23560 raft_consensus.cc:493] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.387974 23614 raft_consensus.cc:493] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.395439 23566 raft_consensus.cc:493] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.397330 23096 leader_election.cc:290] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:51.378212 23613 raft_consensus.cc:493] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:51.398962 22813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52d63f523d24195bc8127113598ff91" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:51.386391 23562 raft_consensus.cc:493] T d1934129c8764c018495eff97d43a3b5 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:51.412070 23504 raft_consensus.cc:515] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.511936 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10066ccf8d174f4a9dd69253613857b8" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.511583 23560 raft_consensus.cc:515] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.408205 22718 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50740 (request call id 73) took 3070 ms. Trace:
I20260709 13:21:51.466179 23585 leader_election.cc:290] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:51.416349 23515 raft_consensus.cc:515] T 78234c90229d4615ab9e2c2a811bd0f8 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.487260 23582 leader_election.cc:290] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:50.819487 23551 consensus_queue.cc:1048] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:51.508664 23589 leader_election.cc:290] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:51.514763 22812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.514045 23560 leader_election.cc:290] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:50.853567 23564 raft_consensus.cc:493] T 171526d2fe264ee8b1168192bec31c7a P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:51.515617 23614 raft_consensus.cc:515] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:50.871793 23571 raft_consensus.cc:493] T 1a4c294dbaeb4040b157ad72102a9e56 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:51.481865 23577 leader_election.cc:290] T cabd9adbf414496bbaab3946b413d313 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:51.488680 23601 raft_consensus.cc:515] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.516737 23571 raft_consensus.cc:515] T 1a4c294dbaeb4040b157ad72102a9e56 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:51.518417 23601 leader_election.cc:290] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:51.518721 23614 leader_election.cc:290] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:51.519089 23571 leader_election.cc:290] T 1a4c294dbaeb4040b157ad72102a9e56 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:51.432616 23561 raft_consensus.cc:515] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:51.511904 23613 raft_consensus.cc:515] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.522433 23561 leader_election.cc:290] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
W20260709 13:21:50.916882 23008 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.483s	user 0.003s	sys 0.014s
W20260709 13:21:51.523094 23008 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.089s	user 0.003s	sys 0.015s
I20260709 13:21:51.524036 23613 leader_election.cc:290] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:51.514972 23610 raft_consensus.cc:515] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.130337 23580 raft_consensus.cc:493] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:51.526206 23580 raft_consensus.cc:515] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:51.526953 23610 leader_election.cc:290] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:51.488664 23604 raft_consensus.cc:515] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.528337 23580 leader_election.cc:290] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:51.281725 23590 raft_consensus.cc:493] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:51.529042 23590 raft_consensus.cc:515] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.514173 23606 raft_consensus.cc:515] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.531306 23590 leader_election.cc:290] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:51.531963 23618 raft_consensus.cc:493] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.532088 23606 leader_election.cc:290] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
W20260709 13:21:51.495415 23036 rpcz_store.cc:269] 0709 13:21:48.424785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:48.424927 (+   142us) service_pool.cc:224] Handling call
0709 13:21:51.363970 (+2939043us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:51.532495 23618 raft_consensus.cc:515] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.515218 23608 raft_consensus.cc:515] T 5e7b082dcf344fbf83e0ddec4b08b6de P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.457248 23579 raft_consensus.cc:515] T 50b783c62bd54d4c8f4f1795b7409958 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:51.535164 23608 leader_election.cc:290] T 5e7b082dcf344fbf83e0ddec4b08b6de P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:51.502091 23085 ts_tablet_manager.cc:1403] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68: Time spent bootstrapping tablet: real 3.182s	user 0.099s	sys 0.127s
I20260709 13:21:51.535715 23579 leader_election.cc:290] T 50b783c62bd54d4c8f4f1795b7409958 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:51.536577 23621 raft_consensus.cc:493] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.537104 23621 raft_consensus.cc:515] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.481194 23545 leader_election.cc:290] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:51.294178 23593 raft_consensus.cc:493] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:51.514513 23562 raft_consensus.cc:515] T d1934129c8764c018495eff97d43a3b5 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.539232 23593 raft_consensus.cc:515] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.540681 23562 leader_election.cc:290] T d1934129c8764c018495eff97d43a3b5 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:51.299098 23578 raft_consensus.cc:493] T 44841295bfd54159a48e285582824bad P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
W20260709 13:21:51.540853 22783 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 530162us
I20260709 13:21:51.541348 23593 leader_election.cc:290] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:51.541404 23578 raft_consensus.cc:515] T 44841295bfd54159a48e285582824bad P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:51.463655 23525 raft_consensus.cc:493] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.513253 22718 rpcz_store.cc:276] 0709 13:21:48.338298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:48.338440 (+   142us) service_pool.cc:224] Handling call
0709 13:21:51.408095 (+3069655us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":592630,"reported_tablets":2}
I20260709 13:21:51.542793 23525 raft_consensus.cc:515] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.500600 23040 rpcz_store.cc:276] 0709 13:21:48.350649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:48.359635 (+  8986us) service_pool.cc:224] Handling call
0709 13:21:48.360013 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:48.374399 (+ 14386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:48.374429 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:48.374961 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 13:21:48.669976 (+295015us) raft_consensus.cc:1645] Marking committed up to 0
0709 13:21:48.714229 (+ 44253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:48.714323 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:51.341459 (+2627136us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b4400022b4c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce1844c85 00007f5ce18456a2 00007f5ce45b71db 00007f5ce45b2aaf 00007f5ce45a686e 00007f5ce45a5bb6 00007f5ce45a57d8 00007f5ce45af221 00007f5ce45af1d9 00007f5ce45af169
0709 13:21:51.499836 (+158377us) raft_consensus.cc:1685] finished
0709 13:21:51.499863 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:51.500096 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20287,"raft.queue_time_us":20446,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":14104,"spinlock_wait_cycles":303174784,"thread_start_us":121426,"threads_started":1,"wal-append.queue_time_us":875991}
I20260709 13:21:51.544950 23525 leader_election.cc:290] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:51.487838 23602 raft_consensus.cc:515] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:51.538326 23515 leader_election.cc:290] T 78234c90229d4615ab9e2c2a811bd0f8 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:51.493791 23557 leader_election.cc:290] T 8f631a34c02c4de8ae5bd70760fba972 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:51.546984 23602 leader_election.cc:290] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:51.487258 23597 leader_election.cc:290] T 086a7123d3ad43d79101a7c5de5158fe P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:51.492131 23565 leader_election.cc:290] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:51.497337 23539 raft_consensus.cc:515] T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.548080 23515 raft_consensus.cc:493] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.487600 23594 leader_election.cc:290] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:51.516373 23564 raft_consensus.cc:515] T 171526d2fe264ee8b1168192bec31c7a P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:51.497366 23609 leader_election.cc:290] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:51.529672 23604 leader_election.cc:290] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:51.511471 23607 leader_election.cc:290] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:51.484900 23599 leader_election.cc:290] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:51.428162 23541 leader_election.cc:290] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:51.551098 23564 leader_election.cc:290] T 171526d2fe264ee8b1168192bec31c7a P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:51.444594 23568 leader_election.cc:290] T 7802e4d0fd394596b42b661c9eb4c7ce P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:51.376963 23612 raft_consensus.cc:493] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:51.481161 23526 raft_consensus.cc:515] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
W20260709 13:21:51.423653 23037 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36850 (request call id 87) took 2966 ms (client timeout 1891 ms). Trace:
I20260709 13:21:51.511219 23556 leader_election.cc:290] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:51.512035 22813 raft_consensus.cc:3060] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:51.551847 23612 raft_consensus.cc:515] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:51.553241 23526 leader_election.cc:290] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:51.508541 23611 leader_election.cc:290] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:51.514110 23566 raft_consensus.cc:515] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.554170 23612 leader_election.cc:290] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:51.511118 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ea5c83aada45d3a98027069e96658c" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.554989 22819 raft_consensus.cc:2468] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
W20260709 13:21:51.296859 22785 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.297s	user 0.041s	sys 0.046s
I20260709 13:21:51.555532 23566 leader_election.cc:290] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:51.497762 23296 leader_election.cc:290] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
W20260709 13:21:51.556504 22785 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.557s	user 0.043s	sys 0.047s
I20260709 13:21:51.463663 23569 raft_consensus.cc:515] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:51.514706 22890 raft_consensus.cc:2468] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:51.513605 23504 leader_election.cc:290] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:51.558348 23569 leader_election.cc:290] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
W20260709 13:21:51.558918 22785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 3.55895e+06us
W20260709 13:21:51.558200 23037 rpcz_store.cc:269] 0709 13:21:48.457568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:48.458186 (+   618us) service_pool.cc:224] Handling call
0709 13:21:51.423395 (+2965209us) spinlock_profiling.cc:243] Waited 47.7 ms on lock 0x7b54006d5454. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:21:51.423571 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":109701888}
I20260709 13:21:51.517127 22893 raft_consensus.cc:2468] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:51.551326 23578 leader_election.cc:290] T 44841295bfd54159a48e285582824bad P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:51.473706 23587 leader_election.cc:290] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:51.549330 23539 leader_election.cc:290] T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:51.564103 22813 raft_consensus.cc:2468] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:51.565511 22811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.566915 22811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e230e64c6984b72a2ba543f2a017e25" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.565140 22962 raft_consensus.cc:3060] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:51.567929 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02259962b5504debb700dd6b7c12915b" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.569156 22812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b752c38e1b47a489a3d720801661d1" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.570360 22812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.570644 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.571084 22812 raft_consensus.cc:2468] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
I20260709 13:21:51.571460 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be220d19020b428a8b1cc356f32ff5ad" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.498668 22820 raft_consensus.cc:2468] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:51.567945 22815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be220d19020b428a8b1cc356f32ff5ad" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.567525 22811 raft_consensus.cc:2468] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
I20260709 13:21:51.570330 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "084ee2ebbf564ed583f84c1227d29bd4" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.480000 23503 leader_election.cc:290] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:51.573349 22892 raft_consensus.cc:2468] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
I20260709 13:21:51.548388 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955a6d8dbbe94ef9be9c1f69b10675d8" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
W20260709 13:21:51.499778 23043 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.038s	user 0.011s	sys 0.012s
I20260709 13:21:51.495162 23538 raft_consensus.cc:493] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:51.574753 23043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36848 (request call id 88) took 3225 ms. Trace:
I20260709 13:21:51.575980 22714 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50746 (request call id 60) took 3286 ms. Trace:
I20260709 13:21:51.575176 23043 rpcz_store.cc:276] 0709 13:21:48.349556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:48.350817 (+  1261us) service_pool.cc:224] Handling call
0709 13:21:48.351269 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:48.382129 (+ 30860us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:48.385463 (+  3334us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:48.411911 (+ 26448us) log.cc:844] Serialized 26 byte log entry
0709 13:21:51.574382 (+3162471us) raft_consensus.cc:1645] Marking committed up to 0
0709 13:21:51.574431 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:51.574506 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:51.574551 (+    45us) raft_consensus.cc:1685] finished
0709 13:21:51.574567 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:51.574696 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38484,"raft.queue_time_us":28880,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":4548,"thread_start_us":3112739,"threads_started":1,"wal-append.queue_time_us":1161014,"wal-append.run_cpu_time_us":2175,"wal-append.run_wall_time_us":1002178}
I20260709 13:21:51.576360 22714 rpcz_store.cc:276] 0709 13:21:48.294940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:48.295801 (+   861us) service_pool.cc:224] Handling call
0709 13:21:51.575918 (+3280117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":112351,"reported_tablets":2}
I20260709 13:21:51.577138 23538 raft_consensus.cc:515] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:51.577980 22715 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50724 (request call id 63) took 3428 ms. Trace:
I20260709 13:21:51.579217 23538 leader_election.cc:290] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:51.579798 23538 raft_consensus.cc:493] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:51.565233 22813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "084ee2ebbf564ed583f84c1227d29bd4" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.580257 23538 raft_consensus.cc:515] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:51.581318 23539 raft_consensus.cc:493] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:51.476413 23506 raft_consensus.cc:515] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:51.581844 23539 raft_consensus.cc:515] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:51.570961 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6a9c5a5fdc42e78dbbe6b411416c1a" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:51.583549 22891 raft_consensus.cc:2393] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 in current term 1: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:51.583657 23540 raft_consensus.cc:493] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:51.583853 23539 leader_election.cc:290] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:51.584220 23540 raft_consensus.cc:515] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:51.584753 23539 raft_consensus.cc:493] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:51.585250 23539 raft_consensus.cc:515] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:51.549686 23515 raft_consensus.cc:515] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:51.587653 23515 leader_election.cc:290] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:51.484885 23596 raft_consensus.cc:493] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:51.588788 23596 raft_consensus.cc:515] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:51.590752 23596 leader_election.cc:290] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:51.505648 23084 raft_consensus.cc:359] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:51.593660 23084 raft_consensus.cc:385] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:51.593987 23084 raft_consensus.cc:740] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:51.594309 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050e9ef11f664c849a40cbb6acb25f19" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.547760 23085 raft_consensus.cc:359] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:51.595497 23085 raft_consensus.cc:385] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:51.595645 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "989cd1a042864c419d8c10665cb8afd0" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.597406 23506 leader_election.cc:290] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:51.598639 23539 leader_election.cc:290] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:51.578397 22715 rpcz_store.cc:276] 0709 13:21:48.189860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:48.198978 (+  9118us) service_pool.cc:224] Handling call
0709 13:21:51.466456 (+3267478us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:51.466488 (+    32us) write_op.cc:454] Released schema lock
0709 13:21:51.577895 (+111407us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:50.755484 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:50.756130 (+   646us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:50.756165 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:21:50.756185 (+    20us) tablet.cc:662] Decoding operations
0709 13:21:50.756634 (+   449us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:50.756718 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:50.756748 (+    30us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:21:50.757147 (+   399us) tablet.cc:701] Row locks acquired
0709 13:21:50.757171 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:21:50.757434 (+   263us) write_op.cc:270] Start()
0709 13:21:50.757552 (+   118us) write_op.cc:276] Timestamp: P: 1783603310757377 usec, L: 0
0709 13:21:50.757576 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:21:50.758348 (+   772us) log.cc:844] Serialized 922 byte log entry
0709 13:21:50.788958 (+ 30610us) op_driver.cc:464] REPLICATION: finished
0709 13:21:50.792455 (+  3497us) write_op.cc:301] APPLY: starting
0709 13:21:50.792520 (+    65us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:50.792607 (+    87us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:50.792707 (+   100us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:50.792986 (+   279us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:50.793148 (+   162us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:21:50.793195 (+    47us) write_op.cc:312] APPLY: finished
0709 13:21:50.793386 (+   191us) log.cc:844] Serialized 28 byte log entry
0709 13:21:50.794105 (+   719us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:50.794203 (+    98us) write_op.cc:454] Released schema lock
0709 13:21:50.794241 (+    38us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":5394,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":3285,"apply.run_cpu_time_us":1565,"apply.run_wall_time_us":2081,"num_ops":2,"prepare.queue_time_us":1237002,"prepare.run_cpu_time_us":10555,"prepare.run_wall_time_us":19894,"raft.queue_time_us":6484,"raft.run_cpu_time_us":3244,"raft.run_wall_time_us":3244,"replication_time_us":31242,"thread_start_us":1972578,"threads_started":4,"wal-append.queue_time_us":16974}]]}
W20260709 13:21:51.603116 22860 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 95777.3us
W20260709 13:21:51.604161 22783 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 58349.3us
I20260709 13:21:51.604895 23085 raft_consensus.cc:740] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Initialized, Role: FOLLOWER
I20260709 13:21:51.594619 23084 consensus_queue.cc:260] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:51.607230 23539 raft_consensus.cc:493] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:51.607419 23540 leader_election.cc:290] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:51.607895 23539 raft_consensus.cc:515] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:51.609086 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8b119f5c5844d1be439423b333ec60" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:51.609724 22968 raft_consensus.cc:3060] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:51.610078 23034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7802e4d0fd394596b42b661c9eb4c7ce" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.593797 22815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f631a34c02c4de8ae5bd70760fba972" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.611397 23506 raft_consensus.cc:493] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:51.611850 23034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a064be2ec640a4a8283957d0b9b98c" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.611900 23296 raft_consensus.cc:493] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:51.612954 23296 raft_consensus.cc:515] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:51.613806 23034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10066ccf8d174f4a9dd69253613857b8" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.614281 23035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "084ee2ebbf564ed583f84c1227d29bd4" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.614960 23035 raft_consensus.cc:2468] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:51.615444 23034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6a9c5a5fdc42e78dbbe6b411416c1a" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.616091 23034 raft_consensus.cc:2468] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:51.603864 22860 leader_election.cc:304] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:51.618386 23557 raft_consensus.cc:2804] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:51.612339 23506 raft_consensus.cc:515] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.616760 23296 leader_election.cc:290] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:51.619105 23557 raft_consensus.cc:697] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:51.618981 23038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94800e302e4641bb857f2d160f1f60ea" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.620141 23557 consensus_queue.cc:237] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.621025 22860 leader_election.cc:304] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:51.618799 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1394978547a410199b4f798a10edb2d" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.621814 23587 consensus_queue.cc:1048] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:51.621897 22860 leader_election.cc:304] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:51.635535 23085 consensus_queue.cc:260] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:51.636761 23084 ts_tablet_manager.cc:1434] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.134s	user 0.006s	sys 0.000s
I20260709 13:21:51.638370 23556 raft_consensus.cc:2804] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:51.639598 22860 leader_election.cc:304] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:51.643409 23008 leader_election.cc:304] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:51.650278 23084 tablet_bootstrap.cc:492] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:51.618523 23576 leader_election.cc:290] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:51.610123 23539 leader_election.cc:290] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:51.622661 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52d63f523d24195bc8127113598ff91" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:51.539198 23621 leader_election.cc:290] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:51.638885 23556 raft_consensus.cc:493] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:51.646914 23568 raft_consensus.cc:2804] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:51.647179 23607 raft_consensus.cc:2804] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:21:51.661198 22934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 200931us
I20260709 13:21:51.654009 23538 leader_election.cc:290] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:51.652173 22813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
W20260709 13:21:51.667327 22785 connection.cc:256] RPC call timeout handler was delayed by -2.206030655999939s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
I20260709 13:21:51.661993 23556 raft_consensus.cc:3060] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:51.660689 23040 raft_consensus.cc:3060] T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:51.661301 23539 raft_consensus.cc:2804] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:51.660008 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1394978547a410199b4f798a10edb2d" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.661664 23603 raft_consensus.cc:493] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:51.683455 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6a9c5a5fdc42e78dbbe6b411416c1a" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:51.665215 23607 raft_consensus.cc:493] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:51.668744 23553 raft_consensus.cc:493] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
W20260709 13:21:51.680719 22785 connection.cc:256] RPC call timeout handler was delayed by -1.9396048070000234s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1829s for a response
I20260709 13:21:51.716588 22968 raft_consensus.cc:2468] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:51.694283 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b752c38e1b47a489a3d720801661d1" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
W20260709 13:21:51.715873 22766 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:51.682878 23539 raft_consensus.cc:493] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.666678 23600 raft_consensus.cc:493] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:51.666872 23568 raft_consensus.cc:697] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:51.707963 22934 leader_election.cc:304] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:51.705086 22964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e638dc505d4b04b802e91b8863d833" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.692093 22961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.733963 23596 raft_consensus.cc:2804] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:51.692437 22811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.691174 22815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505eab84b06f46ee9d336abf01353dee" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.736435 22961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e230e64c6984b72a2ba543f2a017e25" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.692654 22967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e230e64c6984b72a2ba543f2a017e25" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.737854 23035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f631a34c02c4de8ae5bd70760fba972" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.690667 23037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.710438 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3139446c554b419fc28cad31653600" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.710258 23506 leader_election.cc:290] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:51.732185 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cabd9adbf414496bbaab3946b413d313" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.732518 22965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd78248be3954373ad4fd60b8ae7593d" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.732707 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39611d2ff3524bc0840e4fd222d96a84" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.733381 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630a5e60917545b3a8072fc8358207c5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.738348 22967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.685858 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02259962b5504debb700dd6b7c12915b" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.726742 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "087d64074b5943f0a7d789ead06f4783" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.735742 22814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7802e4d0fd394596b42b661c9eb4c7ce" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.721127 23607 raft_consensus.cc:3060] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:51.724004 23553 raft_consensus.cc:515] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.734436 23596 raft_consensus.cc:493] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.733424 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086a7123d3ad43d79101a7c5de5158fe" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.773438 23596 raft_consensus.cc:3060] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:51.767511 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "087d64074b5943f0a7d789ead06f4783" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.774698 22968 raft_consensus.cc:2468] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:51.751904 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b154956cc17f4ba68f906daffbcedff6" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.745123 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.745297 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1934129c8764c018495eff97d43a3b5" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.757071 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.738291 23036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44841295bfd54159a48e285582824bad" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.755668 23037 raft_consensus.cc:2468] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:51.778466 22960 raft_consensus.cc:2468] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
I20260709 13:21:51.748839 22812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e09daa4f2dd474d96ae9c4dfa1a8758" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.746628 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968e16cef40146509d654274b9fd2687" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.746944 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac141e204e3f4d3c84d6651893bdfc6f" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.745961 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b154956cc17f4ba68f906daffbcedff6" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.750154 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.781988 23041 raft_consensus.cc:2468] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:51.773471 22967 raft_consensus.cc:2468] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:51.784739 23553 leader_election.cc:290] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:51.778123 22969 raft_consensus.cc:2410] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 for term 3 because replica has last-logged OpId of term: 2 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:21:51.786057 22811 raft_consensus.cc:2468] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2.
I20260709 13:21:51.719300 23603 raft_consensus.cc:515] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.743093 22934 leader_election.cc:304] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:51.743911 22815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abacd5d0d1e54db38a397e3c35c0cb81" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.750629 23034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ecf8a5022f43c787aaff1cf7679264" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.734902 23600 raft_consensus.cc:515] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:51.750258 23038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74a5704470044c1bb9cd7c18e3b6000" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
W20260709 13:21:51.745580 22749 debug-util.cc:398] Leaking SignalData structure 0x7b08003eb220 after lost signal to thread 14432
I20260709 13:21:51.787513 23556 raft_consensus.cc:515] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.788157 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78234c90229d4615ab9e2c2a811bd0f8" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.788625 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1934129c8764c018495eff97d43a3b5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.789361 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ea5c83aada45d3a98027069e96658c" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.788866 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.753517 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014651e80f9f44e599d2d8e4d308d2a6" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.790323 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.805341 23600 leader_election.cc:290] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:51.805971 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.807175 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3139446c554b419fc28cad31653600" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:51.807988 23556 leader_election.cc:290] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:51.801730 23397 raft_consensus.cc:2804] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:51.809456 23397 raft_consensus.cc:493] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:51.809839 23397 raft_consensus.cc:3060] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:51.789433 22889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f631a34c02c4de8ae5bd70760fba972" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.747300 23039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014651e80f9f44e599d2d8e4d308d2a6" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.812897 23039 raft_consensus.cc:2468] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:51.796384 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4c294dbaeb4040b157ad72102a9e56" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.726214 23618 leader_election.cc:290] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:51.730418 23539 raft_consensus.cc:3060] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:21:51.799631 22934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 56643.8us
W20260709 13:21:51.800499 22785 leader_election.cc:343] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:51.798481 22961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ea5c83aada45d3a98027069e96658c" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.817546 22934 leader_election.cc:304] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 
W20260709 13:21:51.822674 22785 leader_election.cc:343] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:51.827713 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1394978547a410199b4f798a10edb2d" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:51.829195 23034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d217f95123f542e886aa049a608a88d4" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.829900 23034 raft_consensus.cc:2468] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:51.829631 23607 raft_consensus.cc:515] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.822911 23600 consensus_queue.cc:1048] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:21:51.832548 23607 leader_election.cc:290] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:51.820722 23568 consensus_queue.cc:237] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:51.824764 22785 leader_election.cc:304] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:51.838174 23600 raft_consensus.cc:2749] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:51.840497 23035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.841257 23035 raft_consensus.cc:2468] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:51.841615 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e09daa4f2dd474d96ae9c4dfa1a8758" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.842309 23043 raft_consensus.cc:2468] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:51.842362 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6a9c5a5fdc42e78dbbe6b411416c1a" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
W20260709 13:21:51.823992 22749 debug-util.cc:398] Leaking SignalData structure 0x7b08003657c0 after lost signal to thread 22752
I20260709 13:21:51.843336 23039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a064be2ec640a4a8283957d0b9b98c" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:51.843892 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e55b4e727be4089ba7454b662d03b9c" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
W20260709 13:21:51.844487 22749 debug-util.cc:398] Leaking SignalData structure 0x7b0800329b80 after lost signal to thread 22844
I20260709 13:21:51.845029 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cabd9adbf414496bbaab3946b413d313" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
W20260709 13:21:51.845733 22749 debug-util.cc:398] Leaking SignalData structure 0x7b08001e23c0 after lost signal to thread 22919
I20260709 13:21:51.847457 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ea5c83aada45d3a98027069e96658c" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
W20260709 13:21:51.848315 22749 debug-util.cc:398] Leaking SignalData structure 0x7b08000c8ae0 after lost signal to thread 22993
W20260709 13:21:51.848647 22749 debug-util.cc:398] Leaking SignalData structure 0x7b0800085b60 after lost signal to thread 22994
I20260709 13:21:51.848567 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad9b8807851437d9a22c0a33f567c15" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.849349 23603 leader_election.cc:290] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:51.823580 23576 raft_consensus.cc:2804] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:51.829739 23084 tablet_bootstrap.cc:654] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:51.847043 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968e16cef40146509d654274b9fd2687" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.868765 23085 ts_tablet_manager.cc:1434] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68: Time spent starting tablet: real 0.333s	user 0.020s	sys 0.011s
I20260709 13:21:51.854360 22860 leader_election.cc:304] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:51.871970 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4c294dbaeb4040b157ad72102a9e56" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
W20260709 13:21:51.853636 22785 leader_election.cc:343] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:51.854051 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "989cd1a042864c419d8c10665cb8afd0" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.879035 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad9b8807851437d9a22c0a33f567c15" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.851614 22967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94800e302e4641bb857f2d160f1f60ea" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.880182 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ea5c83aada45d3a98027069e96658c" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:51.880612 22967 raft_consensus.cc:2468] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:51.881346 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4c294dbaeb4040b157ad72102a9e56" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.856796 23576 raft_consensus.cc:493] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:51.856843 22961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.854954 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b74a5704470044c1bb9cd7c18e3b6000" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.860809 23596 raft_consensus.cc:515] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.883102 22961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.883211 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505eab84b06f46ee9d336abf01353dee" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.883958 23596 leader_election.cc:290] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:51.856292 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ecf8a5022f43c787aaff1cf7679264" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.881450 22967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3139446c554b419fc28cad31653600" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.857079 22965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44841295bfd54159a48e285582824bad" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
W20260709 13:21:51.849861 22749 debug-util.cc:398] Leaking SignalData structure 0x7b080028b3e0 after lost signal to thread 23067
I20260709 13:21:51.894205 23587 raft_consensus.cc:2804] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:21:51.894515 22749 debug-util.cc:398] Leaking SignalData structure 0x7b0800226860 after lost signal to thread 23068
I20260709 13:21:51.829762 23040 raft_consensus.cc:2468] T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:51.855573 22962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050e9ef11f664c849a40cbb6acb25f19" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.855301 23037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b752c38e1b47a489a3d720801661d1" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.849980 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955a6d8dbbe94ef9be9c1f69b10675d8" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.857525 22815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a064be2ec640a4a8283957d0b9b98c" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:51.852064 22812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ea5c83aada45d3a98027069e96658c" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:51.895260 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ecf8a5022f43c787aaff1cf7679264" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.857206 23034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1934129c8764c018495eff97d43a3b5" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.857754 22964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968e16cef40146509d654274b9fd2687" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.898813 22785 leader_election.cc:304] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:51.882238 23576 raft_consensus.cc:3060] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:51.901804 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505eab84b06f46ee9d336abf01353dee" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.906052 22893 raft_consensus.cc:2468] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:51.901621 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ecf8a5022f43c787aaff1cf7679264" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.906512 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94800e302e4641bb857f2d160f1f60ea" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.907271 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02259962b5504debb700dd6b7c12915b" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.907938 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "087d64074b5943f0a7d789ead06f4783" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.908514 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94800e302e4641bb857f2d160f1f60ea" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
W20260709 13:21:51.909286 22785 consensus_peers.cc:597] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 -> Peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Couldn't send request to peer 8ef3887fb3874f8b81079e542d252944. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:51.909561 22889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050e9ef11f664c849a40cbb6acb25f19" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.910179 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086a7123d3ad43d79101a7c5de5158fe" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.910645 22785 leader_election.cc:304] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:51.902925 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955a6d8dbbe94ef9be9c1f69b10675d8" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
W20260709 13:21:51.905023 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 253174us
I20260709 13:21:51.889058 22961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d217f95123f542e886aa049a608a88d4" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:51.913164 22785 leader_election.cc:304] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:51.912460 22895 raft_consensus.cc:2468] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:51.896723 23587 raft_consensus.cc:493] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:51.904143 23037 raft_consensus.cc:2468] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:51.922698 22888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.918783 23587 raft_consensus.cc:3060] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:51.928218 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4c294dbaeb4040b157ad72102a9e56" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.917797 23545 raft_consensus.cc:2804] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:51.923489 23600 raft_consensus.cc:2804] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:51.931237 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171526d2fe264ee8b1168192bec31c7a" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.931444 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44841295bfd54159a48e285582824bad" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.931681 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cabd9adbf414496bbaab3946b413d313" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.932804 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.931350 23600 raft_consensus.cc:493] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:51.934088 23039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b783c62bd54d4c8f4f1795b7409958" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.934425 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44841295bfd54159a48e285582824bad" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.934713 23545 raft_consensus.cc:697] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:51.935626 22889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e7b082dcf344fbf83e0ddec4b08b6de" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:51.936054 23600 raft_consensus.cc:3060] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:51.935635 23545 consensus_queue.cc:237] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:51.937552 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cabd9adbf414496bbaab3946b413d313" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.938948 23038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e638dc505d4b04b802e91b8863d833" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.926358 23553 raft_consensus.cc:2804] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:51.940269 23553 raft_consensus.cc:493] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:51.940658 23553 raft_consensus.cc:3060] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:51.945887 23600 raft_consensus.cc:515] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:51.948575 23600 leader_election.cc:290] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:51.949409 23037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e55b4e727be4089ba7454b662d03b9c" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.950445 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6a9c5a5fdc42e78dbbe6b411416c1a" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:51.952662 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.953511 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8be856e3d604f8d8a1c56a88ea18643" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.965226 23043 raft_consensus.cc:2468] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:51.915640 23036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02259962b5504debb700dd6b7c12915b" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.972622 22888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "745e7e3bed5b4d59949aeddcede543f3" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
W20260709 13:21:51.979954 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 53026.5us
I20260709 13:21:51.959751 23035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171526d2fe264ee8b1168192bec31c7a" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:51.964335 23039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "745e7e3bed5b4d59949aeddcede543f3" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:52.008023 22785 leader_election.cc:304] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
W20260709 13:21:52.015635 22860 leader_election.cc:343] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:52.006556 23008 leader_election.cc:304] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:52.022639 22858 leader_election.cc:304] T 7802e4d0fd394596b42b661c9eb4c7ce P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:51.986472 23042 raft_consensus.cc:2468] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:52.019666 23040 raft_consensus.cc:2393] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 in current term 1: Already voted for candidate 27745ef236cd494684f62a1269411a68 in this term.
I20260709 13:21:52.019464 23037 raft_consensus.cc:2393] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 in current term 1: Already voted for candidate 91f33bdbdd63438294071eb3776db679 in this term.
I20260709 13:21:51.953426 23539 raft_consensus.cc:515] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:52.027585 22785 leader_election.cc:304] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:52.021036 23397 raft_consensus.cc:515] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:51.973855 23593 consensus_queue.cc:1048] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.044s
I20260709 13:21:52.032428 23008 leader_election.cc:304] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:52.023699 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "087d64074b5943f0a7d789ead06f4783" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:52.041932 22785 leader_election.cc:304] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:52.029129 23544 raft_consensus.cc:2749] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:52.035996 23008 leader_election.cc:304] T be220d19020b428a8b1cc356f32ff5ad P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
W20260709 13:21:52.043972 22860 leader_election.cc:343] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:52.032740 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1e18beffe450f87ff940a4e9165ea" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:52.035589 23557 raft_consensus.cc:2749] T 7802e4d0fd394596b42b661c9eb4c7ce P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.047839 22860 leader_election.cc:304] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.053153 22858 leader_election.cc:304] T 10066ccf8d174f4a9dd69253613857b8 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.040892 22965 raft_consensus.cc:1240] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:52.031490 23603 consensus_queue.cc:1048] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:21:52.041852 23618 raft_consensus.cc:2804] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:52.062455 22858 leader_election.cc:304] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.031395 23551 consensus_queue.cc:1048] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 13:21:52.069981 22860 leader_election.cc:304] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.047209 23576 raft_consensus.cc:515] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:52.049412 23631 raft_consensus.cc:2804] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:52.071040 22814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955a6d8dbbe94ef9be9c1f69b10675d8" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.049492 23496 raft_consensus.cc:2749] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:21:52.055697 23506 raft_consensus.cc:2749] T be220d19020b428a8b1cc356f32ff5ad P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.072160 22932 leader_election.cc:304] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:52.056598 23600 raft_consensus.cc:2749] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.068799 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:52.072820 23590 raft_consensus.cc:2749] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.075331 22858 leader_election.cc:304] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:52.077414 23570 raft_consensus.cc:2749] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.066212 23618 raft_consensus.cc:493] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:52.082034 22815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086a7123d3ad43d79101a7c5de5158fe" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.076709 22860 leader_election.cc:304] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:52.068557 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "087d64074b5943f0a7d789ead06f4783" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.060369 22964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:52.095491 22932 leader_election.cc:304] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:52.079214 23562 raft_consensus.cc:2804] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:52.064786 23556 raft_consensus.cc:2749] T 10066ccf8d174f4a9dd69253613857b8 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.078372 23631 raft_consensus.cc:493] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:52.086107 23596 raft_consensus.cc:2804] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:52.100669 23567 raft_consensus.cc:2804] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:52.106582 23578 raft_consensus.cc:2804] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:52.105705 22967 raft_consensus.cc:2393] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 in current term 2: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:52.106596 23397 leader_election.cc:290] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:52.109599 23096 raft_consensus.cc:2804] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:52.118829 23096 raft_consensus.cc:493] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:52.122077 23096 raft_consensus.cc:3060] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.103129 23618 raft_consensus.cc:3060] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.114048 22860 leader_election.cc:304] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.106765 23587 raft_consensus.cc:515] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:52.124346 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.124951 23587 leader_election.cc:290] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:52.108206 23562 raft_consensus.cc:493] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:52.107911 23539 leader_election.cc:290] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:52.108677 23631 raft_consensus.cc:3060] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.114518 23567 raft_consensus.cc:493] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:52.112475 23596 raft_consensus.cc:493] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:52.129282 23084 tablet_bootstrap.cc:492] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:52.130927 23084 ts_tablet_manager.cc:1403] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.481s	user 0.034s	sys 0.053s
I20260709 13:21:52.114620 23578 raft_consensus.cc:493] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:52.133315 23578 raft_consensus.cc:3060] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.125809 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e55b4e727be4089ba7454b662d03b9c" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.134298 22815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abacd5d0d1e54db38a397e3c35c0cb81" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.135581 22860 leader_election.cc:304] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.137147 22860 leader_election.cc:304] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.138902 23557 raft_consensus.cc:2749] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.139847 23590 raft_consensus.cc:2749] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.145298 22813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505eab84b06f46ee9d336abf01353dee" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.108671 23553 raft_consensus.cc:515] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:52.146517 23578 raft_consensus.cc:515] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:52.146974 22813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b783c62bd54d4c8f4f1795b7409958" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.147694 23553 leader_election.cc:290] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
W20260709 13:21:52.149868 22858 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 54799.7us
I20260709 13:21:52.128623 23562 raft_consensus.cc:3060] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:21:52.153288 22860 leader_election.cc:343] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:52.158687 22858 leader_election.cc:304] T 8f631a34c02c4de8ae5bd70760fba972 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.131450 23567 raft_consensus.cc:3060] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.160048 22860 leader_election.cc:304] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.131750 23596 raft_consensus.cc:3060] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.164232 22858 leader_election.cc:304] T cabd9adbf414496bbaab3946b413d313 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:52.165673 22858 leader_election.cc:304] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:52.166944 22858 leader_election.cc:304] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.168593 23084 raft_consensus.cc:359] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:52.169533 23084 raft_consensus.cc:385] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:52.169848 23084 raft_consensus.cc:740] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:52.171288 22860 leader_election.cc:304] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:52.171950 22858 leader_election.cc:304] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.132725 22814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e638dc505d4b04b802e91b8863d833" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.170284 23096 raft_consensus.cc:515] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:52.173627 22858 leader_election.cc:304] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.170511 23084 consensus_queue.cc:260] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:52.138732 23556 raft_consensus.cc:2749] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.168792 22812 raft_consensus.cc:1240] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Rejecting Update request from peer 49cbf11e18ce4268a7fcca76872afb23 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:52.168573 23590 raft_consensus.cc:2749] T 8f631a34c02c4de8ae5bd70760fba972 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.170248 23035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd78248be3954373ad4fd60b8ae7593d" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:52.193718 23096 leader_election.cc:290] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:52.195842 23084 ts_tablet_manager.cc:1434] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.065s	user 0.007s	sys 0.000s
I20260709 13:21:52.171169 23557 raft_consensus.cc:2749] T cabd9adbf414496bbaab3946b413d313 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.199059 22813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:52.168826 23586 raft_consensus.cc:2749] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.170902 23568 raft_consensus.cc:2804] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:52.203289 23568 raft_consensus.cc:493] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:52.203714 23568 raft_consensus.cc:3060] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.184729 23565 raft_consensus.cc:2804] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:52.204950 23565 raft_consensus.cc:493] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:52.205325 23565 raft_consensus.cc:3060] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.176635 23570 raft_consensus.cc:2749] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.206406 23576 leader_election.cc:290] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:52.175943 23578 leader_election.cc:290] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:52.200462 23566 raft_consensus.cc:2804] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:52.199945 23036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:52.187669 22814 raft_consensus.cc:2468] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:52.200546 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b752c38e1b47a489a3d720801661d1" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.185010 23587 raft_consensus.cc:2749] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.186467 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e7b082dcf344fbf83e0ddec4b08b6de" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.213258 23568 raft_consensus.cc:515] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:52.173261 23037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e230e64c6984b72a2ba543f2a017e25" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:52.214032 23566 raft_consensus.cc:493] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:52.223439 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e638dc505d4b04b802e91b8863d833" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:52.222714 23039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630a5e60917545b3a8072fc8358207c5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:52.170737 23600 consensus_queue.cc:1059] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:21:52.191985 23618 raft_consensus.cc:515] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:52.223363 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
W20260709 13:21:52.236194 22934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 139501us
I20260709 13:21:52.226941 23566 raft_consensus.cc:3060] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.243216 22934 leader_election.cc:304] T 086a7123d3ad43d79101a7c5de5158fe P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.232566 23039 raft_consensus.cc:2468] T 630a5e60917545b3a8072fc8358207c5 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:52.248682 22934 leader_election.cc:304] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
W20260709 13:21:52.250737 22932 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 155309us
I20260709 13:21:52.252964 22932 leader_election.cc:304] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:52.253495 22934 leader_election.cc:304] T 50b783c62bd54d4c8f4f1795b7409958 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.257866 22934 leader_election.cc:304] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68
I20260709 13:21:52.247129 23397 raft_consensus.cc:2749] T 086a7123d3ad43d79101a7c5de5158fe P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.252846 23038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39611d2ff3524bc0840e4fd222d96a84" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:52.255270 23504 raft_consensus.cc:2749] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.275858 22934 leader_election.cc:304] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:52.260524 23576 raft_consensus.cc:2804] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:21:52.281265 22858 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 107714us
I20260709 13:21:52.281975 22858 leader_election.cc:304] T 1a4c294dbaeb4040b157ad72102a9e56 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:52.283309 22858 leader_election.cc:304] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:52.284125 23096 raft_consensus.cc:2749] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.262538 23036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78234c90229d4615ab9e2c2a811bd0f8" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:52.285305 22858 leader_election.cc:304] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:52.286926 22858 leader_election.cc:304] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:52.287995 22858 leader_election.cc:304] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.289062 22858 leader_election.cc:304] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.259865 23602 raft_consensus.cc:2749] T 50b783c62bd54d4c8f4f1795b7409958 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.290148 23500 raft_consensus.cc:2749] T 1a4c294dbaeb4040b157ad72102a9e56 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.290441 23039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086a7123d3ad43d79101a7c5de5158fe" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:52.290714 23096 raft_consensus.cc:2749] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.291241 23556 raft_consensus.cc:2804] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:52.291889 23605 raft_consensus.cc:2749] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.292454 23607 raft_consensus.cc:2804] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:52.292656 22858 leader_election.cc:304] T d1934129c8764c018495eff97d43a3b5 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.275450 23037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1934129c8764c018495eff97d43a3b5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:52.333964 23037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:52.334677 23037 raft_consensus.cc:2468] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:52.335587 23037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f631a34c02c4de8ae5bd70760fba972" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:52.340634 22783 leader_election.cc:304] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:52.342160 22783 leader_election.cc:304] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:52.342890 23600 raft_consensus.cc:2749] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.343322 22783 leader_election.cc:304] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:52.344192 23042 raft_consensus.cc:1240] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:52.344969 23603 raft_consensus.cc:2749] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.294577 23397 raft_consensus.cc:2749] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.298524 22934 leader_election.cc:304] T 8ad9b8807851437d9a22c0a33f567c15 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:52.300081 23096 raft_consensus.cc:2749] T d1934129c8764c018495eff97d43a3b5 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.294260 22965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:52.346390 23096 consensus_queue.cc:1059] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:52.346545 22783 leader_election.cc:304] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:52.307173 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:52.347332 23544 raft_consensus.cc:2749] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.311754 22895 raft_consensus.cc:1240] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Rejecting Update request from peer 8ef3887fb3874f8b81079e542d252944 for earlier term 2. Current term is 3. Ops: []
I20260709 13:21:52.347898 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "084ee2ebbf564ed583f84c1227d29bd4" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:52.348965 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171526d2fe264ee8b1168192bec31c7a" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:52.349056 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014651e80f9f44e599d2d8e4d308d2a6" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:52.310878 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:52.276438 23596 raft_consensus.cc:515] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:52.350090 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0957b803701e45648f538789ec62e507" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:52.280768 23640 raft_consensus.cc:2804] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:21:52.315377 23639 log.cc:927] Time spent T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:21:52.352178 22783 leader_election.cc:304] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:52.315769 23566 raft_consensus.cc:515] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:52.317684 23035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8b119f5c5844d1be439423b333ec60" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:52.318392 23618 leader_election.cc:290] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:52.320043 22814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78234c90229d4615ab9e2c2a811bd0f8" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
W20260709 13:21:52.377851 22934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 79384.5us
I20260709 13:21:52.321286 22813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.322286 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e09daa4f2dd474d96ae9c4dfa1a8758" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.299486 23607 raft_consensus.cc:493] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:52.394498 23607 raft_consensus.cc:3060] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.396322 23562 raft_consensus.cc:515] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:52.323146 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52d63f523d24195bc8127113598ff91" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.398151 22819 raft_consensus.cc:2393] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 in current term 1: Already voted for candidate 49cbf11e18ce4268a7fcca76872afb23 in this term.
I20260709 13:21:52.399235 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.400139 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e230e64c6984b72a2ba543f2a017e25" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:52.400596 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8b119f5c5844d1be439423b333ec60" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.324230 22969 raft_consensus.cc:1240] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:52.401444 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abacd5d0d1e54db38a397e3c35c0cb81" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.324579 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:52.324278 22886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "745e7e3bed5b4d59949aeddcede543f3" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:52.374583 22960 raft_consensus.cc:1240] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:52.325733 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78234c90229d4615ab9e2c2a811bd0f8" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:52.326452 22888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e09daa4f2dd474d96ae9c4dfa1a8758" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:52.404775 23567 raft_consensus.cc:515] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:52.406324 22815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8be856e3d604f8d8a1c56a88ea18643" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.407351 23567 leader_election.cc:290] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:52.330279 22889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52d63f523d24195bc8127113598ff91" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:52.407859 22815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e55b4e727be4089ba7454b662d03b9c" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.299576 23556 raft_consensus.cc:493] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:52.281599 23576 raft_consensus.cc:493] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:52.409861 23084 tablet_bootstrap.cc:492] T b1394978547a410199b4f798a10edb2d P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:52.341697 23593 raft_consensus.cc:2749] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.347381 23096 raft_consensus.cc:3055] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:52.304073 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:52.351123 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b154956cc17f4ba68f906daffbcedff6" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:52.312346 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94800e302e4641bb857f2d160f1f60ea" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:52.350986 23631 raft_consensus.cc:515] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:52.378199 22783 leader_election.cc:304] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:52.378706 22934 leader_election.cc:304] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.373880 23568 leader_election.cc:290] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:52.382593 23603 raft_consensus.cc:2804] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:52.360548 23640 raft_consensus.cc:493] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:52.383365 22887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1e18beffe450f87ff940a4e9165ea" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:52.383628 22965 raft_consensus.cc:3060] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:52.341069 23545 consensus_queue.cc:1048] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.043s
I20260709 13:21:52.414317 22886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:52.340752 22812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "084ee2ebbf564ed583f84c1227d29bd4" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:52.358258 23596 leader_election.cc:290] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:52.384361 23602 raft_consensus.cc:2749] T 8ad9b8807851437d9a22c0a33f567c15 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.384639 22968 raft_consensus.cc:1240] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:52.386842 23035 raft_consensus.cc:2376] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1. Re-sending same reply.
I20260709 13:21:52.387190 22967 raft_consensus.cc:1240] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:52.388813 22814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39611d2ff3524bc0840e4fd222d96a84" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.402729 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014651e80f9f44e599d2d8e4d308d2a6" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.330737 23570 consensus_queue.cc:1048] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:52.358798 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.359229 22811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050e9ef11f664c849a40cbb6acb25f19" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
W20260709 13:21:52.415832 22932 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 161747us
W20260709 13:21:52.438578 22934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 59948.7us
I20260709 13:21:52.439806 23096 raft_consensus.cc:740] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:52.443189 23603 raft_consensus.cc:493] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:52.474274 23096 consensus_queue.cc:260] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:52.477667 23096 raft_consensus.cc:3060] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.411260 22891 raft_consensus.cc:2393] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 in current term 2: Already voted for candidate 49cbf11e18ce4268a7fcca76872afb23 in this term.
I20260709 13:21:52.414675 23556 raft_consensus.cc:3060] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.444115 22785 leader_election.cc:304] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:52.444957 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014651e80f9f44e599d2d8e4d308d2a6" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:52.479763 23041 raft_consensus.cc:3060] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.393657 23565 raft_consensus.cc:515] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:52.409945 22813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd78248be3954373ad4fd60b8ae7593d" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.415640 23566 leader_election.cc:290] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:52.441615 23576 raft_consensus.cc:3060] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.445101 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.443892 23600 raft_consensus.cc:2749] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.494997 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "087d64074b5943f0a7d789ead06f4783" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:52.497357 23553 raft_consensus.cc:3055] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:52.497846 23553 raft_consensus.cc:740] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:52.500089 22889 raft_consensus.cc:1240] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:52.445529 23504 raft_consensus.cc:2804] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:52.473342 22932 leader_election.cc:304] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.478391 23603 raft_consensus.cc:3060] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.437389 23544 consensus_queue.cc:1048] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:21:52.473495 22934 leader_election.cc:304] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
W20260709 13:21:52.476364 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 52895.6us
I20260709 13:21:52.451246 23039 raft_consensus.cc:1240] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Rejecting Update request from peer 49cbf11e18ce4268a7fcca76872afb23 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:52.415127 23618 raft_consensus.cc:493] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:52.444670 23640 raft_consensus.cc:3060] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.519102 23618 raft_consensus.cc:515] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:52.480962 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b154956cc17f4ba68f906daffbcedff6" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.520110 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.485605 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6a9c5a5fdc42e78dbbe6b411416c1a" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:52.521224 23618 leader_election.cc:290] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:52.521433 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:52.507244 22715 catalog_manager.cc:5697] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:52.512756 22932 leader_election.cc:304] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
W20260709 13:21:52.440847 22783 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 62732us
I20260709 13:21:52.525206 22783 leader_election.cc:304] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
W20260709 13:21:52.517174 22934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 43755.2us
I20260709 13:21:52.526237 22934 leader_election.cc:304] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:52.482512 23551 raft_consensus.cc:2749] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:21:52.530987 22934 leader_election.cc:304] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:52.536263 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac141e204e3f4d3c84d6651893bdfc6f" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.536448 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b752c38e1b47a489a3d720801661d1" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
W20260709 13:21:52.537271 23645 log.cc:927] Time spent T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:21:52.523886 23546 consensus_queue.cc:1059] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:21:52.521246 23042 raft_consensus.cc:3060] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.538688 23034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:52.539291 22894 raft_consensus.cc:1240] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:52.539937 22783 leader_election.cc:304] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:52.540493 23544 raft_consensus.cc:2749] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.540890 23612 raft_consensus.cc:2749] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.541522 23035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968e16cef40146509d654274b9fd2687" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:52.531157 23569 raft_consensus.cc:2749] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.543789 22895 raft_consensus.cc:1240] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:21:52.537071 23641 raft_consensus.cc:2749] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:52.549805 23640 raft_consensus.cc:515] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:52.554013 22715 catalog_manager.cc:5697] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:52.558771 22934 leader_election.cc:304] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
W20260709 13:21:52.513883 23648 log.cc:927] Time spent T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:21:52.568081 23084 tablet_bootstrap.cc:654] T b1394978547a410199b4f798a10edb2d P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:52.515825 23579 raft_consensus.cc:2804] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:52.511168 23504 raft_consensus.cc:493] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:52.540995 23039 raft_consensus.cc:1240] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Rejecting Update request from peer 8ef3887fb3874f8b81079e542d252944 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:52.581017 23041 raft_consensus.cc:2468] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2.
I20260709 13:21:52.566190 23397 raft_consensus.cc:2749] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.592285 23603 raft_consensus.cc:515] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:52.596592 23641 raft_consensus.cc:2749] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.598618 23042 raft_consensus.cc:2468] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2.
I20260709 13:21:52.576916 23602 consensus_queue.cc:1048] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:21:52.600637 23565 leader_election.cc:290] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:52.601136 22785 leader_election.cc:304] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:52.619616 23631 leader_election.cc:290] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:52.620381 23546 raft_consensus.cc:2749] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.603677 23562 leader_election.cc:290] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:52.620560 23594 consensus_queue.cc:1059] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:52.621752 23594 raft_consensus.cc:3055] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:52.621977 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "084ee2ebbf564ed583f84c1227d29bd4" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:52.622303 23594 raft_consensus.cc:740] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:52.622568 22817 raft_consensus.cc:1240] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Rejecting Update request from peer 49cbf11e18ce4268a7fcca76872afb23 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:52.623283 23594 consensus_queue.cc:260] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:52.624622 23594 raft_consensus.cc:3060] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.625658 23576 raft_consensus.cc:515] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:52.626756 22966 raft_consensus.cc:1240] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:52.611917 22961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:52.627753 23607 raft_consensus.cc:515] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:52.628468 22961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "171526d2fe264ee8b1168192bec31c7a" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:52.610867 23579 raft_consensus.cc:493] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:52.623167 23556 raft_consensus.cc:515] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:52.606076 22811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "084ee2ebbf564ed583f84c1227d29bd4" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:52.609875 22860 leader_election.cc:304] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.606705 23593 raft_consensus.cc:2749] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.620704 22785 leader_election.cc:304] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:52.596660 23600 raft_consensus.cc:3055] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:52.633831 23600 raft_consensus.cc:740] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:52.634452 22785 leader_election.cc:304] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:52.634654 23600 consensus_queue.cc:260] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:52.632488 23579 raft_consensus.cc:3060] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.583752 22967 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0957b803701e45648f538789ec62e507" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:52.613689 23570 consensus_queue.cc:1059] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:21:52.633137 22888 raft_consensus.cc:1240] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:52.614178 22858 leader_election.cc:304] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
W20260709 13:21:52.606526 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 82919us
I20260709 13:21:52.640442 23593 raft_consensus.cc:2749] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.623822 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b154956cc17f4ba68f906daffbcedff6" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:52.613198 23504 raft_consensus.cc:3060] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.642359 23546 raft_consensus.cc:2749] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.629732 22961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "745e7e3bed5b4d59949aeddcede543f3" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:52.629720 23607 leader_election.cc:290] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:52.646526 23040 raft_consensus.cc:2393] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 in current term 2: Already voted for candidate 27745ef236cd494684f62a1269411a68 in this term.
W20260709 13:21:52.652165 22934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 93525.2us
I20260709 13:21:52.652182 22858 leader_election.cc:304] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:52.628479 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:52.642392 23562 consensus_queue.cc:1048] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:21:52.639504 23590 raft_consensus.cc:2749] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.680320 23008 leader_election.cc:304] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:52.688539 23008 leader_election.cc:304] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.684548 22934 leader_election.cc:304] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.677253 22820 raft_consensus.cc:1240] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Rejecting Update request from peer 49cbf11e18ce4268a7fcca76872afb23 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:52.690675 23008 leader_election.cc:304] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.691426 22934 leader_election.cc:304] T 171526d2fe264ee8b1168192bec31c7a P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:52.678237 23568 raft_consensus.cc:2804] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:52.695837 23570 consensus_queue.cc:1048] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:52.677253 22962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd78248be3954373ad4fd60b8ae7593d" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:52.691534 23506 raft_consensus.cc:2749] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.691730 23631 raft_consensus.cc:2749] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:52.694032 23602 raft_consensus.cc:2804] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:52.706321 23603 leader_election.cc:290] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:52.706535 22891 raft_consensus.cc:1240] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:52.682415 23496 consensus_queue.cc:1059] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:21:52.708387 23496 raft_consensus.cc:3055] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:52.708734 23496 raft_consensus.cc:740] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:52.646195 23553 consensus_queue.cc:260] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:52.702015 23008 leader_election.cc:304] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:52.686421 23607 raft_consensus.cc:2804] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:21:52.703419 23651 log.cc:927] Time spent T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:21:52.723716 23579 raft_consensus.cc:515] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:52.710597 22934 leader_election.cc:304] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:52.702987 23525 raft_consensus.cc:2749] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.706621 23602 raft_consensus.cc:493] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:52.725435 23602 raft_consensus.cc:3060] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.713577 22783 leader_election.cc:304] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:52.713891 23553 raft_consensus.cc:3060] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:21:52.714249 23544 consensus_queue.cc:1175] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "49cbf11e18ce4268a7fcca76872afb23" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 13:21:52.687937 23034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d217f95123f542e886aa049a608a88d4" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:52.715154 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abacd5d0d1e54db38a397e3c35c0cb81" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:52.712742 23545 consensus_queue.cc:1059] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:21:52.716291 23496 consensus_queue.cc:260] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:52.720166 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b154956cc17f4ba68f906daffbcedff6" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:52.710980 23397 raft_consensus.cc:2749] T 171526d2fe264ee8b1168192bec31c7a P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.720721 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0abfe80c1fa642ec8a0d075f75264816" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:52.691030 23600 raft_consensus.cc:3060] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.722365 22814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:52.705309 23568 raft_consensus.cc:697] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
W20260709 13:21:52.787747 22786 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:52.727671 22934 leader_election.cc:304] T 44841295bfd54159a48e285582824bad P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
W20260709 13:21:52.797403 22934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 69817.2us
I20260709 13:21:52.798151 22934 leader_election.cc:304] T 5e7b082dcf344fbf83e0ddec4b08b6de P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.799518 22934 leader_election.cc:304] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
W20260709 13:21:52.791610 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 51895.9us
I20260709 13:21:52.732439 23546 raft_consensus.cc:2749] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.732573 23556 leader_election.cc:290] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:52.732268 22783 leader_election.cc:304] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:52.733151 23631 raft_consensus.cc:2804] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:52.732556 22891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630a5e60917545b3a8072fc8358207c5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:52.731339 23036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630a5e60917545b3a8072fc8358207c5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:52.730297 23607 raft_consensus.cc:697] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:52.731010 23655 raft_consensus.cc:2749] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.875408 22890 raft_consensus.cc:1240] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Rejecting Update request from peer 27745ef236cd494684f62a1269411a68 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:52.725871 23579 leader_election.cc:290] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:52.876293 23607 consensus_queue.cc:237] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:52.790151 23034 raft_consensus.cc:3060] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.790243 23565 consensus_queue.cc:1059] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:21:52.880164 23557 raft_consensus.cc:3055] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:52.880549 23557 raft_consensus.cc:740] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:52.790122 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e09daa4f2dd474d96ae9c4dfa1a8758" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:52.881361 23557 consensus_queue.cc:260] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:52.811209 23660 raft_consensus.cc:2749] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.730429 23566 consensus_queue.cc:1059] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 13:21:52.802435 23008 leader_election.cc:304] T 78234c90229d4615ab9e2c2a811bd0f8 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.802079 23612 raft_consensus.cc:2749] T 5e7b082dcf344fbf83e0ddec4b08b6de P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.874580 23664 raft_consensus.cc:3055] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:52.873740 22893 raft_consensus.cc:1240] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Rejecting Update request from peer 27745ef236cd494684f62a1269411a68 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:52.895294 23664 raft_consensus.cc:740] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:52.876703 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e09daa4f2dd474d96ae9c4dfa1a8758" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
W20260709 13:21:52.795904 22785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 60976us
I20260709 13:21:52.800639 23596 raft_consensus.cc:2749] T 44841295bfd54159a48e285582824bad P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:21:52.893220 22783 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 161033us
I20260709 13:21:52.899204 23557 raft_consensus.cc:3060] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.899845 22783 leader_election.cc:304] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:52.796489 22820 raft_consensus.cc:2468] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 0.
W20260709 13:21:52.892855 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 90491.3us
I20260709 13:21:52.901067 23586 raft_consensus.cc:3055] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:52.901497 23586 raft_consensus.cc:740] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:52.902495 23545 raft_consensus.cc:2804] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:21:52.903055 22784 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:52.902407 23506 raft_consensus.cc:2749] T 78234c90229d4615ab9e2c2a811bd0f8 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.881778 23568 consensus_queue.cc:237] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:52.900017 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8b119f5c5844d1be439423b333ec60" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:52.905158 23008 leader_election.cc:304] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
W20260709 13:21:52.892781 23635 log.cc:927] Time spent T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:21:52.842098 23039 raft_consensus.cc:1240] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Rejecting Update request from peer 49cbf11e18ce4268a7fcca76872afb23 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:52.882853 23594 consensus_queue.cc:1059] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:21:52.828009 23631 raft_consensus.cc:493] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:52.921280 23008 leader_election.cc:304] T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.923679 23008 leader_election.cc:304] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.924916 23008 leader_election.cc:304] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
W20260709 13:21:52.928867 22860 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:52.896574 23504 raft_consensus.cc:515] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:52.956990 23008 leader_election.cc:304] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
W20260709 13:21:52.954906 22934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 155449us
I20260709 13:21:52.903005 23545 raft_consensus.cc:493] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:52.964262 22934 leader_election.cc:304] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.968665 23008 leader_election.cc:304] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.920277 23506 raft_consensus.cc:2749] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.936795 23650 raft_consensus.cc:2749] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.926628 23525 raft_consensus.cc:2749] T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:52.920102 22960 raft_consensus.cc:2468] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
I20260709 13:21:52.897850 23602 raft_consensus.cc:515] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
W20260709 13:21:52.992125 22785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 87550us
I20260709 13:21:53.009155 22895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968e16cef40146509d654274b9fd2687" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:53.014040 22860 leader_election.cc:304] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:52.937167 23631 raft_consensus.cc:3060] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.994339 22893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:52.991974 23612 raft_consensus.cc:2749] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.020397 22934 leader_election.cc:304] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:53.020630 23603 consensus_queue.cc:1059] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:21:53.021116 22785 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:53.022188 23603 raft_consensus.cc:2749] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.972092 23576 leader_election.cc:290] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:52.902419 23586 consensus_queue.cc:260] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
W20260709 13:21:52.966044 23652 log.cc:927] Time spent T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68: Append to log took a long time: real 0.281s	user 0.000s	sys 0.001s
I20260709 13:21:52.971876 23618 raft_consensus.cc:2749] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:52.966599 23545 raft_consensus.cc:3060] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:21:53.034255 22783 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 134501us
I20260709 13:21:52.969986 23540 raft_consensus.cc:2749] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.035661 22934 leader_election.cc:304] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:53.036913 22892 raft_consensus.cc:1240] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Rejecting Update request from peer 8ef3887fb3874f8b81079e542d252944 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:53.038067 23660 raft_consensus.cc:2749] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.038923 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b752c38e1b47a489a3d720801661d1" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:52.972797 23539 raft_consensus.cc:2749] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.051959 22934 leader_election.cc:304] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:53.053198 22934 leader_election.cc:304] T ac141e204e3f4d3c84d6651893bdfc6f P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:53.027362 23570 raft_consensus.cc:3060] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.012693 23579 consensus_queue.cc:1059] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:21:52.990787 23661 consensus_queue.cc:1059] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.064s
I20260709 13:21:53.075325 23666 raft_consensus.cc:3055] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:53.075836 23666 raft_consensus.cc:740] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:53.054236 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3139446c554b419fc28cad31653600" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:53.056557 23641 raft_consensus.cc:2749] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:21:53.067677 23596 raft_consensus.cc:2749] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.067194 23586 raft_consensus.cc:3060] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.095466 23084 tablet_bootstrap.cc:492] T b1394978547a410199b4f798a10edb2d P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:53.096150 23084 ts_tablet_manager.cc:1403] T b1394978547a410199b4f798a10edb2d P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.687s	user 0.029s	sys 0.009s
W20260709 13:21:53.095817 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 123186us
I20260709 13:21:53.078966 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:53.074471 23576 consensus_queue.cc:1059] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:21:53.079411 23666 consensus_queue.cc:260] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:53.159819 22812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:53.161396 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8be856e3d604f8d8a1c56a88ea18643" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:53.165611 23602 leader_election.cc:290] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:53.166734 22888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955a6d8dbbe94ef9be9c1f69b10675d8" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:53.169775 22965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d217f95123f542e886aa049a608a88d4" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:53.174507 23666 raft_consensus.cc:3060] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.175465 23034 raft_consensus.cc:2468] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2.
W20260709 13:21:53.179281 22860 leader_election.cc:343] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:53.227280 23504 leader_election.cc:290] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:53.285759 23084 raft_consensus.cc:359] T b1394978547a410199b4f798a10edb2d P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:53.286695 23084 raft_consensus.cc:385] T b1394978547a410199b4f798a10edb2d P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:53.286998 23084 raft_consensus.cc:740] T b1394978547a410199b4f798a10edb2d P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
W20260709 13:21:53.288056 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 182449us
I20260709 13:21:53.287977 23084 consensus_queue.cc:260] T b1394978547a410199b4f798a10edb2d P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:53.290954 23008 leader_election.cc:304] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:53.227988 22811 raft_consensus.cc:2468] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2.
I20260709 13:21:53.299589 22858 leader_election.cc:304] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:53.304503 22858 leader_election.cc:304] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:53.306380 23084 ts_tablet_manager.cc:1434] T b1394978547a410199b4f798a10edb2d P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.210s	user 0.008s	sys 0.000s
W20260709 13:21:53.335685 22932 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.006s	sys 0.023s
I20260709 13:21:53.340939 22815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014651e80f9f44e599d2d8e4d308d2a6" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
W20260709 13:21:53.343797 22932 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.007s	sys 0.024s
W20260709 13:21:53.344192 22932 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 771729us
I20260709 13:21:53.345288 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39611d2ff3524bc0840e4fd222d96a84" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:53.346244 23037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e638dc505d4b04b802e91b8863d833" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:53.349541 23037 raft_consensus.cc:1240] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Rejecting Update request from peer 8ef3887fb3874f8b81079e542d252944 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:21:53.351162 23084 tablet_bootstrap.cc:492] T 0abfe80c1fa642ec8a0d075f75264816 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:53.353279 23037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:53.355336 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6a9c5a5fdc42e78dbbe6b411416c1a" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:53.355824 23037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8be856e3d604f8d8a1c56a88ea18643" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
W20260709 13:21:53.330206 22783 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 290739us
I20260709 13:21:53.294531 23540 raft_consensus.cc:2804] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:53.295046 22888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505eab84b06f46ee9d336abf01353dee" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:53.372076 22887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3139446c554b419fc28cad31653600" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:53.327234 23565 raft_consensus.cc:2749] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:21:53.368561 22785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 265184us
I20260709 13:21:53.367539 23590 raft_consensus.cc:2749] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:21:53.385085 22858 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 54026.2us
I20260709 13:21:53.385052 22783 leader_election.cc:304] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:53.369560 23540 raft_consensus.cc:493] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:53.405818 22858 leader_election.cc:304] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:53.414793 23008 leader_election.cc:304] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:53.379235 23603 consensus_queue.cc:1048] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.035s
I20260709 13:21:53.405015 23540 raft_consensus.cc:3060] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:53.423800 23594 consensus_queue.cc:1059] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:53.419579 23008 leader_election.cc:304] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:53.407740 23546 raft_consensus.cc:493] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:53.412175 23593 raft_consensus.cc:2749] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.417225 23640 leader_election.cc:290] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:53.418233 23556 raft_consensus.cc:2804] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:53.413283 23565 consensus_queue.cc:1059] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:21:53.456385 22932 leader_election.cc:304] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:53.431031 22969 raft_consensus.cc:1240] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:53.454991 23008 leader_election.cc:304] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:53.459218 22932 leader_election.cc:304] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:53.416255 23650 consensus_queue.cc:1059] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
W20260709 13:21:53.462209 22783 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 49639.5us
I20260709 13:21:53.415733 23621 consensus_queue.cc:1059] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:21:53.450109 22812 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505eab84b06f46ee9d336abf01353dee" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:53.474960 23008 leader_election.cc:304] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
W20260709 13:21:53.492815 23676 log.cc:927] Time spent T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:21:53.448588 23546 raft_consensus.cc:515] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
W20260709 13:21:53.498663 23687 log.cc:927] Time spent T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 13:21:53.504551 23683 log.cc:927] Time spent T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:21:53.501466 23660 raft_consensus.cc:2749] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.505882 23546 leader_election.cc:290] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:53.507129 22890 raft_consensus.cc:1240] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Rejecting Update request from peer 8ef3887fb3874f8b81079e542d252944 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:53.507620 23570 raft_consensus.cc:2749] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.509346 23585 raft_consensus.cc:3055] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:53.509871 23585 raft_consensus.cc:740] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:53.511152 23621 raft_consensus.cc:3055] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:53.510825 23585 consensus_queue.cc:260] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:53.511577 23621 raft_consensus.cc:740] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:53.512065 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e55b4e727be4089ba7454b662d03b9c" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:53.512492 23621 consensus_queue.cc:260] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:53.462522 23602 raft_consensus.cc:2749] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:21:53.518705 23557 consensus_meta.cc:287] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23: Time spent flushing consensus metadata: real 0.619s	user 0.010s	sys 0.000s
I20260709 13:21:53.464406 23691 raft_consensus.cc:2749] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.477602 23506 raft_consensus.cc:2749] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.469203 23631 raft_consensus.cc:515] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:53.524649 23631 leader_election.cc:290] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:53.525068 23544 raft_consensus.cc:3055] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:53.525503 23544 raft_consensus.cc:740] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:53.526304 23544 consensus_queue.cc:260] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:53.527287 23544 raft_consensus.cc:3060] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.528208 23034 raft_consensus.cc:1240] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Rejecting Update request from peer 49cbf11e18ce4268a7fcca76872afb23 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:53.531374 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0abfe80c1fa642ec8a0d075f75264816" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:53.532047 22963 raft_consensus.cc:3060] T b1394978547a410199b4f798a10edb2d P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:53.532830 22969 raft_consensus.cc:1240] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:53.466209 23539 raft_consensus.cc:2749] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.534454 23553 raft_consensus.cc:3055] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:53.534933 23553 raft_consensus.cc:740] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:53.513845 23621 raft_consensus.cc:3060] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.535859 23553 consensus_queue.cc:260] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:53.537264 23553 raft_consensus.cc:3060] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.536338 23540 raft_consensus.cc:515] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:53.538028 22887 raft_consensus.cc:2468] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 2.
I20260709 13:21:53.545168 22932 leader_election.cc:304] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
W20260709 13:21:53.496137 23675 log.cc:927] Time spent T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
I20260709 13:21:53.496459 23041 raft_consensus.cc:1275] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:21:53.465112 23682 raft_consensus.cc:3055] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:53.497669 23545 raft_consensus.cc:515] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:53.468235 22817 raft_consensus.cc:2468] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 2.
I20260709 13:21:53.495111 22814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955a6d8dbbe94ef9be9c1f69b10675d8" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:53.557554 23525 raft_consensus.cc:2749] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:21:53.548448 23096 consensus_queue.cc:1059] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:53.557601 23568 consensus_queue.cc:1048] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:53.559005 23096 raft_consensus.cc:3055] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:53.559337 23096 raft_consensus.cc:740] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:53.560158 23096 consensus_queue.cc:260] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:53.561131 23096 raft_consensus.cc:3060] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.562112 23682 raft_consensus.cc:740] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:53.562638 22968 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52354 (request call id 133) took 1003 ms. Trace:
I20260709 13:21:53.563568 22820 raft_consensus.cc:1240] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Rejecting Update request from peer 8ef3887fb3874f8b81079e542d252944 for earlier term 1. Current term is 2. Ops: []
W20260709 13:21:53.549254 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 74392us
I20260709 13:21:53.569680 23084 tablet_bootstrap.cc:654] T 0abfe80c1fa642ec8a0d075f75264816 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:53.569824 23008 leader_election.cc:304] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:53.551602 22932 leader_election.cc:304] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:53.569301 23556 raft_consensus.cc:697] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:53.567694 23660 raft_consensus.cc:2749] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.581427 23008 leader_election.cc:304] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:53.571195 23579 raft_consensus.cc:2749] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.595466 23696 raft_consensus.cc:2749] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.564860 23578 consensus_queue.cc:1059] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:21:53.583789 23008 leader_election.cc:304] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:53.584169 23525 raft_consensus.cc:2749] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.563071 22968 rpcz_store.cc:276] 0709 13:21:52.559419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:52.565761 (+  6342us) service_pool.cc:224] Handling call
0709 13:21:53.511884 (+946123us) spinlock_profiling.cc:243] Waited 8.93 ms on lock 0x7b5800179198. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99d10a0 00007f5ce99d0984 00007f5ce2f4b6d4 00007f5ce2f4b662
0709 13:21:53.562575 (+ 50691us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20529408}
I20260709 13:21:53.562975 23682 consensus_queue.cc:260] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:53.599989 23008 leader_election.cc:304] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:53.569368 23602 consensus_queue.cc:1059] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:21:53.608752 22962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94800e302e4641bb857f2d160f1f60ea" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:53.609447 22962 raft_consensus.cc:3060] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.606216 23008 leader_election.cc:304] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:53.612941 23008 leader_election.cc:304] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
W20260709 13:21:53.614060 23008 leader_election.cc:343] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:53.615778 23696 raft_consensus.cc:2749] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.610342 23672 raft_consensus.cc:2749] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.618127 23545 leader_election.cc:290] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:53.618899 23525 raft_consensus.cc:2804] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:53.619369 23525 raft_consensus.cc:493] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:53.619725 23525 raft_consensus.cc:3060] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.604476 23663 raft_consensus.cc:2749] T ac141e204e3f4d3c84d6651893bdfc6f P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:53.620927 23039 raft_consensus.cc:1275] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:21:53.623072 22964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e230e64c6984b72a2ba543f2a017e25" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:53.623883 22964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52354 (request call id 142) took 1052 ms. Trace:
I20260709 13:21:53.625371 23557 consensus_queue.cc:1048] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:53.615410 23008 leader_election.cc:304] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:53.634310 22894 raft_consensus.cc:1240] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:53.634441 23602 raft_consensus.cc:3055] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:53.637388 23602 raft_consensus.cc:740] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:21:53.624197 22964 rpcz_store.cc:276] 0709 13:21:52.571245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:52.601484 (+ 30239us) service_pool.cc:224] Handling call
0709 13:21:53.622893 (+1021409us) spinlock_profiling.cc:243] Waited 6.22 ms on lock 0x7b5800179198. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99d10a0 00007f5ce99d0984 00007f5ce2f4b6d4 00007f5ce2f4b662
0709 13:21:53.623826 (+   933us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14304512}
I20260709 13:21:53.604962 23674 raft_consensus.cc:2749] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.635881 22888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02259962b5504debb700dd6b7c12915b" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:53.617520 23662 consensus_queue.cc:1059] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:21:53.641613 23594 consensus_queue.cc:1059] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:21:53.656818 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:53.657231 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02259962b5504debb700dd6b7c12915b" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:53.655234 22886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac141e204e3f4d3c84d6651893bdfc6f" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:53.642081 23545 consensus_queue.cc:1059] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 13:21:53.658798 23629 raft_consensus.cc:3055] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:53.630343 23672 raft_consensus.cc:2749] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.638867 23693 raft_consensus.cc:3055] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:53.639653 23540 leader_election.cc:290] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:53.654209 22969 raft_consensus.cc:1240] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:53.659694 22962 raft_consensus.cc:2468] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2.
I20260709 13:21:53.670024 22962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:53.640529 23556 consensus_queue.cc:237] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:53.670626 22963 raft_consensus.cc:1240] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Rejecting Update request from peer 27745ef236cd494684f62a1269411a68 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:53.666783 23546 raft_consensus.cc:3055] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:53.671739 23546 raft_consensus.cc:740] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:53.672506 22963 raft_consensus.cc:1240] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Rejecting Update request from peer 27745ef236cd494684f62a1269411a68 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:53.674275 23545 raft_consensus.cc:3055] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:53.672566 23546 consensus_queue.cc:260] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:53.676890 22963 raft_consensus.cc:1240] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:53.678853 22963 raft_consensus.cc:1240] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Rejecting Update request from peer 27745ef236cd494684f62a1269411a68 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:53.666509 22964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac141e204e3f4d3c84d6651893bdfc6f" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:53.680186 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0abfe80c1fa642ec8a0d075f75264816" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:53.666692 23693 raft_consensus.cc:740] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:21:53.683223 23693 consensus_queue.cc:260] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:53.652441 23602 consensus_queue.cc:260] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:53.684674 23693 raft_consensus.cc:3060] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.685014 23602 raft_consensus.cc:3060] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.696337 22818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49352 (request call id 139) took 1345 ms. Trace:
I20260709 13:21:53.673609 23594 raft_consensus.cc:3055] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:53.698671 23594 raft_consensus.cc:740] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:53.665995 23629 raft_consensus.cc:740] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:53.699474 23594 consensus_queue.cc:260] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:53.699760 23629 consensus_queue.cc:260] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:53.700628 23594 raft_consensus.cc:3060] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.701040 23629 raft_consensus.cc:3060] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.704227 23525 raft_consensus.cc:515] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:53.705446 22819 raft_consensus.cc:2468] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 2.
I20260709 13:21:53.696805 22818 rpcz_store.cc:276] 0709 13:21:52.345171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:52.565960 (+220789us) service_pool.cc:224] Handling call
0709 13:21:52.566315 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:52.566534 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:21:52.793613 (+227079us) log.cc:844] Serialized 14 byte log entry
0709 13:21:53.685901 (+892288us) spinlock_profiling.cc:243] Waited 4.32 ms on lock 0x7b18003ae760. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7f35b1 00007f5ce45b34bc 00007f5ce190148b 00007f5cea75bab9 00007f5ce191d69b 00007f5ce191b93f 00007f5ce191b455 00007f5ce4a03060 00007f5ce4a15716 00007f5ce4a26287
0709 13:21:53.690193 (+  4292us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:53.690251 (+    58us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:21:53.690299 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:53.690371 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:53.690501 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1814,"prepare.queue_time_us":400590,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":1392,"raft.queue_time_us":98933,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":9943936,"thread_start_us":443023,"threads_started":2}
I20260709 13:21:53.707624 23612 raft_consensus.cc:3055] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:53.708050 23612 raft_consensus.cc:740] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:21:53.712679 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:53.712849 23008 leader_election.cc:304] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:53.713907 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0abfe80c1fa642ec8a0d075f75264816" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:53.714385 23036 raft_consensus.cc:1240] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:21:53.714560 22818 raft_consensus.cc:3060] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:53.693008 23545 raft_consensus.cc:740] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:53.716015 23038 raft_consensus.cc:1240] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:21:53.716668 23506 raft_consensus.cc:2749] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.718364 23641 raft_consensus.cc:3055] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:53.718753 23641 raft_consensus.cc:740] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
W20260709 13:21:53.723788 22785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 55692.2us
I20260709 13:21:53.722658 22818 raft_consensus.cc:2468] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
I20260709 13:21:53.719152 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8b119f5c5844d1be439423b333ec60" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:53.708837 23612 consensus_queue.cc:260] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:53.716053 23545 consensus_queue.cc:260] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:53.728210 23612 raft_consensus.cc:3060] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:53.728157 23641 consensus_queue.cc:260] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:53.729292 23641 raft_consensus.cc:3060] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.731791 23690 consensus_queue.cc:1059] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:53.733837 22813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e638dc505d4b04b802e91b8863d833" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:53.728719 23545 raft_consensus.cc:3060] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.753077 23525 leader_election.cc:290] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:53.757333 23664 consensus_queue.cc:260] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:53.758819 23664 raft_consensus.cc:3060] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.762503 23496 raft_consensus.cc:3060] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.770066 23525 consensus_queue.cc:1059] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:53.770902 23585 raft_consensus.cc:3060] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.771311 23525 raft_consensus.cc:3055] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:53.771716 23525 raft_consensus.cc:740] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:53.772666 23525 consensus_queue.cc:260] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
W20260709 13:21:53.770402 23008 leader_election.cc:343] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:53.796247 22783 leader_election.cc:304] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:53.798825 23008 leader_election.cc:304] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
W20260709 13:21:53.775126 23685 log.cc:927] Time spent T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679: Append to log took a long time: real 0.507s	user 0.001s	sys 0.002s
W20260709 13:21:53.794425 22861 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:53.807499 22785 leader_election.cc:304] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:53.795078 23525 raft_consensus.cc:3060] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.801803 23594 raft_consensus.cc:2749] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.797297 23539 consensus_queue.cc:1059] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:21:53.805490 23701 raft_consensus.cc:2804] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:53.810029 23553 raft_consensus.cc:2749] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.831022 23556 raft_consensus.cc:493] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:53.831329 22811 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49352 (request call id 143) took 1227 ms. Trace:
I20260709 13:21:53.832041 23556 raft_consensus.cc:515] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:53.834524 23556 leader_election.cc:290] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:53.837054 23701 raft_consensus.cc:697] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:53.837221 23566 raft_consensus.cc:493] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:53.837790 23566 raft_consensus.cc:515] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:53.838418 22815 raft_consensus.cc:2410] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
W20260709 13:21:53.839850 22815 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49352 (request call id 142) took 1261 ms (client timeout 1558 ms). Trace:
W20260709 13:21:53.840278 22815 rpcz_store.cc:269] 0709 13:21:52.603552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:52.609989 (+  6437us) service_pool.cc:224] Handling call
0709 13:21:53.340749 (+730760us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b5800991598. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99d10a0 00007f5ce99d0984 00007f5ce2f4b6d4 00007f5ce2f4b662
0709 13:21:53.839562 (+498813us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b54007e2a54. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:21:53.839790 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1174747392}
I20260709 13:21:53.842033 22814 raft_consensus.cc:1275] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 13:21:53.842482 22814 pending_rounds.cc:85] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679: Aborting all ops after (but not including) 0
I20260709 13:21:53.842741 22814 pending_rounds.cc:107] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:21:53.844696 22814 raft_consensus.cc:2889] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:53.802948 23670 consensus_queue.cc:1059] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:21:53.845777 23037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0957b803701e45648f538789ec62e507" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:53.845996 23701 consensus_queue.cc:237] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:53.831745 22811 rpcz_store.cc:276] 0709 13:21:52.603989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:52.605736 (+  1747us) service_pool.cc:224] Handling call
0709 13:21:53.831114 (+1225378us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b54001bf8d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:21:53.831235 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":372683648}
I20260709 13:21:53.847682 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a064be2ec640a4a8283957d0b9b98c" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:53.848239 22813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8b119f5c5844d1be439423b333ec60" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:53.820145 23594 consensus_queue.cc:1059] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:21:53.850024 23594 raft_consensus.cc:3055] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:53.848361 23041 raft_consensus.cc:2468] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2.
I20260709 13:21:53.853678 22858 leader_election.cc:304] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:53.853874 22888 raft_consensus.cc:2468] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 2.
I20260709 13:21:53.839891 23566 leader_election.cc:290] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:53.854537 23556 raft_consensus.cc:2804] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:21:53.854993 23556 raft_consensus.cc:493] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:53.855365 23556 raft_consensus.cc:3060] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:53.860059 23594 raft_consensus.cc:740] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:53.861001 23594 consensus_queue.cc:260] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:53.862416 23594 raft_consensus.cc:3060] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.863087 23008 leader_election.cc:304] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:53.863236 23556 raft_consensus.cc:515] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:53.865399 23556 leader_election.cc:290] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:53.866006 23556 raft_consensus.cc:493] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:53.866425 23556 raft_consensus.cc:515] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:53.866570 23037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a064be2ec640a4a8283957d0b9b98c" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:53.867347 23037 raft_consensus.cc:3060] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:21:53.867887 23678 log.cc:927] Time spent T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23: Append to log took a long time: real 0.589s	user 0.000s	sys 0.001s
I20260709 13:21:53.868450 23556 leader_election.cc:290] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:53.866798 23672 raft_consensus.cc:2804] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:53.875770 23672 raft_consensus.cc:697] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [term 2 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:53.876345 23038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36860 (request call id 138) took 1148 ms. Trace:
I20260709 13:21:53.876883 23038 rpcz_store.cc:276] 0709 13:21:52.728019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:52.735203 (+  7184us) service_pool.cc:224] Handling call
0709 13:21:53.527510 (+792307us) spinlock_profiling.cc:243] Waited 1.03 ms on lock 0x7b5800190898. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce17a6db1 00007f5ce18e6b71 00007f5ce18e6a22 00007f5cea7ffc09 00007f5ce9959ee0 00007f5ce9a230ab 00007f5ce9a23131 00007f5ce99e4fc0 00007f5ce99cf929 00007f5ce99cf3d2 00007f5ce2f46bd4
0709 13:21:53.715759 (+188249us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:53.716445 (+   686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:53.876088 (+159643us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54009763d0. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0a8ad 00007f5ce99cf419 00007f5ce2f46bd4 00007f5ce2f46b62 00007f5ce2f46a91 00007f5ce2f46a0c 00007f5ce2f45cc2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:21:53.876247 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434823296}
I20260709 13:21:53.878939 23546 raft_consensus.cc:3060] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.876611 23672 consensus_queue.cc:237] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
W20260709 13:21:53.886932 22860 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 112050us
I20260709 13:21:53.887651 23084 tablet_bootstrap.cc:492] T 0abfe80c1fa642ec8a0d075f75264816 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:53.888254 23084 ts_tablet_manager.cc:1403] T 0abfe80c1fa642ec8a0d075f75264816 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.537s	user 0.020s	sys 0.008s
I20260709 13:21:53.891161 23084 raft_consensus.cc:359] T 0abfe80c1fa642ec8a0d075f75264816 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:53.891837 23084 raft_consensus.cc:385] T 0abfe80c1fa642ec8a0d075f75264816 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:53.892109 23084 raft_consensus.cc:740] T 0abfe80c1fa642ec8a0d075f75264816 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:53.893824 23596 raft_consensus.cc:493] T b1394978547a410199b4f798a10edb2d P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:53.894260 23674 raft_consensus.cc:3055] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:53.894912 23674 raft_consensus.cc:740] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:53.893847 22934 leader_election.cc:304] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:53.895797 23674 consensus_queue.cc:260] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:53.896672 22888 raft_consensus.cc:1240] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Rejecting Update request from peer 8ef3887fb3874f8b81079e542d252944 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:53.894927 23596 raft_consensus.cc:515] T b1394978547a410199b4f798a10edb2d P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:53.897285 23674 raft_consensus.cc:3060] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.898797 23596 leader_election.cc:290] T b1394978547a410199b4f798a10edb2d P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:53.896519 23612 raft_consensus.cc:2749] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.900701 22934 leader_election.cc:304] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:53.894981 22887 raft_consensus.cc:1240] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Rejecting Update request from peer 8ef3887fb3874f8b81079e542d252944 for earlier term 2. Current term is 3. Ops: []
I20260709 13:21:53.903420 23682 raft_consensus.cc:3060] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.903659 23612 raft_consensus.cc:2749] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.903491 22934 leader_election.cc:304] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:53.905498 23612 raft_consensus.cc:2749] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.906709 22934 leader_election.cc:304] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:53.908174 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a064be2ec640a4a8283957d0b9b98c" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:53.910852 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a064be2ec640a4a8283957d0b9b98c" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:53.912762 23612 raft_consensus.cc:2804] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:53.916574 23612 raft_consensus.cc:697] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [term 2 LEADER]: Becoming Leader. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:21:53.917266 23690 raft_consensus.cc:3055] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:53.917721 23690 raft_consensus.cc:740] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:53.918599 23690 consensus_queue.cc:260] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:53.919582 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ea5c83aada45d3a98027069e96658c" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:53.919935 23690 raft_consensus.cc:3060] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.918604 23641 consensus_queue.cc:1059] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:21:53.922293 23596 raft_consensus.cc:3055] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:53.922559 22715 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50724 (request call id 64) took 1617 ms. Trace:
I20260709 13:21:53.925838 22715 rpcz_store.cc:276] 0709 13:21:52.304924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:52.305125 (+   201us) service_pool.cc:224] Handling call
0709 13:21:53.922465 (+1617340us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:52.716751 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:52.717305 (+   554us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:52.717336 (+    31us) write_op.cc:435] Acquired schema lock
0709 13:21:52.717357 (+    21us) tablet.cc:662] Decoding operations
0709 13:21:52.717738 (+   381us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:52.717794 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:52.717817 (+    23us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:21:52.718163 (+   346us) tablet.cc:701] Row locks acquired
0709 13:21:52.718177 (+    14us) write_op.cc:260] PREPARE: finished
0709 13:21:52.718325 (+   148us) write_op.cc:270] Start()
0709 13:21:52.718428 (+   103us) write_op.cc:276] Timestamp: P: 1783603312718281 usec, L: 0
0709 13:21:52.718445 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:21:52.719090 (+   645us) log.cc:844] Serialized 922 byte log entry
0709 13:21:53.333053 (+613963us) op_driver.cc:464] REPLICATION: finished
0709 13:21:53.457726 (+124673us) write_op.cc:301] APPLY: starting
0709 13:21:53.463492 (+  5766us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:53.475338 (+ 11846us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:53.491346 (+ 16008us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:53.491721 (+   375us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:53.491891 (+   170us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:21:53.491935 (+    44us) write_op.cc:312] APPLY: finished
0709 13:21:53.492131 (+   196us) log.cc:844] Serialized 28 byte log entry
0709 13:21:53.492291 (+   160us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:53.492380 (+    89us) write_op.cc:454] Released schema lock
0709 13:21:53.492418 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:53.492814 (+   396us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:53.492841 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":112868,"apply.run_cpu_time_us":1941,"apply.run_wall_time_us":40279,"num_ops":2,"prepare.queue_time_us":106027,"prepare.run_cpu_time_us":33452,"prepare.run_wall_time_us":292694,"raft.queue_time_us":294814,"raft.run_cpu_time_us":22655,"raft.run_wall_time_us":74525,"replication_time_us":614425,"thread_start_us":631668,"threads_started":4,"wal-append.queue_time_us":317090}]]}
I20260709 13:21:53.923509 22934 leader_election.cc:304] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68
I20260709 13:21:53.924731 22718 catalog_manager.cc:5697] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ef3887fb3874f8b81079e542d252944 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "8ef3887fb3874f8b81079e542d252944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:53.922760 23596 raft_consensus.cc:740] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:21:53.942493 23693 raft_consensus.cc:2804] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:53.917572 23612 consensus_queue.cc:237] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:53.929760 22886 raft_consensus.cc:1240] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 1. Current term is 2. Ops: []
I20260709 13:21:53.930171 23084 consensus_queue.cc:260] T 0abfe80c1fa642ec8a0d075f75264816 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:53.947938 23084 ts_tablet_manager.cc:1434] T 0abfe80c1fa642ec8a0d075f75264816 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.059s	user 0.006s	sys 0.000s
I20260709 13:21:53.948737 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0957b803701e45648f538789ec62e507" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:53.948966 23084 tablet_bootstrap.cc:492] T d217f95123f542e886aa049a608a88d4 P 8ef3887fb3874f8b81079e542d252944: Bootstrap starting.
I20260709 13:21:53.951417 22961 raft_consensus.cc:1275] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 13:21:53.951830 22961 pending_rounds.cc:85] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944: Aborting all ops after (but not including) 0
I20260709 13:21:53.952055 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ea5c83aada45d3a98027069e96658c" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:53.952737 22966 raft_consensus.cc:2468] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2.
I20260709 13:21:53.942870 23596 consensus_queue.cc:260] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:53.953733 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94800e302e4641bb857f2d160f1f60ea" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:53.939669 23037 raft_consensus.cc:2468] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 3.
I20260709 13:21:53.940244 22860 leader_election.cc:304] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:53.954416 22963 raft_consensus.cc:2376] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2. Re-sending same reply.
I20260709 13:21:53.940348 23566 consensus_queue.cc:1048] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:53.954906 23596 raft_consensus.cc:3060] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:53.956008 23566 raft_consensus.cc:2749] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:21:53.957180 22860 leader_election.cc:304] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68
I20260709 13:21:53.959685 22858 leader_election.cc:304] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:53.961234 23699 raft_consensus.cc:2804] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:53.961800 23699 raft_consensus.cc:697] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:53.962668 23699 consensus_queue.cc:237] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:53.965026 23566 raft_consensus.cc:2804] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:53.965543 23566 raft_consensus.cc:697] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 3 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:53.961292 23096 consensus_queue.cc:1048] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:21:53.966361 23566 consensus_queue.cc:237] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:53.970095 22886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1394978547a410199b4f798a10edb2d" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:53.973107 22860 leader_election.cc:304] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:53.975361 23096 raft_consensus.cc:2749] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:21:53.969844 22811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1394978547a410199b4f798a10edb2d" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:53.941200 23546 raft_consensus.cc:493] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:53.942999 23693 raft_consensus.cc:697] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [term 2 LEADER]: Becoming Leader. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:21:53.952064 22961 pending_rounds.cc:107] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:21:53.931504 23008 leader_election.cc:304] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:53.982062 22860 leader_election.cc:304] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 91f33bdbdd63438294071eb3776db679
W20260709 13:21:53.988191 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 56778.8us
I20260709 13:21:53.988168 22934 leader_election.cc:304] T b1394978547a410199b4f798a10edb2d P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:53.987460 22961 raft_consensus.cc:2889] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:53.995074 22860 leader_election.cc:304] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68
I20260709 13:21:53.981101 23084 tablet_bootstrap.cc:654] T d217f95123f542e886aa049a608a88d4 P 8ef3887fb3874f8b81079e542d252944: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:54.001767 23096 raft_consensus.cc:2804] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:54.002437 23096 raft_consensus.cc:697] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:54.010550 23096 consensus_queue.cc:237] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:53.989765 23672 raft_consensus.cc:2749] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:53.999087 23569 raft_consensus.cc:2749] T b1394978547a410199b4f798a10edb2d P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:54.000470 23556 raft_consensus.cc:2804] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:21:54.025015 23556 raft_consensus.cc:493] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:54.025456 23556 raft_consensus.cc:3060] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:54.033632 23693 consensus_queue.cc:237] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:53.989102 23546 raft_consensus.cc:515] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:54.067693 23569 raft_consensus.cc:493] T 0abfe80c1fa642ec8a0d075f75264816 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:54.069001 23690 raft_consensus.cc:493] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:54.069227 23699 consensus_queue.cc:1048] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
W20260709 13:21:54.085005 22766 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:54.081704 23569 raft_consensus.cc:515] T 0abfe80c1fa642ec8a0d075f75264816 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:54.088044 23569 leader_election.cc:290] T 0abfe80c1fa642ec8a0d075f75264816 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:54.098984 22814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0abfe80c1fa642ec8a0d075f75264816" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:54.093116 23586 raft_consensus.cc:493] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:54.083822 23690 raft_consensus.cc:515] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:54.110054 22814 raft_consensus.cc:2393] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 in current term 1: Already voted for candidate 27745ef236cd494684f62a1269411a68 in this term.
I20260709 13:21:54.107722 23038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0abfe80c1fa642ec8a0d075f75264816" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:54.108285 23556 raft_consensus.cc:515] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:54.116711 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:54.110109 23586 raft_consensus.cc:515] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:54.128067 22932 leader_election.cc:304] T 0abfe80c1fa642ec8a0d075f75264816 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:54.129112 23556 leader_election.cc:290] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:54.130113 22894 raft_consensus.cc:2468] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:54.130579 23690 leader_election.cc:290] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:54.131400 23586 leader_election.cc:290] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:54.132133 22814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ea5c83aada45d3a98027069e96658c" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:54.134308 22966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ea5c83aada45d3a98027069e96658c" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:54.134879 22966 raft_consensus.cc:3060] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:54.134714 23039 raft_consensus.cc:1275] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:21:54.140148 22814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:54.141016 22814 raft_consensus.cc:2468] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2.
I20260709 13:21:54.146433 23586 consensus_queue.cc:1048] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:54.149281 22785 leader_election.cc:304] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:54.150105 23594 raft_consensus.cc:2804] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:21:54.150209 23546 leader_election.cc:290] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:54.150592 23594 raft_consensus.cc:493] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:54.150998 23594 raft_consensus.cc:3060] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:54.156605 23084 tablet_bootstrap.cc:492] T d217f95123f542e886aa049a608a88d4 P 8ef3887fb3874f8b81079e542d252944: No bootstrap required, opened a new log
I20260709 13:21:54.157824 23084 ts_tablet_manager.cc:1403] T d217f95123f542e886aa049a608a88d4 P 8ef3887fb3874f8b81079e542d252944: Time spent bootstrapping tablet: real 0.209s	user 0.020s	sys 0.008s
I20260709 13:21:54.162418 22860 leader_election.cc:304] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:54.163134 23585 raft_consensus.cc:493] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:54.150903 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:54.159878 23569 raft_consensus.cc:3060] T 0abfe80c1fa642ec8a0d075f75264816 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:54.166368 23556 raft_consensus.cc:2804] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:54.172595 22966 raft_consensus.cc:2468] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 3.
I20260709 13:21:54.173877 22860 leader_election.cc:304] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:54.182514 22969 raft_consensus.cc:2393] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 in current term 2: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:54.171075 23556 raft_consensus.cc:493] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:54.186031 23556 raft_consensus.cc:3060] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:54.190778 22961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:54.191533 22961 raft_consensus.cc:2468] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 2.
I20260709 13:21:54.168381 22888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:54.197841 23562 raft_consensus.cc:2804] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:54.168381 23585 raft_consensus.cc:515] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:54.206511 23562 raft_consensus.cc:697] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 3 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:54.216550 23586 raft_consensus.cc:493] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:54.217145 23586 raft_consensus.cc:515] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:54.225144 23585 leader_election.cc:290] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:54.228621 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:54.231230 22811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:54.228571 23562 consensus_queue.cc:237] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:54.231937 22811 raft_consensus.cc:2468] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2.
I20260709 13:21:54.233322 22860 leader_election.cc:304] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:54.218787 23569 raft_consensus.cc:2749] T 0abfe80c1fa642ec8a0d075f75264816 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:54.235321 23525 raft_consensus.cc:493] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:54.236869 23525 raft_consensus.cc:515] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:54.238963 23525 leader_election.cc:290] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:54.234284 23585 raft_consensus.cc:2804] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:21:54.239879 23585 raft_consensus.cc:493] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:54.240247 23585 raft_consensus.cc:3060] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:54.246891 22814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8be856e3d604f8d8a1c56a88ea18643" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:54.231163 23594 raft_consensus.cc:515] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:54.247848 22814 raft_consensus.cc:2393] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 in current term 2: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:54.248569 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:54.249176 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8be856e3d604f8d8a1c56a88ea18643" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:54.250984 23594 leader_election.cc:290] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:54.251698 22889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:54.252653 23586 leader_election.cc:290] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:54.253875 23008 leader_election.cc:304] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:54.254520 22785 leader_election.cc:304] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:54.254662 22814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630a5e60917545b3a8072fc8358207c5" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:54.254679 23037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630a5e60917545b3a8072fc8358207c5" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:54.255045 23672 raft_consensus.cc:3060] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:54.256127 22814 raft_consensus.cc:2468] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2.
I20260709 13:21:54.256578 23690 raft_consensus.cc:2749] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:21:54.255828 23037 raft_consensus.cc:2468] T 630a5e60917545b3a8072fc8358207c5 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:54.257558 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:54.258248 22858 leader_election.cc:304] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:54.258139 22960 raft_consensus.cc:3060] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:54.259176 23096 raft_consensus.cc:2804] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:21:54.259538 23096 raft_consensus.cc:493] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:54.259853 23096 raft_consensus.cc:3060] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:54.261794 23556 raft_consensus.cc:515] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:54.264086 23672 raft_consensus.cc:2749] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:54.268994 23096 raft_consensus.cc:515] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:54.287298 23096 leader_election.cc:290] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:54.269186 23585 raft_consensus.cc:515] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:54.287827 23556 leader_election.cc:290] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:54.288686 22814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630a5e60917545b3a8072fc8358207c5" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:54.289321 22814 raft_consensus.cc:3060] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:54.290174 23585 leader_election.cc:290] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:54.291126 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:54.292397 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:54.293143 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:54.292434 22811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:54.287899 22889 raft_consensus.cc:3060] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:21:54.295431 22860 consensus_peers.cc:597] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Couldn't send request to peer 8ef3887fb3874f8b81079e542d252944. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.293745 22816 raft_consensus.cc:3060] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:54.300209 22811 raft_consensus.cc:2368] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 for earlier term 2. Current term is 3.
I20260709 13:21:54.301546 22860 leader_election.cc:400] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Vote denied by peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155) with higher term. Message: Invalid argument: T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 for earlier term 2. Current term is 3.
I20260709 13:21:54.301860 22860 leader_election.cc:403] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:21:54.304689 22889 raft_consensus.cc:2468] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 3.
I20260709 13:21:54.305387 23585 raft_consensus.cc:2749] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155) with higher term. Message: Invalid argument: T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 for earlier term 2. Current term is 3.
I20260709 13:21:54.308678 23084 raft_consensus.cc:359] T d217f95123f542e886aa049a608a88d4 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:54.309460 23084 raft_consensus.cc:385] T d217f95123f542e886aa049a608a88d4 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:54.309723 23084 raft_consensus.cc:740] T d217f95123f542e886aa049a608a88d4 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Initialized, Role: FOLLOWER
I20260709 13:21:54.310817 23084 consensus_queue.cc:260] T d217f95123f542e886aa049a608a88d4 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:54.312057 22814 raft_consensus.cc:2468] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 3.
I20260709 13:21:54.312937 23084 ts_tablet_manager.cc:1434] T d217f95123f542e886aa049a608a88d4 P 8ef3887fb3874f8b81079e542d252944: Time spent starting tablet: real 0.155s	user 0.005s	sys 0.001s
I20260709 13:21:54.313218 22860 leader_election.cc:304] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:54.314085 23562 raft_consensus.cc:2804] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:54.316174 22718 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50740 (request call id 74) took 1304 ms. Trace:
I20260709 13:21:54.326640 22960 raft_consensus.cc:2468] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 3.
I20260709 13:21:54.320469 22714 catalog_manager.cc:5697] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:54.332101 22714 catalog_manager.cc:5697] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:54.333561 22714 catalog_manager.cc:5697] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:54.316687 23562 raft_consensus.cc:697] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:54.316959 22961 raft_consensus.cc:3060] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:54.319828 22785 leader_election.cc:304] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:54.342425 23594 raft_consensus.cc:2804] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:54.342993 23594 raft_consensus.cc:697] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 3 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:54.322083 23039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630a5e60917545b3a8072fc8358207c5" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:54.343590 23039 raft_consensus.cc:3060] T 630a5e60917545b3a8072fc8358207c5 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:21:54.343809 23594 consensus_queue.cc:237] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:54.344543 23586 raft_consensus.cc:493] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:54.345515 23586 raft_consensus.cc:515] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:54.350016 23586 leader_election.cc:290] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:54.349855 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "087d64074b5943f0a7d789ead06f4783" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:54.323530 22816 raft_consensus.cc:2468] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 3.
I20260709 13:21:54.353526 22961 raft_consensus.cc:1275] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Refusing update from remote peer 27745ef236cd494684f62a1269411a68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:54.324676 22718 rpcz_store.cc:276] 0709 13:21:53.025422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:53.027829 (+  2407us) service_pool.cc:224] Handling call
0709 13:21:53.923991 (+896162us) spinlock_profiling.cc:243] Waited 25.3 ms on lock 0x7b68000c0b60. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5cea7e6de1 00007f5cea7c2654 00007f5cea7c24ce 00007f5cea7bfc2b 00007f5cea8c8c3e 00007f5ce8d892a4 00007f5ce8d89232 00007f5ce8d89161 00007f5ce8d890dc 00007f5ce8d88392 00007f5ce2d4648c
0709 13:21:54.316111 (+392120us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:54.141896 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:54.142514 (+   618us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:54.142549 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:21:54.142561 (+    12us) tablet.cc:662] Decoding operations
0709 13:21:54.142879 (+   318us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:54.142932 (+    53us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:54.142957 (+    25us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:21:54.143133 (+   176us) tablet.cc:701] Row locks acquired
0709 13:21:54.143149 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:21:54.143288 (+   139us) write_op.cc:270] Start()
0709 13:21:54.143397 (+   109us) write_op.cc:276] Timestamp: P: 1783603314143247 usec, L: 0
0709 13:21:54.143438 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 13:21:54.144087 (+   649us) log.cc:844] Serialized 537 byte log entry
0709 13:21:54.211495 (+ 67408us) op_driver.cc:464] REPLICATION: finished
0709 13:21:54.266319 (+ 54824us) write_op.cc:301] APPLY: starting
0709 13:21:54.266408 (+    89us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:54.288711 (+ 22303us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:54.288837 (+   126us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:54.289059 (+   222us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:54.289211 (+   152us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:21:54.289250 (+    39us) write_op.cc:312] APPLY: finished
0709 13:21:54.289465 (+   215us) log.cc:844] Serialized 22 byte log entry
0709 13:21:54.295738 (+  6273us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:54.295823 (+    85us) write_op.cc:454] Released schema lock
0709 13:21:54.295923 (+   100us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:54.296282 (+   359us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:54.296324 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":11,"spinlock_wait_cycles":58125568,"child_traces":[["op",{"apply.queue_time_us":48185,"apply.run_cpu_time_us":1664,"apply.run_wall_time_us":29961,"num_ops":1,"prepare.queue_time_us":149883,"prepare.run_cpu_time_us":2575,"prepare.run_wall_time_us":9045,"replication_time_us":65465,"thread_start_us":100176,"threads_started":2}]]}
I20260709 13:21:54.359885 22860 leader_election.cc:304] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:54.360891 23672 consensus_queue.cc:1048] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:54.361263 22811 raft_consensus.cc:1275] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:54.363340 23556 raft_consensus.cc:2804] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:54.364140 23556 raft_consensus.cc:697] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:54.364988 23556 consensus_queue.cc:237] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:54.366078 22717 catalog_manager.cc:5697] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 0 to 2, leader changed from <none> to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:54.367553 23585 consensus_queue.cc:1048] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:54.368036 22717 catalog_manager.cc:5697] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 1 to 2, leader changed from 27745ef236cd494684f62a1269411a68 (127.14.23.68) to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:54.370224 22717 catalog_manager.cc:5697] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 1 to 2, leader changed from 8ef3887fb3874f8b81079e542d252944 (127.14.23.67) to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:54.366153 22814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "087d64074b5943f0a7d789ead06f4783" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:54.366679 23562 consensus_queue.cc:237] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:54.375269 22717 catalog_manager.cc:5697] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 1 to 2, leader changed from 91f33bdbdd63438294071eb3776db679 (127.14.23.65) to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:54.378888 22714 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50708 (request call id 73) took 1788 ms. Trace:
I20260709 13:21:54.377528 22814 raft_consensus.cc:2468] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2.
I20260709 13:21:54.379244 22717 catalog_manager.cc:5697] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 1 to 2, leader changed from 91f33bdbdd63438294071eb3776db679 (127.14.23.65) to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:54.382769 22811 raft_consensus.cc:1275] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Refusing update from remote peer 8ef3887fb3874f8b81079e542d252944: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:54.384289 22860 leader_election.cc:304] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:54.386826 22717 catalog_manager.cc:5697] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 1 to 3, leader changed from 27745ef236cd494684f62a1269411a68 (127.14.23.68) to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 3 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:54.391688 23039 raft_consensus.cc:2468] T 630a5e60917545b3a8072fc8358207c5 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 3.
I20260709 13:21:54.390811 23585 raft_consensus.cc:2804] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:21:54.379361 22714 rpcz_store.cc:276] 0709 13:21:52.590673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:52.590856 (+   183us) service_pool.cc:224] Handling call
0709 13:21:54.378837 (+1787981us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:54.336560 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:54.337085 (+   525us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:54.337123 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:21:54.337134 (+    11us) tablet.cc:662] Decoding operations
0709 13:21:54.337679 (+   545us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:54.337729 (+    50us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:54.337757 (+    28us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:21:54.338081 (+   324us) tablet.cc:701] Row locks acquired
0709 13:21:54.338094 (+    13us) write_op.cc:260] PREPARE: finished
0709 13:21:54.338239 (+   145us) write_op.cc:270] Start()
0709 13:21:54.338362 (+   123us) write_op.cc:276] Timestamp: P: 1783603314338194 usec, L: 0
0709 13:21:54.338378 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 13:21:54.338955 (+   577us) log.cc:844] Serialized 1307 byte log entry
0709 13:21:54.340949 (+  1994us) op_driver.cc:464] REPLICATION: finished
0709 13:21:54.341353 (+   404us) write_op.cc:301] APPLY: starting
0709 13:21:54.341429 (+    76us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:54.341556 (+   127us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:54.341652 (+    96us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:54.342089 (+   437us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:54.342242 (+   153us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:21:54.342282 (+    40us) write_op.cc:312] APPLY: finished
0709 13:21:54.342511 (+   229us) log.cc:844] Serialized 34 byte log entry
0709 13:21:54.342664 (+   153us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:54.342750 (+    86us) write_op.cc:454] Released schema lock
0709 13:21:54.342789 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:54.343083 (+   294us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:54.343108 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":419421,"reported_tablets":14,"child_traces":[["op",{"apply.queue_time_us":217,"apply.run_cpu_time_us":1690,"apply.run_wall_time_us":1699,"num_ops":3,"prepare.queue_time_us":108,"prepare.run_cpu_time_us":2653,"prepare.run_wall_time_us":2653,"replication_time_us":2466}]]}
I20260709 13:21:54.399832 23585 raft_consensus.cc:493] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:54.400249 23585 raft_consensus.cc:3060] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:54.402868 23672 consensus_queue.cc:1048] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:54.409663 23594 raft_consensus.cc:493] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:54.410573 23594 raft_consensus.cc:515] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:54.398411 23569 consensus_queue.cc:1048] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:21:54.412827 23585 raft_consensus.cc:515] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:54.415972 22889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:54.416199 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "087d64074b5943f0a7d789ead06f4783" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:54.416831 22820 raft_consensus.cc:3060] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:54.423777 23631 raft_consensus.cc:493] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:54.424453 23631 raft_consensus.cc:515] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:54.426851 23631 leader_election.cc:290] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:54.428645 22820 raft_consensus.cc:2468] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 3.
I20260709 13:21:54.432374 22860 leader_election.cc:304] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:54.432667 23585 leader_election.cc:290] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:54.433279 23585 raft_consensus.cc:2804] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:54.433741 23585 raft_consensus.cc:697] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:54.429883 23594 leader_election.cc:290] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:54.415355 23556 raft_consensus.cc:493] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:54.433545 23690 raft_consensus.cc:493] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:54.434176 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "087d64074b5943f0a7d789ead06f4783" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:54.429581 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:54.434687 23585 consensus_queue.cc:237] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:54.434748 23690 raft_consensus.cc:515] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:54.435665 22969 raft_consensus.cc:2393] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 in current term 2: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:54.439435 22718 catalog_manager.cc:5697] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 1 to 2, leader changed from 91f33bdbdd63438294071eb3776db679 (127.14.23.65) to 8ef3887fb3874f8b81079e542d252944 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "8ef3887fb3874f8b81079e542d252944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:54.445953 22718 catalog_manager.cc:5697] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 1 to 2, leader changed from 91f33bdbdd63438294071eb3776db679 (127.14.23.65) to 8ef3887fb3874f8b81079e542d252944 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "8ef3887fb3874f8b81079e542d252944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:54.447638 23039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:54.442672 22888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a064be2ec640a4a8283957d0b9b98c" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:54.450351 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:54.450742 23621 raft_consensus.cc:493] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:54.451073 22820 raft_consensus.cc:2468] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 2.
I20260709 13:21:54.442652 23038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a064be2ec640a4a8283957d0b9b98c" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:54.451948 23038 raft_consensus.cc:2368] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 for earlier term 2. Current term is 3.
I20260709 13:21:54.437422 23690 leader_election.cc:290] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:54.443682 23556 raft_consensus.cc:515] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:54.455812 23621 raft_consensus.cc:515] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:54.461016 22783 leader_election.cc:400] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509) with higher term. Message: Invalid argument: T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 for earlier term 2. Current term is 3.
I20260709 13:21:54.470260 22783 leader_election.cc:403] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:54.472798 22889 raft_consensus.cc:2468] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 2.
I20260709 13:21:54.475508 22886 raft_consensus.cc:1275] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Refusing update from remote peer 27745ef236cd494684f62a1269411a68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:54.476943 23038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:54.477655 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b752c38e1b47a489a3d720801661d1" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:54.478603 22969 raft_consensus.cc:2393] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 in current term 2: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:54.482563 23008 leader_election.cc:304] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:54.467113 23556 leader_election.cc:290] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:54.484483 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:54.485208 22819 raft_consensus.cc:2468] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2.
I20260709 13:21:54.486305 22886 raft_consensus.cc:1275] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Refusing update from remote peer 8ef3887fb3874f8b81079e542d252944: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:54.488251 22785 leader_election.cc:304] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68
I20260709 13:21:54.464404 23037 raft_consensus.cc:1275] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:54.489876 23596 consensus_queue.cc:1048] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:54.490130 23546 raft_consensus.cc:2804] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:21:54.490865 23546 raft_consensus.cc:493] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:54.491230 23546 raft_consensus.cc:3060] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:54.480494 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b752c38e1b47a489a3d720801661d1" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:54.491393 22860 leader_election.cc:304] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68
I20260709 13:21:54.491998 22817 raft_consensus.cc:2468] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 2.
I20260709 13:21:54.492558 23096 raft_consensus.cc:2804] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:21:54.492947 23096 raft_consensus.cc:493] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:54.493315 23096 raft_consensus.cc:3060] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:54.492841 22969 raft_consensus.cc:1275] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:54.495958 22819 raft_consensus.cc:1275] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Refusing update from remote peer 8ef3887fb3874f8b81079e542d252944: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:54.467535 23621 leader_election.cc:290] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:54.472504 23690 raft_consensus.cc:3060] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:21:54.499744 23569 consensus_queue.cc:1048] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:54.501248 23096 raft_consensus.cc:515] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:54.505874 23008 leader_election.cc:304] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:54.513562 23096 leader_election.cc:290] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:54.508742 23621 raft_consensus.cc:2804] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:54.524672 22715 catalog_manager.cc:5697] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 1 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:54.522374 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:54.528025 23621 raft_consensus.cc:493] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:54.528951 23621 raft_consensus.cc:3060] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:54.514765 23525 raft_consensus.cc:2804] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:54.530325 23525 raft_consensus.cc:493] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:54.530706 23525 raft_consensus.cc:3060] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:54.514605 23672 consensus_queue.cc:1048] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:21:54.501248 23546 raft_consensus.cc:515] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:54.504956 23699 consensus_queue.cc:1048] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:21:54.510905 23566 consensus_queue.cc:1048] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:21:54.510741 23631 consensus_queue.cc:1048] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 13:21:54.542717 22715 catalog_manager.cc:5697] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 1 to 2, leader changed from 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66) to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 2 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:54.535495 23038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:54.556358 22820 raft_consensus.cc:2393] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 in current term 3: Already voted for candidate 91f33bdbdd63438294071eb3776db679 in this term.
I20260709 13:21:54.559580 23034 raft_consensus.cc:1275] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Refusing update from remote peer 8ef3887fb3874f8b81079e542d252944: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:54.567288 23596 consensus_queue.cc:1048] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:54.548036 23037 raft_consensus.cc:1275] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:54.590363 23690 raft_consensus.cc:2749] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509) with higher term. Message: Invalid argument: T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 for earlier term 2. Current term is 3.
I20260709 13:21:54.592623 22820 raft_consensus.cc:1275] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:54.599502 22858 leader_election.cc:304] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:54.609398 23096 consensus_queue.cc:1048] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:54.608661 23557 consensus_queue.cc:1048] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:21:54.616492 22962 raft_consensus.cc:3060] T d217f95123f542e886aa049a608a88d4 P 8ef3887fb3874f8b81079e542d252944 [term 0 FOLLOWER]: Advancing to term 2
I20260709 13:21:54.605151 23585 raft_consensus.cc:2749] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:21:54.630709 23039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:54.639259 23621 raft_consensus.cc:515] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:54.639811 23525 raft_consensus.cc:515] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
W20260709 13:21:54.641793 23736 log.cc:927] Time spent T 0abfe80c1fa642ec8a0d075f75264816 P 8ef3887fb3874f8b81079e542d252944: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:21:54.649044 22811 raft_consensus.cc:3060] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:21:54.652275 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:54.653250 22968 raft_consensus.cc:2393] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 in current term 2: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
W20260709 13:21:54.661715 23735 log.cc:927] Time spent T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:21:54.671904 22811 raft_consensus.cc:1275] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:54.673101 23569 raft_consensus.cc:493] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:54.675088 23569 raft_consensus.cc:515] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:54.677255 23569 leader_election.cc:290] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:54.677376 23663 raft_consensus.cc:493] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:54.677989 23663 raft_consensus.cc:515] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:54.679975 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:54.680825 22819 raft_consensus.cc:2368] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 for earlier term 2. Current term is 3.
I20260709 13:21:54.673867 22889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:54.690137 22934 leader_election.cc:400] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155) with higher term. Message: Invalid argument: T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 for earlier term 2. Current term is 3.
I20260709 13:21:54.690578 22934 leader_election.cc:403] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:54.667415 23546 leader_election.cc:290] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:54.679982 23663 leader_election.cc:290] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:54.694180 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "087d64074b5943f0a7d789ead06f4783" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:54.695031 23569 raft_consensus.cc:3060] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:21:54.694332 23690 raft_consensus.cc:493] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:54.699451 22886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:54.701303 23690 raft_consensus.cc:515] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
W20260709 13:21:54.707785 23742 log.cc:927] Time spent T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:21:54.710282 23690 leader_election.cc:290] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:54.701751 22889 raft_consensus.cc:2393] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 in current term 3: Already voted for candidate 49cbf11e18ce4268a7fcca76872afb23 in this term.
I20260709 13:21:54.711695 23586 consensus_queue.cc:1048] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:54.711083 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e230e64c6984b72a2ba543f2a017e25" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:54.713289 22969 raft_consensus.cc:2468] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 2.
I20260709 13:21:54.714388 22785 leader_election.cc:304] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:54.721338 22785 leader_election.cc:304] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:54.714407 23037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e230e64c6984b72a2ba543f2a017e25" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:54.727638 23037 raft_consensus.cc:2393] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 in current term 2: Already voted for candidate 27745ef236cd494684f62a1269411a68 in this term.
I20260709 13:21:54.722775 23546 raft_consensus.cc:2749] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:21:54.723743 23690 raft_consensus.cc:2804] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:54.734704 23690 raft_consensus.cc:493] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:54.735098 23690 raft_consensus.cc:3060] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:54.736791 23525 leader_election.cc:290] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:54.740082 23034 raft_consensus.cc:1275] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:54.742035 23621 leader_election.cc:290] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:54.738674 23569 raft_consensus.cc:2749] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155) with higher term. Message: Invalid argument: T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 for earlier term 2. Current term is 3.
I20260709 13:21:54.745450 23690 raft_consensus.cc:515] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:54.747898 23690 leader_election.cc:290] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:54.699586 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "087d64074b5943f0a7d789ead06f4783" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:54.791513 22819 raft_consensus.cc:2368] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 for earlier term 2. Current term is 3.
I20260709 13:21:54.796008 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b752c38e1b47a489a3d720801661d1" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:54.801105 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b752c38e1b47a489a3d720801661d1" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:54.802820 22969 raft_consensus.cc:2393] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 in current term 2: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:54.803571 23036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e230e64c6984b72a2ba543f2a017e25" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:54.819317 23036 raft_consensus.cc:2393] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 in current term 2: Already voted for candidate 27745ef236cd494684f62a1269411a68 in this term.
I20260709 13:21:54.826589 22717 catalog_manager.cc:5697] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:54.789492 22785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 68186.2us
I20260709 13:21:54.845073 23566 raft_consensus.cc:493] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:54.847910 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e230e64c6984b72a2ba543f2a017e25" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:54.848839 22961 raft_consensus.cc:1275] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:54.848146 22934 leader_election.cc:400] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155) with higher term. Message: Invalid argument: T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 for earlier term 2. Current term is 3.
I20260709 13:21:54.850723 22934 leader_election.cc:403] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:54.836931 23557 consensus_queue.cc:1048] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:54.846093 23566 raft_consensus.cc:515] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:54.853689 23621 raft_consensus.cc:493] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:54.854022 23663 raft_consensus.cc:3060] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:21:54.854231 23566 leader_election.cc:290] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:54.859036 22819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:54.866650 22815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:54.870483 23039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:54.871470 23039 raft_consensus.cc:2468] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2.
I20260709 13:21:54.876261 23568 consensus_queue.cc:1048] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:54.881793 22963 raft_consensus.cc:1275] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:54.879817 23621 raft_consensus.cc:515] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:54.893038 23568 consensus_queue.cc:1048] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:54.896247 23621 leader_election.cc:290] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:54.896802 22963 raft_consensus.cc:1275] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:54.946533 22714 catalog_manager.cc:5697] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } }
W20260709 13:21:54.942941 22934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 43881.8us
I20260709 13:21:54.949307 22714 catalog_manager.cc:5697] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:54.958617 22858 leader_election.cc:304] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:54.959802 22817 raft_consensus.cc:2468] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 2.
I20260709 13:21:54.960335 23556 raft_consensus.cc:2804] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:21:54.960851 23556 raft_consensus.cc:493] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:54.961212 23556 raft_consensus.cc:3060] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:54.949658 23690 consensus_queue.cc:1048] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:21:54.986810 23008 leader_election.cc:304] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:54.994191 23525 raft_consensus.cc:2804] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:55.003253 22714 catalog_manager.cc:5697] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:55.006022 22960 raft_consensus.cc:2468] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 2.
I20260709 13:21:55.009079 22819 raft_consensus.cc:2468] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 2.
I20260709 13:21:55.011422 23525 raft_consensus.cc:697] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 2 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:55.013049 23525 consensus_queue.cc:237] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.014506 22785 leader_election.cc:304] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68
I20260709 13:21:55.023916 22888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94800e302e4641bb857f2d160f1f60ea" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:55.027660 23672 raft_consensus.cc:493] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:55.030036 23672 raft_consensus.cc:515] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.032110 22819 raft_consensus.cc:1275] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:55.029117 22888 raft_consensus.cc:1275] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:55.032859 23672 leader_election.cc:290] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:55.028821 22714 catalog_manager.cc:5697] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 1 to 3, leader changed from 8ef3887fb3874f8b81079e542d252944 (127.14.23.67) to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 3 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:55.034099 23586 consensus_queue.cc:1048] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:55.038220 23690 raft_consensus.cc:2804] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:55.040557 23690 raft_consensus.cc:697] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 2 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:55.041951 23690 consensus_queue.cc:237] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:54.987565 23663 raft_consensus.cc:2749] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155) with higher term. Message: Invalid argument: T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 for earlier term 2. Current term is 3.
I20260709 13:21:55.049134 22969 raft_consensus.cc:1275] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:55.027666 22817 raft_consensus.cc:1275] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:55.062603 22962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94800e302e4641bb857f2d160f1f60ea" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
W20260709 13:21:55.066010 22785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 51635.1us
I20260709 13:21:55.057623 23586 consensus_queue.cc:1048] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:21:55.082896 23568 consensus_queue.cc:1048] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:55.084534 23663 raft_consensus.cc:493] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:55.085193 23663 raft_consensus.cc:515] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.112740 23039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b752c38e1b47a489a3d720801661d1" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:55.117264 22820 raft_consensus.cc:1275] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:55.118292 23546 consensus_queue.cc:1048] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 13:21:55.119725 23562 raft_consensus.cc:493] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:55.120417 23562 raft_consensus.cc:515] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:55.094734 23556 raft_consensus.cc:515] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:55.093174 23663 leader_election.cc:290] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:55.131716 22814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505eab84b06f46ee9d336abf01353dee" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:55.132619 22814 raft_consensus.cc:2468] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2.
I20260709 13:21:55.133755 23682 raft_consensus.cc:493] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:55.134677 23682 raft_consensus.cc:515] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:55.137297 23682 leader_election.cc:290] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:55.141857 23556 leader_election.cc:290] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:55.144948 22962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505eab84b06f46ee9d336abf01353dee" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:55.146169 22962 raft_consensus.cc:2393] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 in current term 2: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:55.149346 23039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:55.156785 23039 raft_consensus.cc:3060] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.159695 23682 raft_consensus.cc:493] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:55.160389 23682 raft_consensus.cc:515] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.162540 23008 leader_election.cc:304] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:55.163985 23744 raft_consensus.cc:2804] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:55.164675 23744 raft_consensus.cc:697] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 2 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:55.165827 23546 raft_consensus.cc:493] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:55.149220 23036 raft_consensus.cc:1275] T 630a5e60917545b3a8072fc8358207c5 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:55.122848 23562 leader_election.cc:290] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:55.166595 23546 raft_consensus.cc:515] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.167526 22860 leader_election.cc:304] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:55.166188 23744 consensus_queue.cc:237] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.168656 23568 raft_consensus.cc:2804] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:55.169072 23568 raft_consensus.cc:493] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:55.169441 23568 raft_consensus.cc:3060] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:55.166666 23556 raft_consensus.cc:493] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:55.171100 23556 raft_consensus.cc:515] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:55.173435 23556 leader_election.cc:290] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
W20260709 13:21:55.174273 22785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 43666.9us
I20260709 13:21:55.186439 23682 leader_election.cc:290] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:55.188102 23546 leader_election.cc:290] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:55.188831 23586 raft_consensus.cc:493] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:55.189038 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e638dc505d4b04b802e91b8863d833" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:55.189556 23586 raft_consensus.cc:515] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:55.168622 23699 consensus_queue.cc:1048] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:21:55.192054 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e638dc505d4b04b802e91b8863d833" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:55.193133 22820 raft_consensus.cc:2468] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
I20260709 13:21:55.194945 23562 consensus_queue.cc:1048] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:55.189844 22963 raft_consensus.cc:2468] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 2.
I20260709 13:21:55.198776 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955a6d8dbbe94ef9be9c1f69b10675d8" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:55.199409 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955a6d8dbbe94ef9be9c1f69b10675d8" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:55.199543 22817 raft_consensus.cc:2468] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2.
I20260709 13:21:55.200322 22960 raft_consensus.cc:2393] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 in current term 2: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:55.204753 22813 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b752c38e1b47a489a3d720801661d1" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:55.205664 22813 raft_consensus.cc:2468] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 2.
I20260709 13:21:55.202625 22860 leader_election.cc:304] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:55.211508 23562 raft_consensus.cc:2804] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:55.212050 23562 raft_consensus.cc:493] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:55.212558 23562 raft_consensus.cc:3060] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:55.214969 23672 raft_consensus.cc:493] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:55.215703 23672 raft_consensus.cc:515] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.219238 23586 leader_election.cc:290] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:55.222322 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968e16cef40146509d654274b9fd2687" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:55.223098 23041 raft_consensus.cc:2468] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2.
I20260709 13:21:55.225570 22962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955a6d8dbbe94ef9be9c1f69b10675d8" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:55.226266 22858 leader_election.cc:304] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:55.228308 22934 leader_election.cc:304] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68
I20260709 13:21:55.232637 23008 leader_election.cc:304] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:55.234891 23008 leader_election.cc:304] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:55.236605 23096 raft_consensus.cc:2804] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:55.236835 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968e16cef40146509d654274b9fd2687" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:55.237121 23096 raft_consensus.cc:493] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:55.237486 23096 raft_consensus.cc:3060] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:55.237762 22960 raft_consensus.cc:2393] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 in current term 2: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:55.238435 23525 raft_consensus.cc:2804] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:21:55.238859 23525 raft_consensus.cc:493] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.239192 23525 raft_consensus.cc:3060] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.239110 23672 leader_election.cc:290] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:55.242072 23596 raft_consensus.cc:2804] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:21:55.242574 23596 raft_consensus.cc:493] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.242970 23596 raft_consensus.cc:3060] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.231591 22814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:55.246019 22814 raft_consensus.cc:3060] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.247655 23744 raft_consensus.cc:493] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:55.248291 23744 raft_consensus.cc:515] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:55.248735 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:55.249511 22963 raft_consensus.cc:2468] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 2.
I20260709 13:21:55.236685 22888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955a6d8dbbe94ef9be9c1f69b10675d8" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:55.253976 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e638dc505d4b04b802e91b8863d833" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:55.254160 23039 raft_consensus.cc:2468] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 3.
I20260709 13:21:55.254868 22960 raft_consensus.cc:2393] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 in current term 2: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:55.226341 22962 raft_consensus.cc:2468] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 2.
I20260709 13:21:55.258919 22858 leader_election.cc:304] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:55.259747 23556 raft_consensus.cc:2804] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:55.260314 23556 raft_consensus.cc:697] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 3 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:55.261327 23556 consensus_queue.cc:237] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:55.266677 23751 raft_consensus.cc:2749] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:21:55.278558 23672 raft_consensus.cc:493] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:55.279171 23672 raft_consensus.cc:515] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.280031 23744 leader_election.cc:290] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:55.281507 23096 raft_consensus.cc:515] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:55.281751 23672 leader_election.cc:290] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:55.276518 23568 raft_consensus.cc:515] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:55.289251 23562 raft_consensus.cc:515] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:55.299785 22888 raft_consensus.cc:2468] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 2.
I20260709 13:21:55.299929 23568 leader_election.cc:290] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:55.300235 23562 leader_election.cc:290] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:55.301017 23663 raft_consensus.cc:493] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:55.301854 23663 raft_consensus.cc:515] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.301940 23039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e638dc505d4b04b802e91b8863d833" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:55.304667 23663 leader_election.cc:290] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:55.306070 23036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:55.307706 23586 raft_consensus.cc:493] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:55.308388 23586 raft_consensus.cc:515] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:55.308820 23525 raft_consensus.cc:515] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.294582 22785 leader_election.cc:304] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:55.310063 23036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:55.310648 23690 raft_consensus.cc:2804] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:21:55.310905 23036 raft_consensus.cc:2468] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:55.311060 23690 raft_consensus.cc:493] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.310132 23039 raft_consensus.cc:2368] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 for earlier term 2. Current term is 3.
I20260709 13:21:55.311758 23690 raft_consensus.cc:3060] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.312247 23586 leader_election.cc:290] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:55.320803 22814 raft_consensus.cc:2468] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 3.
I20260709 13:21:55.322389 22785 leader_election.cc:304] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:55.291118 23596 raft_consensus.cc:515] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.333447 23096 leader_election.cc:290] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:55.337654 23596 leader_election.cc:290] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:55.321447 23690 raft_consensus.cc:515] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.333384 23039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b154956cc17f4ba68f906daffbcedff6" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:55.343688 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e230e64c6984b72a2ba543f2a017e25" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:55.339475 23039 raft_consensus.cc:2468] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 1.
I20260709 13:21:55.326167 23546 raft_consensus.cc:493] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:55.344375 22969 raft_consensus.cc:2468] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 2.
I20260709 13:21:55.333549 23682 raft_consensus.cc:2804] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:21:55.363698 22888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:55.364585 22888 raft_consensus.cc:2368] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 for earlier term 2. Current term is 3.
I20260709 13:21:55.365490 23036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b752c38e1b47a489a3d720801661d1" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:55.368427 22962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505eab84b06f46ee9d336abf01353dee" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:55.369419 22962 raft_consensus.cc:2393] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 in current term 2: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:55.355337 23682 raft_consensus.cc:493] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.361006 23546 raft_consensus.cc:515] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:55.375808 23682 raft_consensus.cc:3060] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.361589 23672 raft_consensus.cc:493] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:55.376937 23672 raft_consensus.cc:515] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:55.362200 23008 leader_election.cc:304] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:55.378645 23621 raft_consensus.cc:2804] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:21:55.379034 23621 raft_consensus.cc:493] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.379128 23672 leader_election.cc:290] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:55.379438 23621 raft_consensus.cc:3060] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.380033 23008 leader_election.cc:400] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323) with higher term. Message: Invalid argument: T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 for earlier term 2. Current term is 3.
I20260709 13:21:55.380430 23008 leader_election.cc:403] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:55.382658 23751 raft_consensus.cc:3060] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.374385 23034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968e16cef40146509d654274b9fd2687" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:55.374909 22858 leader_election.cc:304] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
W20260709 13:21:55.392956 22785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 65751.4us
I20260709 13:21:55.399951 23557 raft_consensus.cc:2804] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:21:55.400324 23663 raft_consensus.cc:493] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:55.400506 23557 raft_consensus.cc:493] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.400888 23557 raft_consensus.cc:3060] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.400938 23663 raft_consensus.cc:515] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:55.380506 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:55.407837 22968 raft_consensus.cc:2393] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 in current term 2: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:55.356956 23596 raft_consensus.cc:493] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:55.394919 22814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505eab84b06f46ee9d336abf01353dee" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:55.413278 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e09daa4f2dd474d96ae9c4dfa1a8758" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:55.414345 22894 raft_consensus.cc:2393] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 in current term 2: Already voted for candidate 49cbf11e18ce4268a7fcca76872afb23 in this term.
I20260709 13:21:55.418488 23771 raft_consensus.cc:493] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:55.419430 23594 raft_consensus.cc:493] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:55.419754 22811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e09daa4f2dd474d96ae9c4dfa1a8758" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:55.419987 23594 raft_consensus.cc:515] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:55.420279 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:55.419782 22815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:55.421226 22820 raft_consensus.cc:2368] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 for earlier term 2. Current term is 3.
I20260709 13:21:55.421444 22815 raft_consensus.cc:2368] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 for earlier term 2. Current term is 3.
I20260709 13:21:55.422289 22820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:55.422602 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b752c38e1b47a489a3d720801661d1" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:55.422327 22818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955a6d8dbbe94ef9be9c1f69b10675d8" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:55.422883 22817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e230e64c6984b72a2ba543f2a017e25" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:55.423236 22816 raft_consensus.cc:3060] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.424005 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955a6d8dbbe94ef9be9c1f69b10675d8" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:55.424309 23744 raft_consensus.cc:493] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:55.425242 22820 raft_consensus.cc:2468] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 2.
I20260709 13:21:55.425722 22815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b154956cc17f4ba68f906daffbcedff6" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:55.426434 22815 raft_consensus.cc:2468] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2.
I20260709 13:21:55.419139 23771 raft_consensus.cc:515] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:55.428807 22963 raft_consensus.cc:2393] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 in current term 2: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:55.424793 22961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968e16cef40146509d654274b9fd2687" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:55.430449 22961 raft_consensus.cc:2393] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 in current term 2: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:55.437515 22969 raft_consensus.cc:1240] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Rejecting Update request from peer 27745ef236cd494684f62a1269411a68 for earlier term 2. Current term is 3. Ops: []
I20260709 13:21:55.439188 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e638dc505d4b04b802e91b8863d833" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:55.439949 22960 raft_consensus.cc:3060] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.424906 23744 raft_consensus.cc:515] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.435743 22811 raft_consensus.cc:2468] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 2.
I20260709 13:21:55.447124 23525 leader_election.cc:290] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 3 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:55.453857 22889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968e16cef40146509d654274b9fd2687" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:55.454753 22889 raft_consensus.cc:2468] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 2.
I20260709 13:21:55.456344 23596 raft_consensus.cc:515] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.464368 22934 leader_election.cc:304] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:55.465543 23569 raft_consensus.cc:2804] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:21:55.466023 23569 raft_consensus.cc:493] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.466363 23569 raft_consensus.cc:3060] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.481703 23663 leader_election.cc:290] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:55.483158 23039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968e16cef40146509d654274b9fd2687" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:55.467447 22718 catalog_manager.cc:5697] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 1 to 2, leader changed from 91f33bdbdd63438294071eb3776db679 (127.14.23.65) to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 2 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:55.500519 23594 leader_election.cc:290] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:55.506453 23008 leader_election.cc:400] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155) with higher term. Message: Invalid argument: T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 for earlier term 2. Current term is 3.
I20260709 13:21:55.508982 23771 leader_election.cc:290] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:55.495741 23525 consensus_queue.cc:1059] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
W20260709 13:21:55.515444 22785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 105646us
I20260709 13:21:55.518500 23008 leader_election.cc:400] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155) with higher term. Message: Invalid argument: T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 for earlier term 2. Current term is 3.
I20260709 13:21:55.518947 23008 leader_election.cc:403] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:55.519659 22934 leader_election.cc:304] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68
I20260709 13:21:55.522219 23039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8b119f5c5844d1be439423b333ec60" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:55.523123 23039 raft_consensus.cc:2393] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ef3887fb3874f8b81079e542d252944 in current term 2: Already voted for candidate 27745ef236cd494684f62a1269411a68 in this term.
I20260709 13:21:55.523406 23756 raft_consensus.cc:2804] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:21:55.523831 23756 raft_consensus.cc:493] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.524209 23756 raft_consensus.cc:3060] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.527508 23034 raft_consensus.cc:2468] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2.
I20260709 13:21:55.530270 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:55.530233 23557 raft_consensus.cc:515] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:55.531136 22968 raft_consensus.cc:2468] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 3.
I20260709 13:21:55.532260 23546 leader_election.cc:290] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
W20260709 13:21:55.533102 22783 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 51081.5us
I20260709 13:21:55.503041 23690 leader_election.cc:290] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:55.536497 23557 leader_election.cc:290] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:55.535842 22785 leader_election.cc:304] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:55.546190 22783 leader_election.cc:400] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509) with higher term. Message: Invalid argument: T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 for earlier term 2. Current term is 3.
I20260709 13:21:55.547641 22783 leader_election.cc:403] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:55.553696 22816 raft_consensus.cc:2468] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 3.
I20260709 13:21:55.542565 23562 raft_consensus.cc:493] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:55.548987 23771 raft_consensus.cc:2804] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:21:55.559242 23690 raft_consensus.cc:3060] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.557281 22889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:55.559656 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
W20260709 13:21:55.581595 22934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 52595.6us
I20260709 13:21:55.564615 23596 leader_election.cc:290] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:55.565191 23771 raft_consensus.cc:493] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:55.574924 23525 raft_consensus.cc:3060] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.570681 22816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e638dc505d4b04b802e91b8863d833" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:55.588109 22858 leader_election.cc:304] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:55.572510 22960 raft_consensus.cc:2468] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 3.
I20260709 13:21:55.565744 23562 raft_consensus.cc:515] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:55.591622 22963 raft_consensus.cc:3060] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.590215 23771 raft_consensus.cc:3060] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.584497 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:55.596889 23586 raft_consensus.cc:2804] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:55.620004 23586 raft_consensus.cc:697] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:55.621018 23586 consensus_queue.cc:237] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:55.607575 23744 leader_election.cc:290] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:55.638238 22889 raft_consensus.cc:2393] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 in current term 3: Already voted for candidate 49cbf11e18ce4268a7fcca76872afb23 in this term.
I20260709 13:21:55.639459 23566 raft_consensus.cc:493] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:55.639968 23566 raft_consensus.cc:515] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:55.651755 23562 leader_election.cc:290] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
W20260709 13:21:55.655536 22934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 61002.6us
I20260709 13:21:55.656391 22934 leader_election.cc:304] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68
I20260709 13:21:55.656384 23621 raft_consensus.cc:515] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.659698 23672 raft_consensus.cc:3055] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:55.660104 23672 raft_consensus.cc:740] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:55.661033 14429 tablet_server.cc:179] TabletServer@127.14.23.65:0 shutting down...
I20260709 13:21:55.661469 23621 leader_election.cc:290] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 3 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:55.662729 23569 raft_consensus.cc:515] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:55.664932 23569 leader_election.cc:290] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:55.672016 23663 raft_consensus.cc:2804] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:55.676998 23751 raft_consensus.cc:2749] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323) with higher term. Message: Invalid argument: T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 for earlier term 2. Current term is 3.
I20260709 13:21:55.681090 23594 raft_consensus.cc:493] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:55.660987 23672 consensus_queue.cc:260] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.700088 23663 raft_consensus.cc:697] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 3 LEADER]: Becoming Leader. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
W20260709 13:21:55.721395 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 203456us
I20260709 13:21:55.729576 22888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e09daa4f2dd474d96ae9c4dfa1a8758" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:55.730368 22888 raft_consensus.cc:2468] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 2.
I20260709 13:21:55.730618 23008 leader_election.cc:304] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:55.731583 23782 raft_consensus.cc:2804] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:55.731959 23782 raft_consensus.cc:493] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:55.732290 23782 raft_consensus.cc:3060] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:55.737594 23772 raft_consensus.cc:493] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:55.738314 23772 raft_consensus.cc:515] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.740550 23772 leader_election.cc:290] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:55.709443 23594 raft_consensus.cc:515] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.743521 23594 leader_election.cc:290] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:55.744470 23663 consensus_queue.cc:237] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.745254 23780 raft_consensus.cc:493] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:55.746317 23782 raft_consensus.cc:515] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:55.746716 23682 raft_consensus.cc:515] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:55.748620 23782 leader_election.cc:290] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:55.748775 23682 leader_election.cc:290] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:55.748986 22818 raft_consensus.cc:2368] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 for earlier term 2. Current term is 3.
I20260709 13:21:55.753281 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e230e64c6984b72a2ba543f2a017e25" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:55.721869 23034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b154956cc17f4ba68f906daffbcedff6" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:55.754077 22960 raft_consensus.cc:3060] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.754251 23034 raft_consensus.cc:3060] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.757622 23008 leader_election.cc:304] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:55.704520 23756 raft_consensus.cc:515] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.758767 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e09daa4f2dd474d96ae9c4dfa1a8758" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:55.774972 23041 raft_consensus.cc:2468] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2.
I20260709 13:21:55.776050 23756 leader_election.cc:290] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:55.777138 22858 leader_election.cc:304] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:55.778069 23699 raft_consensus.cc:2804] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:21:55.778175 22961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8b119f5c5844d1be439423b333ec60" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:55.778926 22961 raft_consensus.cc:2468] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:55.780599 22962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955a6d8dbbe94ef9be9c1f69b10675d8" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:55.781237 22962 raft_consensus.cc:3060] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.782143 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:55.778498 23699 raft_consensus.cc:493] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.783020 23699 raft_consensus.cc:3060] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.786332 23036 raft_consensus.cc:1275] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:55.788949 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:55.745749 23780 raft_consensus.cc:515] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.789589 23041 raft_consensus.cc:2468] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 3.
I20260709 13:21:55.760970 22889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968e16cef40146509d654274b9fd2687" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:55.790267 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:55.761219 23621 raft_consensus.cc:2804] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:55.790935 22968 raft_consensus.cc:2468] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 2.
I20260709 13:21:55.791532 23621 raft_consensus.cc:697] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 3 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:55.762197 22886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955a6d8dbbe94ef9be9c1f69b10675d8" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:55.764537 23690 raft_consensus.cc:2749] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509) with higher term. Message: Invalid argument: T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 for earlier term 2. Current term is 3.
I20260709 13:21:55.792853 22886 raft_consensus.cc:3060] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.792562 23621 consensus_queue.cc:237] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.792558 23039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e09daa4f2dd474d96ae9c4dfa1a8758" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:55.797081 23039 raft_consensus.cc:2468] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 2.
I20260709 13:21:55.768422 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968e16cef40146509d654274b9fd2687" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:55.798127 23043 raft_consensus.cc:3060] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.800428 22889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e09daa4f2dd474d96ae9c4dfa1a8758" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:55.762221 23566 leader_election.cc:290] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:55.762251 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:55.801962 23780 leader_election.cc:290] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:55.757108 22785 leader_election.cc:304] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:55.802416 22969 raft_consensus.cc:2468] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 2.
W20260709 13:21:55.802801 22785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 45779.5us
I20260709 13:21:55.803409 22783 leader_election.cc:304] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:55.771951 23525 raft_consensus.cc:2749] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155) with higher term. Message: Invalid argument: T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 for earlier term 2. Current term is 3.
I20260709 13:21:55.760888 22963 raft_consensus.cc:2468] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 3.
I20260709 13:21:55.788851 23556 consensus_queue.cc:1048] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:55.790985 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:55.765542 23672 raft_consensus.cc:3060] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.772828 22960 raft_consensus.cc:2468] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 3.
I20260709 13:21:55.804158 22785 leader_election.cc:304] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:55.827878 22785 leader_election.cc:304] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:55.832793 22969 raft_consensus.cc:1240] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 2. Current term is 3. Ops: []
I20260709 13:21:55.804541 23690 raft_consensus.cc:2804] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:21:55.818009 22814 raft_consensus.cc:2468] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 2.
I20260709 13:21:55.840057 23770 raft_consensus.cc:2804] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:21:55.842494 22785 leader_election.cc:304] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:55.836774 23772 raft_consensus.cc:2804] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:21:55.837904 23690 raft_consensus.cc:493] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.840415 23039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8b119f5c5844d1be439423b333ec60" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:55.843442 23770 raft_consensus.cc:493] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:55.860749 22934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 153578us
I20260709 13:21:55.830085 23771 raft_consensus.cc:515] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
W20260709 13:21:55.862883 22860 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 103338us
I20260709 13:21:55.853880 23772 raft_consensus.cc:493] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.856860 23690 raft_consensus.cc:3060] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.863204 23770 raft_consensus.cc:3060] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:55.880553 23775 raft_consensus.cc:2804] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:55.867666 23039 raft_consensus.cc:2468] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 2.
I20260709 13:21:55.869764 23772 raft_consensus.cc:3060] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.887547 23546 consensus_queue.cc:1059] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:55.888510 23594 raft_consensus.cc:3055] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:55.888911 23594 raft_consensus.cc:740] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:55.889706 23035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:55.890472 23035 raft_consensus.cc:2468] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:55.892287 23771 leader_election.cc:290] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:55.879419 23774 raft_consensus.cc:2804] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:21:55.894351 23774 raft_consensus.cc:493] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.894716 23774 raft_consensus.cc:3060] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.895922 23042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:55.896654 23042 raft_consensus.cc:2468] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 1.
I20260709 13:21:55.896989 23034 raft_consensus.cc:2468] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 3.
I20260709 13:21:55.881162 23775 raft_consensus.cc:697] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 3 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:55.901731 23041 raft_consensus.cc:2468] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 3.
I20260709 13:21:55.906111 22858 leader_election.cc:304] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:55.872470 22886 raft_consensus.cc:2468] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 3.
I20260709 13:21:55.914644 23790 raft_consensus.cc:2804] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:21:55.925319 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 119686us
I20260709 13:21:55.935019 22785 leader_election.cc:304] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:55.933979 22932 leader_election.cc:304] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:55.932878 23008 leader_election.cc:304] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:55.940032 23008 leader_election.cc:304] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:55.941373 23037 raft_consensus.cc:1240] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 2. Current term is 3. Ops: []
I20260709 13:21:55.941699 23781 raft_consensus.cc:2804] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:21:55.942260 23781 raft_consensus.cc:493] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.942595 23781 raft_consensus.cc:3060] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.938654 23043 raft_consensus.cc:2468] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 3.
I20260709 13:21:55.945343 22962 raft_consensus.cc:2468] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 3.
I20260709 13:21:55.951964 23772 raft_consensus.cc:515] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
W20260709 13:21:55.941810 23785 log.cc:927] Time spent T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:21:55.954875 23621 raft_consensus.cc:2804] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:55.955334 23621 raft_consensus.cc:493] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:55.955713 23621 raft_consensus.cc:3060] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:55.956204 22962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8b119f5c5844d1be439423b333ec60" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:55.958251 22962 raft_consensus.cc:3060] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.960299 23774 raft_consensus.cc:515] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.962846 23770 raft_consensus.cc:515] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:55.963325 23682 raft_consensus.cc:2804] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:55.963896 23682 raft_consensus.cc:697] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 3 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
W20260709 13:21:55.953740 22934 leader_election.cc:336] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:55.922295 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:55.965104 22934 leader_election.cc:304] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:55.968557 23682 consensus_queue.cc:237] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:55.889735 23594 consensus_queue.cc:260] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:55.971220 23594 raft_consensus.cc:3060] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:55.967216 22932 leader_election.cc:304] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:55.964787 22969 raft_consensus.cc:1240] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Rejecting Update request from peer 27745ef236cd494684f62a1269411a68 for earlier term 2. Current term is 3. Ops: []
I20260709 13:21:55.930963 23790 raft_consensus.cc:697] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
W20260709 13:21:55.975431 23771 consensus_queue.cc:1175] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "27745ef236cd494684f62a1269411a68" responder_term: 3 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:21:55.976679 23790 consensus_queue.cc:237] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:55.982429 22887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:55.983405 22887 raft_consensus.cc:2393] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 in current term 3: Already voted for candidate 49cbf11e18ce4268a7fcca76872afb23 in this term.
I20260709 13:21:55.987353 23569 raft_consensus.cc:2804] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:21:55.970338 22934 consensus_peers.cc:597] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:55.988897 22717 catalog_manager.cc:5697] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } }
W20260709 13:21:55.989015 22934 leader_election.cc:336] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:56.001849 22934 leader_election.cc:304] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
W20260709 13:21:56.003299 22934 leader_election.cc:336] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:56.008904 22860 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 67677.5us
I20260709 13:21:55.962764 23690 raft_consensus.cc:515] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:55.987506 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:56.001737 22962 raft_consensus.cc:2468] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 3.
I20260709 13:21:56.009606 23596 raft_consensus.cc:2749] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:21:55.993292 23569 raft_consensus.cc:697] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 3 LEADER]: Becoming Leader. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:21:56.059737 23631 raft_consensus.cc:2804] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:56.034431 23772 leader_election.cc:290] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:56.062661 23774 leader_election.cc:290] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
W20260709 13:21:56.004350 22934 consensus_peers.cc:597] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:56.194890 22934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 190595us
I20260709 13:21:56.165161 22816 raft_consensus.cc:2468] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 3.
W20260709 13:21:56.165558 22860 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 21 similar messages]
I20260709 13:21:55.930429 23775 consensus_queue.cc:237] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
W20260709 13:21:56.173102 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 162746us
I20260709 13:21:56.209945 22860 leader_election.cc:400] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Vote denied by peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155) with higher term. Message: Invalid argument: T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 for earlier term 2. Current term is 3.
I20260709 13:21:56.210337 22860 leader_election.cc:403] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:21:56.203078 23631 raft_consensus.cc:697] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 3 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:56.214255 23034 raft_consensus.cc:1240] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Rejecting Update request from peer 49cbf11e18ce4268a7fcca76872afb23 for earlier term 2. Current term is 3. Ops: []
I20260709 13:21:56.214673 23631 consensus_queue.cc:237] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:56.215371 23569 consensus_queue.cc:237] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
W20260709 13:21:56.216771 23007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:56.220568 23621 raft_consensus.cc:515] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:56.221503 23770 leader_election.cc:290] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:56.197839 22960 raft_consensus.cc:2468] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 3.
I20260709 13:21:56.197350 23756 raft_consensus.cc:2749] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:56.209244 23781 raft_consensus.cc:515] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:56.225400 23779 consensus_queue.cc:1059] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:21:56.226413 23779 raft_consensus.cc:3055] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:56.231163 22860 leader_election.cc:304] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:56.232946 22785 leader_election.cc:304] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:56.238190 22785 leader_election.cc:304] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:56.250895 23557 raft_consensus.cc:2804] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:56.230773 23779 raft_consensus.cc:740] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:56.252552 23779 consensus_queue.cc:260] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:56.253808 23779 raft_consensus.cc:3060] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:56.225430 23781 leader_election.cc:290] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:56.236840 23568 raft_consensus.cc:2749] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155) with higher term. Message: Invalid argument: T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 for earlier term 2. Current term is 3.
I20260709 13:21:56.260089 22963 raft_consensus.cc:2393] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 in current term 3: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:56.256528 23770 raft_consensus.cc:2804] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:56.232071 23586 consensus_queue.cc:1059] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:21:56.251066 23621 leader_election.cc:290] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:56.261794 22887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:56.263657 23682 raft_consensus.cc:2804] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:21:56.336373 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 131441us
I20260709 13:21:56.309860 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:56.349588 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:56.350337 22969 raft_consensus.cc:2468] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 3.
I20260709 13:21:56.333590 23770 raft_consensus.cc:697] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 3 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:56.335503 23682 raft_consensus.cc:697] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 3 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:56.341078 23663 raft_consensus.cc:493] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:56.350615 23043 raft_consensus.cc:3060] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:21:56.357261 23699 consensus_meta.cc:287] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23: Time spent flushing consensus metadata: real 0.574s	user 0.011s	sys 0.011s
I20260709 13:21:56.372164 23557 raft_consensus.cc:697] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:56.373739 23690 leader_election.cc:290] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:56.402305 23770 consensus_queue.cc:237] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:56.405020 22969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:56.368636 23663 raft_consensus.cc:515] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:56.372383 23699 raft_consensus.cc:515] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:56.386476 23038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6972fb562de24785b0feffe6bcd061c5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:56.413537 23040 raft_consensus.cc:1275] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Refusing update from remote peer 8ef3887fb3874f8b81079e542d252944: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
W20260709 13:21:56.413600 22934 consensus_peers.cc:597] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:56.426558 22968 raft_consensus.cc:1240] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 3 LEADER]: Rejecting Update request from peer 49cbf11e18ce4268a7fcca76872afb23 for earlier term 2. Current term is 3. Ops: []
I20260709 13:21:56.418673 23040 pending_rounds.cc:85] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68: Aborting all ops after (but not including) 1
I20260709 13:21:56.429226 23040 pending_rounds.cc:107] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:21:56.429556 23040 raft_consensus.cc:2889] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:56.431633 23557 consensus_queue.cc:237] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:56.438201 23569 consensus_queue.cc:1048] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:21:56.450908 22860 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:43011, user_credentials={real_user=slave}} blocked reactor thread for 49478us
I20260709 13:21:56.424322 23036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
W20260709 13:21:56.465883 22858 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.68:44509, user_credentials={real_user=slave}} blocked reactor thread for 203738us
I20260709 13:21:56.466801 22889 raft_consensus.cc:2368] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 for earlier term 2. Current term is 3.
I20260709 13:21:56.458398 23546 raft_consensus.cc:493] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
W20260709 13:21:56.468973 22860 consensus_peers.cc:597] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:56.477839 22860 leader_election.cc:336] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:56.479969 22860 consensus_peers.cc:597] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:56.419624 23682 consensus_queue.cc:237] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
W20260709 13:21:56.481159 22860 consensus_peers.cc:597] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:56.491009 22860 consensus_peers.cc:597] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:56.478778 23546 raft_consensus.cc:515] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
W20260709 13:21:56.492962 22860 leader_election.cc:336] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:56.506964 22860 leader_election.cc:304] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:56.511246 23586 raft_consensus.cc:2804] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:21:56.513975 23546 leader_election.cc:290] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:56.521011 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7802e4d0fd394596b42b661c9eb4c7ce" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:56.521678 23043 raft_consensus.cc:2468] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 3.
I20260709 13:21:56.528455 22783 leader_election.cc:304] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:56.529193 23699 leader_election.cc:290] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:56.529613 23775 raft_consensus.cc:2749] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:21:56.530893 22932 leader_election.cc:304] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
W20260709 13:21:56.535599 22860 leader_election.cc:336] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:56.518277 23586 raft_consensus.cc:493] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:56.538160 23586 raft_consensus.cc:3060] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:56.532065 23038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e09daa4f2dd474d96ae9c4dfa1a8758" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
W20260709 13:21:56.536353 22860 consensus_peers.cc:597] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:56.536805 23799 raft_consensus.cc:2804] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:21:56.543170 23008 consensus_peers.cc:597] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:56.542659 23038 raft_consensus.cc:3060] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:56.548278 23043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7802e4d0fd394596b42b661c9eb4c7ce" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
W20260709 13:21:56.545130 23008 leader_election.cc:336] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:56.545305 22860 consensus_peers.cc:597] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:56.539866 23663 leader_election.cc:290] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
W20260709 13:21:56.555755 22860 consensus_peers.cc:597] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:56.548964 23568 raft_consensus.cc:3055] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:56.556991 23568 raft_consensus.cc:740] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:56.558579 23568 consensus_queue.cc:260] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:56.559746 23568 raft_consensus.cc:3060] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:56.562114 23799 raft_consensus.cc:697] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 3 LEADER]: Becoming Leader. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:21:56.563161 23799 consensus_queue.cc:237] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:56.564043 23586 raft_consensus.cc:515] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
W20260709 13:21:56.568734 23659 log.cc:927] Time spent T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c: Append to log took a long time: real 0.090s	user 0.001s	sys 0.001s
I20260709 13:21:56.570063 23775 raft_consensus.cc:493] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:56.570667 23775 raft_consensus.cc:515] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
W20260709 13:21:56.572172 22934 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 148111us
I20260709 13:21:56.576783 23038 raft_consensus.cc:2468] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 3.
I20260709 13:21:56.579751 23040 raft_consensus.cc:1275] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:21:56.580302 22860 consensus_peers.cc:597] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:56.582422 23568 consensus_queue.cc:1048] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:56.584328 23770 raft_consensus.cc:493] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:56.584897 23770 raft_consensus.cc:515] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:56.586403 22887 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505eab84b06f46ee9d336abf01353dee" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:56.587514 23770 leader_election.cc:290] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
W20260709 13:21:56.592011 23811 log.cc:927] Time spent T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:21:56.592312 22858 leader_election.cc:304] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
W20260709 13:21:56.554055 23008 consensus_peers.cc:597] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:56.593406 23008 leader_election.cc:336] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:56.594554 23008 leader_election.cc:336] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:56.595427 23008 consensus_peers.cc:597] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:56.597136 23008 consensus_peers.cc:597] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:56.591430 23770 raft_consensus.cc:493] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:56.595804 22860 consensus_peers.cc:597] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:56.589715 23039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8b119f5c5844d1be439423b333ec60" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
W20260709 13:21:56.612062 22860 consensus_peers.cc:597] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:56.601150 23568 raft_consensus.cc:2804] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:56.614481 23039 raft_consensus.cc:3060] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:56.616467 23594 raft_consensus.cc:493] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
W20260709 13:21:56.615407 22860 consensus_peers.cc:597] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:56.628664 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0abfe80c1fa642ec8a0d075f75264816" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:56.632665 22891 raft_consensus.cc:1275] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:56.632877 23774 raft_consensus.cc:493] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:56.621546 23568 raft_consensus.cc:697] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:56.633507 23774 raft_consensus.cc:515] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:56.634685 23568 consensus_queue.cc:237] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:56.635761 23774 leader_election.cc:290] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:56.636336 23774 raft_consensus.cc:493] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:56.636785 23774 raft_consensus.cc:515] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:56.626273 23594 raft_consensus.cc:515] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:56.638955 22816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49344 (request call id 190) took 1103 ms. Trace:
I20260709 13:21:56.638500 22783 leader_election.cc:304] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:56.639393 22816 rpcz_store.cc:276] 0709 13:21:55.552895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:55.556867 (+  3972us) service_pool.cc:224] Handling call
0709 13:21:56.638770 (+1081903us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b540052c1d4. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ce4974571 00007f5ce4a0f2fa 00007f5ce99d0b17 00007f5ce2f4b6d4 00007f5ce2f4b662 00007f5ce2f4b591 00007f5ce2f4b50c 00007f5ce2f4a7c2 00007f5ce2d4648c 00007f5ce2d458c6 00007f5ce2d4526f
0709 13:21:56.638876 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":456190336}
I20260709 13:21:56.640376 23594 leader_election.cc:290] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:56.611008 23770 raft_consensus.cc:515] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:56.640914 23594 raft_consensus.cc:2749] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:56.644318 22963 raft_consensus.cc:1275] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:56.646991 23775 leader_election.cc:290] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:56.648595 22888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e09daa4f2dd474d96ae9c4dfa1a8758" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
W20260709 13:21:56.650149 22934 consensus_peers.cc:597] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:56.649422 23039 raft_consensus.cc:2468] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 3.
I20260709 13:21:56.656340 23586 leader_election.cc:290] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 4 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:56.647038 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0abfe80c1fa642ec8a0d075f75264816" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:56.660876 23036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e09daa4f2dd474d96ae9c4dfa1a8758" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:56.661623 23770 leader_election.cc:290] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:56.665947 23774 leader_election.cc:290] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
W20260709 13:21:56.667577 22934 leader_election.cc:336] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:56.669063 22934 consensus_peers.cc:597] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:56.625854 22860 consensus_peers.cc:597] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:56.661772 23036 raft_consensus.cc:2393] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 in current term 3: Already voted for candidate 49cbf11e18ce4268a7fcca76872afb23 in this term.
W20260709 13:21:56.671715 22860 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:43155, user_credentials={real_user=slave}} blocked reactor thread for 50730us
I20260709 13:21:56.671130 22785 leader_election.cc:304] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:56.677810 22785 leader_election.cc:304] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
W20260709 13:21:56.675647 22860 consensus_peers.cc:597] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:56.676362 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "084ee2ebbf564ed583f84c1227d29bd4" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
W20260709 13:21:56.680107 22860 consensus_peers.cc:597] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:56.682387 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "084ee2ebbf564ed583f84c1227d29bd4" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
W20260709 13:21:56.682579 22860 consensus_peers.cc:597] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:56.683239 22860 consensus_peers.cc:597] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:56.683394 23594 raft_consensus.cc:2749] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:56.683841 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:56.678436 22889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1394978547a410199b4f798a10edb2d" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:56.686090 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:56.674573 23775 raft_consensus.cc:2749] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:21:56.683779 22860 leader_election.cc:336] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:56.671380 23774 consensus_queue.cc:1048] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:21:56.683782 22961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1394978547a410199b4f798a10edb2d" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:56.686108 22894 raft_consensus.cc:2468] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 3.
I20260709 13:21:56.696961 23770 raft_consensus.cc:493] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:56.700069 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:56.700675 22968 raft_consensus.cc:3060] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:56.701611 23008 leader_election.cc:400] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: Vote denied by peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323) with higher term. Message: Invalid argument: T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 for earlier term 2. Current term is 3.
I20260709 13:21:56.702003 23008 leader_election.cc:403] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:21:56.693723 23035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b783c62bd54d4c8f4f1795b7409958" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:56.695000 23037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:56.706849 22783 leader_election.cc:304] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:56.708084 22783 leader_election.cc:304] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:56.708158 23801 raft_consensus.cc:2749] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323) with higher term. Message: Invalid argument: T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 for earlier term 2. Current term is 3.
I20260709 13:21:56.709734 23770 raft_consensus.cc:515] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:56.710500 23690 raft_consensus.cc:2749] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:21:56.711130 23690 raft_consensus.cc:2749] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:21:56.703766 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:56.711951 23770 leader_election.cc:290] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:56.712970 23037 raft_consensus.cc:2468] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 3.
I20260709 13:21:56.713853 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50b783c62bd54d4c8f4f1795b7409958" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:56.697996 22960 raft_consensus.cc:1275] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Refusing update from remote peer 27745ef236cd494684f62a1269411a68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:56.704195 22785 leader_election.cc:304] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:56.699509 23772 consensus_queue.cc:1048] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:21:56.718641 23770 raft_consensus.cc:2804] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:21:56.721226 23770 raft_consensus.cc:493] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:56.722640 23770 raft_consensus.cc:3060] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:56.723904 23036 raft_consensus.cc:1275] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Refusing update from remote peer 8ef3887fb3874f8b81079e542d252944: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:56.725872 23775 raft_consensus.cc:493] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:56.726521 23775 raft_consensus.cc:515] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
W20260709 13:21:56.728659 23816 log.cc:927] Time spent T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
W20260709 13:21:56.725534 22860 leader_election.cc:336] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:56.729815 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e7b082dcf344fbf83e0ddec4b08b6de" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
W20260709 13:21:56.716367 22934 leader_election.cc:336] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:56.733815 22934 leader_election.cc:304] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:56.732690 22785 leader_election.cc:304] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:56.730741 23663 consensus_queue.cc:1048] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:21:56.715198 22968 raft_consensus.cc:2468] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 4.
I20260709 13:21:56.728713 23775 leader_election.cc:290] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:56.754544 22785 leader_election.cc:304] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:56.741161 23756 raft_consensus.cc:2749] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:21:56.741099 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e7b082dcf344fbf83e0ddec4b08b6de" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:56.755934 22860 leader_election.cc:304] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:56.754897 23775 raft_consensus.cc:2749] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:21:56.761219 23008 consensus_peers.cc:597] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:56.767055 22890 raft_consensus.cc:1275] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Refusing update from remote peer 8ef3887fb3874f8b81079e542d252944: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:21:56.767550 22890 pending_rounds.cc:85] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23: Aborting all ops after (but not including) 1
I20260709 13:21:56.767773 22890 pending_rounds.cc:107] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:21:56.767997 22890 raft_consensus.cc:2889] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:56.770865 23096 raft_consensus.cc:2804] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:21:56.771416 23096 raft_consensus.cc:697] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:56.772190 23096 consensus_queue.cc:237] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:56.783453 23663 consensus_queue.cc:1048] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:56.801124 23775 raft_consensus.cc:2749] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:21:56.802412 23775 raft_consensus.cc:493] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:56.802901 23775 raft_consensus.cc:515] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:56.803474 23594 raft_consensus.cc:493] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:56.803994 23594 raft_consensus.cc:515] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:56.805398 23775 leader_election.cc:290] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:56.806027 23594 leader_election.cc:290] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:56.807978 23037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10066ccf8d174f4a9dd69253613857b8" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
W20260709 13:21:56.808979 22785 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 54495.6us
I20260709 13:21:56.809855 23037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd78248be3954373ad4fd60b8ae7593d" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:56.811632 23037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a064be2ec640a4a8283957d0b9b98c" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
W20260709 13:21:56.811935 22934 consensus_peers.cc:597] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:56.813501 23774 raft_consensus.cc:493] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:56.813657 22889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a064be2ec640a4a8283957d0b9b98c" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:56.814147 23774 raft_consensus.cc:515] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:56.816433 23774 leader_election.cc:290] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:56.812017 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10066ccf8d174f4a9dd69253613857b8" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:56.818099 22785 leader_election.cc:304] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:56.820998 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd78248be3954373ad4fd60b8ae7593d" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:56.821413 23672 consensus_queue.cc:1048] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:56.838474 23774 raft_consensus.cc:2749] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:56.839529 22785 leader_election.cc:304] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:56.840644 23774 raft_consensus.cc:2749] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:56.847693 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39611d2ff3524bc0840e4fd222d96a84" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:56.860438 22783 leader_election.cc:304] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:56.861774 22783 leader_election.cc:304] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:56.862505 23821 raft_consensus.cc:2749] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:56.862972 22783 leader_election.cc:304] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:56.862830 23774 raft_consensus.cc:2749] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:21:56.863938 23008 consensus_peers.cc:597] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:56.865875 23008 consensus_peers.cc:597] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:56.866480 23008 leader_election.cc:336] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:56.871299 23008 leader_election.cc:304] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
W20260709 13:21:56.872506 23008 consensus_peers.cc:597] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:56.877521 23672 raft_consensus.cc:2749] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:56.867187 23772 raft_consensus.cc:2749] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:21:56.885598 23037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39611d2ff3524bc0840e4fd222d96a84" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:56.867213 23770 raft_consensus.cc:515] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:56.899900 23770 leader_election.cc:290] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:56.902652 22717 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50724 (request call id 68) took 1374 ms. Trace:
I20260709 13:21:56.911178 22785 leader_election.cc:304] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:56.904500 22715 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50740 (request call id 80) took 1222 ms. Trace:
I20260709 13:21:56.911981 22715 rpcz_store.cc:276] 0709 13:21:55.697895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:55.703099 (+  5204us) service_pool.cc:224] Handling call
0709 13:21:56.904328 (+1201229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8977,"reported_tablets":1}
I20260709 13:21:56.914523 23770 raft_consensus.cc:2749] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:56.916464 22718 catalog_manager.cc:5697] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 1 to 3, leader changed from 8ef3887fb3874f8b81079e542d252944 (127.14.23.67) to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 3 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:56.920095 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:56.920768 22894 raft_consensus.cc:3060] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:56.920828 23775 raft_consensus.cc:493] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:56.921377 23775 raft_consensus.cc:515] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:56.923503 23775 leader_election.cc:290] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:56.926469 22718 catalog_manager.cc:5697] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:56.927167 23775 raft_consensus.cc:493] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:56.927840 23775 raft_consensus.cc:515] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:56.928205 22718 catalog_manager.cc:5697] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 2 to 3, leader changed from 27745ef236cd494684f62a1269411a68 (127.14.23.68) to 8ef3887fb3874f8b81079e542d252944 (127.14.23.67). New cstate: current_term: 3 leader_uuid: "8ef3887fb3874f8b81079e542d252944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } }
I20260709 13:21:56.930343 22894 raft_consensus.cc:2468] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 4.
I20260709 13:21:56.930593 23775 leader_election.cc:290] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:56.932166 23037 raft_consensus.cc:1275] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:56.932950 23775 raft_consensus.cc:493] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:56.932578 23036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:56.934320 23036 raft_consensus.cc:3060] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:56.911175 22717 rpcz_store.cc:276] 0709 13:21:55.528065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:55.528221 (+   156us) service_pool.cc:224] Handling call
0709 13:21:56.902556 (+1374335us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:56.010010 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:56.412696 (+402686us) spinlock_profiling.cc:243] Waited 24 ms on lock 0x7b5c0040308c. stack: 00007f5ce18ec343 00007f5ce18ec240 00007f5ce127b1d4 00007f5ceafb9304 00007f5ceafb9299 00007f5ceabfd331 00007f5ceabfd221 00007f5ceabf7f6c 00007f5ce8455d69 00007f5ce84410a6 00007f5ce8440d58 00007f5ce84459f1 00007f5ce84459a9 00007f5ce8445939 00007f5ce8445901 00007f5ce8444b2d
0709 13:21:56.412769 (+    73us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:56.412802 (+    33us) write_op.cc:435] Acquired schema lock
0709 13:21:56.412819 (+    17us) tablet.cc:662] Decoding operations
0709 13:21:56.413211 (+   392us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:56.413272 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:56.413297 (+    25us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:21:56.413493 (+   196us) tablet.cc:701] Row locks acquired
0709 13:21:56.413510 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:21:56.413651 (+   141us) write_op.cc:270] Start()
0709 13:21:56.413844 (+   193us) write_op.cc:276] Timestamp: P: 1783603316413607 usec, L: 0
0709 13:21:56.413868 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:21:56.469891 (+ 56023us) log.cc:844] Serialized 537 byte log entry
0709 13:21:56.716281 (+246390us) op_driver.cc:464] REPLICATION: finished
0709 13:21:56.858399 (+142118us) write_op.cc:301] APPLY: starting
0709 13:21:56.858483 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:56.858579 (+    96us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:56.858607 (+    28us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:56.858824 (+   217us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:56.858974 (+   150us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:21:56.859016 (+    42us) write_op.cc:312] APPLY: finished
0709 13:21:56.863248 (+  4232us) log.cc:844] Serialized 22 byte log entry
0709 13:21:56.866894 (+  3646us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:56.870853 (+  3959us) write_op.cc:454] Released schema lock
0709 13:21:56.874487 (+  3634us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:56.891916 (+ 17429us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:56.891952 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":15637,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":140934,"apply.run_cpu_time_us":1641,"apply.run_wall_time_us":33506,"num_ops":1,"prepare.queue_time_us":138,"prepare.run_cpu_time_us":18651,"prepare.run_wall_time_us":480231,"replication_time_us":290197,"spinlock_wait_cycles":55245568,"thread_start_us":132814,"threads_started":1}]]}
I20260709 13:21:56.949506 23594 raft_consensus.cc:493] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:56.950188 23594 raft_consensus.cc:515] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:56.952639 23775 raft_consensus.cc:515] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:56.953176 23772 raft_consensus.cc:493] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 27745ef236cd494684f62a1269411a68)
I20260709 13:21:56.953784 23772 raft_consensus.cc:515] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:56.955808 23772 leader_election.cc:290] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:56.957607 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1934129c8764c018495eff97d43a3b5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:56.957906 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630a5e60917545b3a8072fc8358207c5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:56.959775 23594 leader_election.cc:290] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:56.960709 23034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1934129c8764c018495eff97d43a3b5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:56.961061 23037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630a5e60917545b3a8072fc8358207c5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:56.962349 23035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52d63f523d24195bc8127113598ff91" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:56.963609 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f631a34c02c4de8ae5bd70760fba972" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:56.965862 23594 raft_consensus.cc:493] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:56.967470 23594 raft_consensus.cc:515] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:56.968245 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086a7123d3ad43d79101a7c5de5158fe" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:56.969177 23036 raft_consensus.cc:2468] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 4.
I20260709 13:21:56.971236 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086a7123d3ad43d79101a7c5de5158fe" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:56.977259 22714 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50708 (request call id 75) took 1581 ms. Trace:
I20260709 13:21:56.977854 22714 rpcz_store.cc:276] 0709 13:21:55.395610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:55.395756 (+   146us) service_pool.cc:224] Handling call
0709 13:21:56.977184 (+1581428us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66183,"reported_tablets":9}
I20260709 13:21:56.979408 22785 leader_election.cc:304] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:21:56.980356 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f631a34c02c4de8ae5bd70760fba972" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:56.980682 23594 leader_election.cc:290] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:56.982803 23770 raft_consensus.cc:2804] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:21:56.983630 23770 raft_consensus.cc:697] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 4 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:56.984148 23772 consensus_queue.cc:1048] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:56.986472 23770 consensus_queue.cc:237] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:56.966248 23775 leader_election.cc:290] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:56.992210 22785 leader_election.cc:304] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:56.993498 22785 leader_election.cc:304] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:56.993626 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52d63f523d24195bc8127113598ff91" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:56.994107 23594 raft_consensus.cc:2749] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:56.995563 23775 raft_consensus.cc:2749] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:21:56.997397 22785 leader_election.cc:304] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:57.005556 23594 raft_consensus.cc:2749] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:57.011950 23770 raft_consensus.cc:493] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:57.012684 23770 raft_consensus.cc:515] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:57.014580 22785 leader_election.cc:304] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:57.015226 23770 leader_election.cc:290] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:57.015576 23772 raft_consensus.cc:2749] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:57.018553 22962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:57.019418 22962 raft_consensus.cc:2393] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 in current term 3: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:57.020147 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:57.020311 22785 leader_election.cc:304] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:57.020901 23594 raft_consensus.cc:493] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:57.021436 23682 raft_consensus.cc:2749] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:57.021534 14429 tablet_replica.cc:333] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.022981 14429 raft_consensus.cc:2243] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.023592 14429 raft_consensus.cc:2272] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.021513 23594 raft_consensus.cc:515] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:57.027794 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:57.029369 23774 raft_consensus.cc:493] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:57.027498 22960 raft_consensus.cc:1275] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:57.029942 23774 raft_consensus.cc:515] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:57.032280 23774 leader_election.cc:290] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:57.032127 23775 consensus_queue.cc:1048] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:57.026428 23594 leader_election.cc:290] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:57.027741 23770 raft_consensus.cc:493] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:57.033722 23770 raft_consensus.cc:515] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:57.035316 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3139446c554b419fc28cad31653600" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:57.036186 23770 leader_election.cc:290] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:21:57.027631 23772 raft_consensus.cc:493] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8ef3887fb3874f8b81079e542d252944)
I20260709 13:21:57.037714 23772 raft_consensus.cc:515] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:57.038189 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8be856e3d604f8d8a1c56a88ea18643" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:57.039501 22962 raft_consensus.cc:1275] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:57.042025 14429 tablet_replica.cc:333] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.041692 23770 consensus_queue.cc:1048] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:21:57.044399 23772 leader_election.cc:290] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:57.046448 14429 raft_consensus.cc:2243] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.046994 14429 raft_consensus.cc:2272] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.047585 22962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1e18beffe450f87ff940a4e9165ea" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:57.049242 14429 tablet_replica.cc:333] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.050446 14429 raft_consensus.cc:2243] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:21:57.053107 14429 pending_rounds.cc:70] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679: Trying to abort 1 pending ops.
I20260709 13:21:57.053375 14429 pending_rounds.cc:77] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679: Aborting op as it isn't in flight: id { term: 3 index: 2 } timestamp: 7305639183994253312 op_type: NO_OP noop_request { }
I20260709 13:21:57.053761 14429 raft_consensus.cc:2889] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
W20260709 13:21:57.041723 22860 consensus_peers.cc:597] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:57.037137 22718 catalog_manager.cc:5697] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } }
I20260709 13:21:57.054523 14429 raft_consensus.cc:2272] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.061017 22889 raft_consensus.cc:1240] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 LEADER]: Rejecting Update request from peer 91f33bdbdd63438294071eb3776db679 for earlier term 3. Current term is 4. Ops: []
I20260709 13:21:57.061717 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:57.064307 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3139446c554b419fc28cad31653600" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:57.059208 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014651e80f9f44e599d2d8e4d308d2a6" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:57.070833 14429 tablet_replica.cc:333] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.060981 22718 catalog_manager.cc:5697] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 1 to 3, leader changed from 27745ef236cd494684f62a1269411a68 (127.14.23.68) to 8ef3887fb3874f8b81079e542d252944 (127.14.23.67). New cstate: current_term: 3 leader_uuid: "8ef3887fb3874f8b81079e542d252944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:57.071884 14429 raft_consensus.cc:2243] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.072479 14429 raft_consensus.cc:2272] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.072060 22718 catalog_manager.cc:5697] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "8ef3887fb3874f8b81079e542d252944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:57.072891 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1e18beffe450f87ff940a4e9165ea" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:57.075492 14429 tablet_replica.cc:333] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.076841 14429 raft_consensus.cc:2243] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:57.077884 14429 raft_consensus.cc:2272] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.077826 23774 raft_consensus.cc:493] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:57.078460 23774 raft_consensus.cc:515] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:57.082551 23774 leader_election.cc:290] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:57.084182 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ea5c83aada45d3a98027069e96658c" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:57.087045 23772 consensus_queue.cc:1059] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:57.087800 23772 consensus_queue.cc:1243] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Peer 49cbf11e18ce4268a7fcca76872afb23 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:21:57.088248 23770 raft_consensus.cc:3055] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:21:57.088631 23770 raft_consensus.cc:740] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:21:57.089454 23770 consensus_queue.cc:260] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:57.090823 23770 raft_consensus.cc:3060] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:57.092034 22785 leader_election.cc:304] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:57.092926 23772 raft_consensus.cc:2749] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:21:57.093394 23036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014651e80f9f44e599d2d8e4d308d2a6" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:57.097203 23772 raft_consensus.cc:493] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:57.098009 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:57.098868 23040 raft_consensus.cc:2410] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:21:57.099144 23036 raft_consensus.cc:1275] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:57.097834 23772 raft_consensus.cc:515] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:57.101488 23772 leader_election.cc:290] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:57.102285 23035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8be856e3d604f8d8a1c56a88ea18643" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:57.105743 14429 tablet_replica.cc:333] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.106140 22783 leader_election.cc:304] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:57.108922 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ea5c83aada45d3a98027069e96658c" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:57.109099 23774 consensus_queue.cc:1048] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:57.108466 23770 raft_consensus.cc:3060] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:57.106562 22785 leader_election.cc:304] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:57.112619 23774 raft_consensus.cc:2749] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:21:57.107496 22714 catalog_manager.cc:5697] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 1 to 3, leader changed from 91f33bdbdd63438294071eb3776db679 (127.14.23.65) to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 3 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:57.114017 22785 leader_election.cc:304] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
W20260709 13:21:57.114976 22785 raft_consensus.cc:2686] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:21:57.115913 22785 leader_election.cc:304] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:57.116570 23772 raft_consensus.cc:2749] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:57.117318 22714 catalog_manager.cc:5697] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 1 to 3, leader changed from 91f33bdbdd63438294071eb3776db679 (127.14.23.65) to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 3 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:57.118363 14429 raft_consensus.cc:2243] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.120142 22714 catalog_manager.cc:5697] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 1 to 2, leader changed from 91f33bdbdd63438294071eb3776db679 (127.14.23.65) to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:57.121791 14429 raft_consensus.cc:2272] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.122490 22714 catalog_manager.cc:5697] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 2 to 4, leader changed from 8ef3887fb3874f8b81079e542d252944 (127.14.23.67) to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 4 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:57.125326 23770 raft_consensus.cc:2749] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:21:57.126247 23774 raft_consensus.cc:493] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:57.127871 23774 raft_consensus.cc:515] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:57.131839 23774 leader_election.cc:290] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:57.112542 22962 raft_consensus.cc:1275] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:57.133745 23699 consensus_queue.cc:1048] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:21:57.135366 23008 consensus_peers.cc:597] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:57.136583 23774 raft_consensus.cc:493] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:57.137144 23774 raft_consensus.cc:515] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:57.141171 23774 leader_election.cc:290] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:57.125427 14429 tablet_replica.cc:333] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.121142 22785 leader_election.cc:304] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:57.126906 22783 leader_election.cc:304] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:57.142439 23770 raft_consensus.cc:2749] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:21:57.143105 23770 raft_consensus.cc:2749] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:21:57.144842 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac141e204e3f4d3c84d6651893bdfc6f" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:57.145397 22962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac141e204e3f4d3c84d6651893bdfc6f" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:57.149058 23035 raft_consensus.cc:1275] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:57.149379 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "087d64074b5943f0a7d789ead06f4783" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:57.151999 23699 consensus_queue.cc:1048] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:57.155501 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "087d64074b5943f0a7d789ead06f4783" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:57.156788 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78234c90229d4615ab9e2c2a811bd0f8" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:57.158736 23774 raft_consensus.cc:493] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:21:57.159355 23774 raft_consensus.cc:515] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:57.161459 23774 leader_election.cc:290] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:57.166747 22717 catalog_manager.cc:5697] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } }
I20260709 13:21:57.168845 22717 catalog_manager.cc:5697] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } }
I20260709 13:21:57.173758 22785 leader_election.cc:304] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:57.175499 22785 leader_election.cc:304] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
I20260709 13:21:57.174290 14429 raft_consensus.cc:2243] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:57.177165 23770 raft_consensus.cc:2749] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:57.177387 23040 raft_consensus.cc:1275] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Refusing update from remote peer 8ef3887fb3874f8b81079e542d252944: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:57.177892 23770 raft_consensus.cc:2749] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:21:57.178671 14429 raft_consensus.cc:2272] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.178999 23663 consensus_queue.cc:1048] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:21:57.183094 22934 consensus_peers.cc:597] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:57.185510 14429 tablet_replica.cc:333] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.186301 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be220d19020b428a8b1cc356f32ff5ad" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:57.186738 14429 raft_consensus.cc:2243] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.187347 14429 raft_consensus.cc:2272] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.191036 14429 tablet_replica.cc:333] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.191907 14429 raft_consensus.cc:2243] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.192435 14429 raft_consensus.cc:2272] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.195262 14429 tablet_replica.cc:333] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.200225 14429 raft_consensus.cc:2243] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.200901 14429 raft_consensus.cc:2272] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.204211 14429 tablet_replica.cc:333] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.205833 23037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78234c90229d4615ab9e2c2a811bd0f8" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:57.207619 23037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be220d19020b428a8b1cc356f32ff5ad" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:57.208417 14429 raft_consensus.cc:2243] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:21:57.210088 22962 raft_consensus.cc:1275] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Refusing update from remote peer 27745ef236cd494684f62a1269411a68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:21:57.210263 22783 leader_election.cc:304] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:57.211580 23774 raft_consensus.cc:2749] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:57.214010 22783 leader_election.cc:304] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:21:57.214740 14429 pending_rounds.cc:70] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679: Trying to abort 1 pending ops.
I20260709 13:21:57.215041 23774 raft_consensus.cc:2749] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:57.215106 14429 pending_rounds.cc:77] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679: Aborting op as it isn't in flight: id { term: 4 index: 2 } timestamp: 7305639186394611712 op_type: NO_OP noop_request { }
I20260709 13:21:57.216774 14429 raft_consensus.cc:2889] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 4 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:21:57.217129 14429 raft_consensus.cc:2272] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.218181 23801 consensus_queue.cc:1048] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:57.227893 14429 tablet_replica.cc:333] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.228955 14429 raft_consensus.cc:2243] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.229502 14429 raft_consensus.cc:2272] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.232707 14429 tablet_replica.cc:333] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.233543 14429 raft_consensus.cc:2243] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.234277 14429 raft_consensus.cc:2272] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.237246 14429 tablet_replica.cc:333] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.239888 14429 raft_consensus.cc:2243] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.240677 14429 raft_consensus.cc:2272] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:57.256969 22860 consensus_peers.cc:597] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:57.265339 14429 tablet_replica.cc:333] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.266403 14429 raft_consensus.cc:2243] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:57.267644 14429 raft_consensus.cc:2272] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.278544 14429 tablet_replica.cc:333] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.279405 14429 raft_consensus.cc:2243] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.281948 14429 raft_consensus.cc:2272] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.284965 22968 raft_consensus.cc:1275] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:21:57.284989 14429 tablet_replica.cc:333] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.286255 14429 raft_consensus.cc:2243] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.286885 14429 raft_consensus.cc:2272] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.288589 23809 consensus_queue.cc:1048] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:57.293181 14429 tablet_replica.cc:333] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.294227 14429 raft_consensus.cc:2243] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.295199 14429 raft_consensus.cc:2272] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.306143 14429 tablet_replica.cc:333] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.309871 14429 raft_consensus.cc:2243] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.310542 14429 raft_consensus.cc:2272] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.312924 23774 raft_consensus.cc:493] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:57.313416 23774 raft_consensus.cc:515] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:57.314759 14429 tablet_replica.cc:333] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.315578 23774 leader_election.cc:290] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:57.315685 14429 raft_consensus.cc:2243] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.316496 14429 raft_consensus.cc:2272] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.321165 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505eab84b06f46ee9d336abf01353dee" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:57.322523 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505eab84b06f46ee9d336abf01353dee" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:57.324163 22785 leader_election.cc:304] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944
W20260709 13:21:57.325222 22860 consensus_peers.cc:597] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:57.325327 23774 raft_consensus.cc:2749] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:21:57.327005 14429 tablet_replica.cc:333] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.327876 14429 raft_consensus.cc:2243] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.328382 14429 pending_rounds.cc:70] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679: Trying to abort 1 pending ops.
I20260709 13:21:57.328615 14429 pending_rounds.cc:77] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305639184656797696 op_type: NO_OP noop_request { }
I20260709 13:21:57.328938 14429 raft_consensus.cc:2889] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:21:57.329277 14429 raft_consensus.cc:2272] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.334550 14429 tablet_replica.cc:333] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.335635 14429 raft_consensus.cc:2243] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.336206 14429 raft_consensus.cc:2272] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.343776 14429 tablet_replica.cc:333] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.344586 14429 raft_consensus.cc:2243] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.345183 14429 raft_consensus.cc:2272] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.348405 14429 tablet_replica.cc:333] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.349218 14429 raft_consensus.cc:2243] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.349753 14429 raft_consensus.cc:2272] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.352977 14429 tablet_replica.cc:333] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.353631 14429 raft_consensus.cc:2243] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.354149 14429 raft_consensus.cc:2272] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.356633 14429 tablet_replica.cc:333] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.357201 14429 raft_consensus.cc:2243] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.357599 14429 raft_consensus.cc:2272] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.359915 14429 tablet_replica.cc:333] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.360584 14429 raft_consensus.cc:2243] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.361032 14429 raft_consensus.cc:2272] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.363795 14429 tablet_replica.cc:333] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.364528 14429 raft_consensus.cc:2243] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:21:57.365499 14429 raft_consensus.cc:2272] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.368602 14429 tablet_replica.cc:333] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.369488 14429 raft_consensus.cc:2243] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.370409 14429 raft_consensus.cc:2272] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.373317 14429 tablet_replica.cc:333] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.374200 14429 raft_consensus.cc:2243] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.374850 14429 raft_consensus.cc:2272] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.376945 14429 tablet_replica.cc:333] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.377863 14429 raft_consensus.cc:2243] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:57.378870 14429 raft_consensus.cc:2272] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.381363 14429 tablet_replica.cc:333] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.382093 14429 raft_consensus.cc:2243] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.382648 14429 raft_consensus.cc:2272] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.385157 14429 tablet_replica.cc:333] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.387387 14429 raft_consensus.cc:2243] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.387938 14429 raft_consensus.cc:2272] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.390586 14429 tablet_replica.cc:333] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.391324 14429 raft_consensus.cc:2243] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:21:57.392370 14429 raft_consensus.cc:2272] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.396369 14429 tablet_replica.cc:333] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.397169 14429 raft_consensus.cc:2243] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.397711 14429 raft_consensus.cc:2272] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.400122 14429 tablet_replica.cc:333] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.400660 14429 raft_consensus.cc:2243] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.401117 14429 raft_consensus.cc:2272] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.403254 14429 tablet_replica.cc:333] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.403754 14429 raft_consensus.cc:2243] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.404150 14429 raft_consensus.cc:2272] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.406266 14429 tablet_replica.cc:333] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.406847 14429 raft_consensus.cc:2243] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.407251 14429 raft_consensus.cc:2272] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.409401 14429 tablet_replica.cc:333] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.410040 14429 raft_consensus.cc:2243] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.410583 14429 raft_consensus.cc:2272] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.413134 14429 tablet_replica.cc:333] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.413769 14429 raft_consensus.cc:2243] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.414324 14429 raft_consensus.cc:2272] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.416566 14429 tablet_replica.cc:333] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.417111 14429 raft_consensus.cc:2243] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:21:57.417876 14429 raft_consensus.cc:2272] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.420280 14429 tablet_replica.cc:333] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.420812 14429 raft_consensus.cc:2243] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.421214 14429 raft_consensus.cc:2272] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.423152 14429 tablet_replica.cc:333] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.423619 14429 raft_consensus.cc:2243] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.424029 14429 raft_consensus.cc:2272] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.425823 14429 tablet_replica.cc:333] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.426333 14429 raft_consensus.cc:2243] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.426682 14429 pending_rounds.cc:70] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679: Trying to abort 1 pending ops.
I20260709 13:21:57.426839 14429 pending_rounds.cc:77] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305639178424836096 op_type: NO_OP noop_request { }
I20260709 13:21:57.427080 14429 raft_consensus.cc:2889] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:21:57.427328 14429 raft_consensus.cc:2272] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.429304 14429 tablet_replica.cc:333] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679: stopping tablet replica
I20260709 13:21:57.429934 14429 raft_consensus.cc:2243] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.430444 14429 raft_consensus.cc:2272] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:57.520161 22860 connection.cc:769] client connection to 127.14.23.65:43155 send error: Network error: sendmsg error: Broken pipe (error 32)
I20260709 13:21:57.543864 14429 tablet_server.cc:196] TabletServer@127.14.23.65:0 shutdown complete.
I20260709 13:21:57.649250 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:57.656347 23844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:57.665380 23847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:57.670558 23845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:57.673640 14429 server_base.cc:1061] running on GCE node
I20260709 13:21:57.677068 23801 raft_consensus.cc:493] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:57.677639 23801 raft_consensus.cc:515] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:57.678097 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:57.678484 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:57.678725 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603317678697 us; error 0 us; skew 500 ppm
I20260709 13:21:57.679575 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:57.680356 23801 leader_election.cc:290] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:57.680830 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:57.681509 22968 raft_consensus.cc:2468] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 3.
I20260709 13:21:57.685047 23008 leader_election.cc:304] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:57.685482 14429 webserver.cc:533] Webserver started at http://127.14.23.65:36953/ using document root <none> and password file <none>
I20260709 13:21:57.686265 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:57.686609 14429 fs_manager.cc:365] Using existing metadata directory in first data directory
W20260709 13:21:57.687752 23008 leader_election.cc:336] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
I20260709 13:21:57.695946 14429 fs_manager.cc:714] Time spent opening directory manager: real 0.008s	user 0.003s	sys 0.004s
I20260709 13:21:57.686143 23801 raft_consensus.cc:2804] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:21:57.705541 23801 raft_consensus.cc:493] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:57.705960 23801 raft_consensus.cc:3060] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:57.707665 23852 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:57.708720 14429 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:21:57.709079 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-0-root
uuid: "91f33bdbdd63438294071eb3776db679"
format_stamp: "Formatted at 2026-07-09 13:21:38 on dist-test-slave-twc1"
I20260709 13:21:57.709583 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603055563165-14429-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:57.735231 23801 raft_consensus.cc:515] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:57.738538 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4148bbaf3347bb942721170efe1ecc" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:57.739193 22968 raft_consensus.cc:3060] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:21:57.747972 23008 leader_election.cc:336] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
I20260709 13:21:57.748855 23801 leader_election.cc:290] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:57.752704 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:57.753433 22968 raft_consensus.cc:2468] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 4.
I20260709 13:21:57.754601 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:57.755043 23008 leader_election.cc:304] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:57.756294 23801 raft_consensus.cc:2804] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:21:57.757128 23801 raft_consensus.cc:697] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 4 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:57.758126 23801 consensus_queue.cc:237] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:57.778292 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:57.798868 23860 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 13:21:57.812500 22717 catalog_manager.cc:5697] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 1 to 4, leader changed from 91f33bdbdd63438294071eb3776db679 (127.14.23.65) to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 4 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:58.123529 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:21:58.123842 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.334s	user 0.010s	sys 0.005s
I20260709 13:21:58.124184 14429 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:21:58.130311 23860 tablet_bootstrap.cc:492] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:58.203998 23860 tablet_bootstrap.cc:492] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:58.204954 23860 tablet_bootstrap.cc:492] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:58.205626 23860 ts_tablet_manager.cc:1403] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.076s	user 0.031s	sys 0.032s
I20260709 13:21:58.208508 23860 raft_consensus.cc:359] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:58.209192 23860 raft_consensus.cc:740] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:58.209908 23860 consensus_queue.cc:260] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:58.212985 23860 ts_tablet_manager.cc:1434] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:21:58.214161 23860 tablet_bootstrap.cc:492] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
W20260709 13:21:58.266953 22934 consensus_peers.cc:597] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:58.279891 23860 tablet_bootstrap.cc:492] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:58.281800 23860 tablet_bootstrap.cc:492] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:58.282509 23860 ts_tablet_manager.cc:1403] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.069s	user 0.043s	sys 0.005s
I20260709 13:21:58.285481 23860 raft_consensus.cc:359] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:58.286510 23860 raft_consensus.cc:740] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:58.287349 23860 consensus_queue.cc:260] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:58.289219 23860 ts_tablet_manager.cc:1434] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:21:58.290046 23860 tablet_bootstrap.cc:492] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
W20260709 13:21:58.311758 23008 consensus_peers.cc:597] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:58.315071 22934 consensus_peers.cc:597] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:58.340999 22960 raft_consensus.cc:1275] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Refusing update from remote peer 27745ef236cd494684f62a1269411a68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:21:58.350788 23621 consensus_queue.cc:1048] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:58.355674 23860 tablet_bootstrap.cc:492] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:58.356673 23860 tablet_bootstrap.cc:492] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:58.357414 23860 ts_tablet_manager.cc:1403] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.068s	user 0.019s	sys 0.000s
I20260709 13:21:58.360543 23860 raft_consensus.cc:359] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:58.361586 23860 raft_consensus.cc:740] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:58.362427 23860 consensus_queue.cc:260] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:58.377964 23860 ts_tablet_manager.cc:1434] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:21:58.378823 23860 tablet_bootstrap.cc:492] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:58.385030 23621 raft_consensus.cc:493] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:58.386822 23621 raft_consensus.cc:515] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:58.397725 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abacd5d0d1e54db38a397e3c35c0cb81" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:58.415335 23860 tablet_bootstrap.cc:492] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:58.416466 23860 tablet_bootstrap.cc:492] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:58.417106 23860 ts_tablet_manager.cc:1403] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.038s	user 0.018s	sys 0.003s
W20260709 13:21:58.418341 23008 leader_election.cc:336] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
I20260709 13:21:58.418941 23008 leader_election.cc:304] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:58.419639 23860 raft_consensus.cc:359] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:58.420409 23860 raft_consensus.cc:740] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:58.421185 23801 raft_consensus.cc:2749] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:58.421218 23860 consensus_queue.cc:260] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:58.422067 23663 raft_consensus.cc:493] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:58.424067 23663 raft_consensus.cc:515] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:58.424762 23860 ts_tablet_manager.cc:1434] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:21:58.426604 23621 leader_election.cc:290] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:58.430661 23797 raft_consensus.cc:493] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:58.451784 23860 tablet_bootstrap.cc:492] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:58.451560 23797 raft_consensus.cc:515] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:58.453418 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e55b4e727be4089ba7454b662d03b9c" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:58.455408 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abacd5d0d1e54db38a397e3c35c0cb81" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:58.453677 23797 leader_election.cc:290] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:58.456019 23040 raft_consensus.cc:2468] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:58.457165 22932 leader_election.cc:304] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:58.458154 23797 raft_consensus.cc:493] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:58.463476 23797 raft_consensus.cc:515] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:58.450637 14429 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:21:58.467649 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.343s	user 0.278s	sys 0.007s
W20260709 13:21:58.466974 22934 leader_election.cc:336] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
I20260709 13:21:58.460753 23663 leader_election.cc:290] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:58.469975 23663 raft_consensus.cc:2804] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:58.470362 23663 raft_consensus.cc:493] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:58.470737 23663 raft_consensus.cc:3060] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:58.472602 23797 leader_election.cc:290] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
W20260709 13:21:58.474538 22934 leader_election.cc:336] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
I20260709 13:21:58.475589 22934 leader_election.cc:304] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:58.478178 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955a6d8dbbe94ef9be9c1f69b10675d8" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:58.478950 22894 raft_consensus.cc:2468] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 3.
I20260709 13:21:58.480284 23797 raft_consensus.cc:2749] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:58.487756 22934 leader_election.cc:304] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:58.490746 23872 raft_consensus.cc:2804] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:21:58.491278 23872 raft_consensus.cc:493] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:58.491691 23872 raft_consensus.cc:3060] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:21:58.498862 22934 leader_election.cc:336] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
I20260709 13:21:58.514716 23663 raft_consensus.cc:515] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:58.531662 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abacd5d0d1e54db38a397e3c35c0cb81" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:21:58.532282 23040 raft_consensus.cc:3060] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:58.540757 23872 raft_consensus.cc:515] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:58.542994 23872 leader_election.cc:290] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 4 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:58.545307 23663 leader_election.cc:290] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:58.548489 23797 raft_consensus.cc:493] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:58.549328 23797 raft_consensus.cc:515] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:58.551421 23797 leader_election.cc:290] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:58.554229 23860 tablet_bootstrap.cc:492] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:58.555225 23860 tablet_bootstrap.cc:492] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:58.540983 23040 raft_consensus.cc:2468] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 2.
I20260709 13:21:58.555884 23860 ts_tablet_manager.cc:1403] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.104s	user 0.026s	sys 0.020s
I20260709 13:21:58.558570 23663 raft_consensus.cc:493] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:58.558722 22932 leader_election.cc:304] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:58.559217 23881 raft_consensus.cc:493] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:58.559552 23797 raft_consensus.cc:2804] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:58.560058 23797 raft_consensus.cc:697] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [term 2 LEADER]: Becoming Leader. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:21:58.560915 23562 raft_consensus.cc:493] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:58.559721 23881 raft_consensus.cc:515] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:58.561437 23562 raft_consensus.cc:515] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:58.562198 23873 raft_consensus.cc:493] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:58.562728 23873 raft_consensus.cc:515] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:58.563514 23881 leader_election.cc:290] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:58.563650 23562 leader_election.cc:290] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:58.564177 23860 raft_consensus.cc:359] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:58.564669 23873 leader_election.cc:290] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:58.565832 23035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8b119f5c5844d1be439423b333ec60" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:58.566430 23860 raft_consensus.cc:740] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:58.566571 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955a6d8dbbe94ef9be9c1f69b10675d8" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:58.567101 22894 raft_consensus.cc:3060] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:58.567184 23860 consensus_queue.cc:260] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:58.568442 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "989cd1a042864c419d8c10665cb8afd0" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:58.569104 22890 raft_consensus.cc:2468] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:58.564625 23797 consensus_queue.cc:237] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:58.577641 23860 ts_tablet_manager.cc:1434] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.001s
I20260709 13:21:58.577824 23878 raft_consensus.cc:493] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:58.579319 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:58.579739 23878 raft_consensus.cc:515] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:58.580093 22892 raft_consensus.cc:2468] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 3.
I20260709 13:21:58.581835 23878 leader_election.cc:290] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:21:58.582427 23878 raft_consensus.cc:493] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:58.582880 23878 raft_consensus.cc:515] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:58.584884 23878 leader_election.cc:290] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
W20260709 13:21:58.580602 22934 leader_election.cc:336] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
I20260709 13:21:58.586589 23860 tablet_bootstrap.cc:492] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
W20260709 13:21:58.587288 22934 leader_election.cc:336] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
W20260709 13:21:58.588006 22934 leader_election.cc:336] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
I20260709 13:21:58.588534 23621 raft_consensus.cc:493] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:58.589207 23621 raft_consensus.cc:515] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
W20260709 13:21:58.588523 22934 consensus_peers.cc:597] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:58.591298 23621 leader_election.cc:290] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:58.593688 22934 leader_election.cc:304] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:58.598191 23621 raft_consensus.cc:493] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:58.598776 23621 raft_consensus.cc:515] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:58.600663 23621 leader_election.cc:290] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:58.602195 23872 raft_consensus.cc:2804] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:58.602432 22934 leader_election.cc:304] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:58.603232 23872 raft_consensus.cc:493] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:58.603668 23872 raft_consensus.cc:3060] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:58.606124 23797 raft_consensus.cc:2804] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:21:58.606570 23797 raft_consensus.cc:493] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:58.606953 23797 raft_consensus.cc:3060] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:58.610266 22894 raft_consensus.cc:2468] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 4.
I20260709 13:21:58.612174 22717 catalog_manager.cc:5697] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 1 to 2, leader changed from 91f33bdbdd63438294071eb3776db679 (127.14.23.65) to 8ef3887fb3874f8b81079e542d252944 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "8ef3887fb3874f8b81079e542d252944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:58.559324 23663 raft_consensus.cc:515] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
W20260709 13:21:58.616266 22860 leader_election.cc:336] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
W20260709 13:21:58.617687 22860 leader_election.cc:336] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
I20260709 13:21:58.618423 22968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "989cd1a042864c419d8c10665cb8afd0" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:58.621582 22962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e55b4e727be4089ba7454b662d03b9c" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
W20260709 13:21:58.622294 22860 leader_election.cc:336] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
I20260709 13:21:58.621258 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8b119f5c5844d1be439423b333ec60" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:58.625494 22960 raft_consensus.cc:2468] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 3.
I20260709 13:21:58.622249 22962 raft_consensus.cc:2468] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 1.
I20260709 13:21:58.626003 23872 raft_consensus.cc:515] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:58.623167 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050e9ef11f664c849a40cbb6acb25f19" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:58.626904 23663 leader_election.cc:290] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:58.620296 23797 raft_consensus.cc:515] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:58.627099 22968 raft_consensus.cc:2393] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 in current term 2: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
I20260709 13:21:58.624351 22965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:58.628931 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:21:58.630827 23797 leader_election.cc:290] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 4 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:58.631202 22965 raft_consensus.cc:2393] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49cbf11e18ce4268a7fcca76872afb23 in current term 4: Already voted for candidate 8ef3887fb3874f8b81079e542d252944 in this term.
W20260709 13:21:58.634069 22934 leader_election.cc:336] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
I20260709 13:21:58.635202 22934 leader_election.cc:304] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
W20260709 13:21:58.636751 22934 leader_election.cc:336] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
I20260709 13:21:58.637107 23797 raft_consensus.cc:2749] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:21:58.634476 23872 leader_election.cc:290] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:58.643949 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:58.644928 22894 raft_consensus.cc:3060] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:58.649885 22934 leader_election.cc:304] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:58.654332 23872 raft_consensus.cc:2804] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:21:58.654851 23872 raft_consensus.cc:697] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 4 LEADER]: Becoming Leader. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:21:58.655943 23860 tablet_bootstrap.cc:492] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:58.655709 23872 consensus_queue.cc:237] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:58.656939 23860 tablet_bootstrap.cc:492] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:58.657613 23860 ts_tablet_manager.cc:1403] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.072s	user 0.016s	sys 0.000s
W20260709 13:21:58.661162 22860 consensus_peers.cc:597] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:58.666460 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "989cd1a042864c419d8c10665cb8afd0" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:58.667213 22890 raft_consensus.cc:3060] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:58.671402 22860 leader_election.cc:304] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:58.669514 23008 leader_election.cc:304] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:58.675995 23621 raft_consensus.cc:2804] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:21:58.676368 23621 raft_consensus.cc:493] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:58.676764 23621 raft_consensus.cc:3060] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:58.678871 22890 raft_consensus.cc:2468] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 2.
I20260709 13:21:58.676368 23860 raft_consensus.cc:359] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:58.683316 23860 raft_consensus.cc:740] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:58.686329 23860 consensus_queue.cc:260] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:58.689461 23860 ts_tablet_manager.cc:1434] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.000s
I20260709 13:21:58.690315 23860 tablet_bootstrap.cc:492] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:58.689934 23621 raft_consensus.cc:515] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:58.693670 23621 leader_election.cc:290] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:58.694514 23008 leader_election.cc:304] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:58.695683 23562 raft_consensus.cc:2749] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:58.697103 22860 leader_election.cc:304] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:58.700484 23801 raft_consensus.cc:2804] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:58.701007 23801 raft_consensus.cc:493] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:58.701345 23801 raft_consensus.cc:3060] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:58.703214 22894 raft_consensus.cc:2468] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 4.
I20260709 13:21:58.705199 22934 leader_election.cc:304] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:58.706032 23872 raft_consensus.cc:2804] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:58.706703 23872 raft_consensus.cc:697] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [term 2 LEADER]: Becoming Leader. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:21:58.707538 23872 consensus_queue.cc:237] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:58.713671 23801 raft_consensus.cc:515] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
W20260709 13:21:58.714212 22934 leader_election.cc:336] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
I20260709 13:21:58.715682 23878 raft_consensus.cc:2749] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:58.720222 22934 leader_election.cc:304] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944; no voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679
W20260709 13:21:58.717861 23008 leader_election.cc:336] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
W20260709 13:21:58.721469 23008 consensus_peers.cc:597] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:58.722024 23663 raft_consensus.cc:2749] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:21:58.725224 23008 leader_election.cc:336] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
W20260709 13:21:58.722849 22934 leader_election.cc:336] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
W20260709 13:21:58.727208 22934 leader_election.cc:336] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
W20260709 13:21:58.728717 22934 consensus_peers.cc:597] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:58.730644 22961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8b119f5c5844d1be439423b333ec60" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:58.731357 22961 raft_consensus.cc:3060] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:58.727634 23801 leader_election.cc:290] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:58.743382 22961 raft_consensus.cc:2468] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 4.
I20260709 13:21:58.744113 22717 catalog_manager.cc:5697] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 1 to 2, leader changed from 91f33bdbdd63438294071eb3776db679 (127.14.23.65) to 8ef3887fb3874f8b81079e542d252944 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "8ef3887fb3874f8b81079e542d252944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:58.745935 22717 catalog_manager.cc:5697] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 3 to 4, leader changed from 91f33bdbdd63438294071eb3776db679 (127.14.23.65) to 8ef3887fb3874f8b81079e542d252944 (127.14.23.67). New cstate: current_term: 4 leader_uuid: "8ef3887fb3874f8b81079e542d252944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:58.746964 22860 leader_election.cc:304] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:21:58.747771 23900 raft_consensus.cc:2749] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:21:58.751933 23896 raft_consensus.cc:493] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:58.752586 23896 raft_consensus.cc:515] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:58.754946 23896 leader_election.cc:290] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:21:58.757055 22934 leader_election.cc:304] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 91f33bdbdd63438294071eb3776db679
W20260709 13:21:58.758638 22860 consensus_peers.cc:597] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:58.760380 22961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e55b4e727be4089ba7454b662d03b9c" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:58.762244 23872 raft_consensus.cc:2804] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:21:58.762777 23872 raft_consensus.cc:697] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 4 LEADER]: Becoming Leader. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:21:58.762854 22961 raft_consensus.cc:3060] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:58.763700 23872 consensus_queue.cc:237] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:58.764837 23860 tablet_bootstrap.cc:492] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:58.765810 23860 tablet_bootstrap.cc:492] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:58.766480 23860 ts_tablet_manager.cc:1403] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.076s	user 0.012s	sys 0.005s
I20260709 13:21:58.766925 23663 raft_consensus.cc:493] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:58.767448 23663 raft_consensus.cc:515] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:58.769057 23860 raft_consensus.cc:359] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:58.769472 23663 leader_election.cc:290] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:58.772720 22961 raft_consensus.cc:2468] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 2.
I20260709 13:21:58.782192 22965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944" is_pre_election: true
I20260709 13:21:58.782884 22965 raft_consensus.cc:2468] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 3.
W20260709 13:21:58.783466 22860 consensus_peers.cc:597] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:58.784837 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050e9ef11f664c849a40cbb6acb25f19" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:58.785533 22894 raft_consensus.cc:2468] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 1.
I20260709 13:21:58.787539 23860 raft_consensus.cc:740] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:58.789546 23860 consensus_queue.cc:260] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:58.792843 23008 leader_election.cc:304] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:58.793766 23860 ts_tablet_manager.cc:1434] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.000s
I20260709 13:21:58.794647 23860 tablet_bootstrap.cc:492] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:58.797616 23801 raft_consensus.cc:2804] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:21:58.798329 23801 raft_consensus.cc:697] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 4 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:58.799218 23801 consensus_queue.cc:237] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:58.803956 23008 leader_election.cc:304] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
W20260709 13:21:58.805948 22934 consensus_peers.cc:597] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:58.807039 22934 leader_election.cc:304] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:58.807808 23801 raft_consensus.cc:2804] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:58.808495 23801 raft_consensus.cc:697] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 2 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:58.809440 23801 consensus_queue.cc:237] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:58.812434 23008 leader_election.cc:304] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:58.813732 23896 raft_consensus.cc:2804] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:21:58.814157 23896 raft_consensus.cc:493] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:58.814529 23896 raft_consensus.cc:3060] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:21:58.815189 23008 leader_election.cc:336] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
W20260709 13:21:58.815986 23008 leader_election.cc:336] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
I20260709 13:21:58.823504 23896 raft_consensus.cc:515] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:58.827199 23896 leader_election.cc:290] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
W20260709 13:21:58.824983 22860 consensus_peers.cc:597] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:58.827760 22717 catalog_manager.cc:5697] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 3 to 4, leader changed from 91f33bdbdd63438294071eb3776db679 (127.14.23.65) to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 4 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:58.827832 23872 raft_consensus.cc:2804] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:21:58.831055 22860 consensus_peers.cc:597] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:58.831278 23872 raft_consensus.cc:493] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:21:58.831727 23872 raft_consensus.cc:3060] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:58.833257 22714 catalog_manager.cc:5697] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 3 to 4, leader changed from 91f33bdbdd63438294071eb3776db679 (127.14.23.65) to 8ef3887fb3874f8b81079e542d252944 (127.14.23.67). New cstate: current_term: 4 leader_uuid: "8ef3887fb3874f8b81079e542d252944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: HEALTHY } } }
W20260709 13:21:58.834106 23008 leader_election.cc:336] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
I20260709 13:21:58.840744 22965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:21:58.841352 22965 raft_consensus.cc:3060] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:21:58.842892 22934 leader_election.cc:336] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
I20260709 13:21:58.843180 23872 raft_consensus.cc:515] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
W20260709 13:21:58.841467 23008 consensus_peers.cc:597] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:58.845301 23872 leader_election.cc:290] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
W20260709 13:21:58.851756 23008 leader_election.cc:336] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
I20260709 13:21:58.854941 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050e9ef11f664c849a40cbb6acb25f19" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:58.855581 22894 raft_consensus.cc:3060] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:58.858189 22965 raft_consensus.cc:2468] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 4.
I20260709 13:21:58.864678 22894 raft_consensus.cc:2468] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 2.
I20260709 13:21:58.867867 22934 leader_election.cc:304] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:21:58.869062 23008 leader_election.cc:304] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:58.871649 23621 raft_consensus.cc:2804] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:21:58.872144 23621 raft_consensus.cc:697] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 4 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:58.872867 23621 consensus_queue.cc:237] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
W20260709 13:21:58.875380 22934 leader_election.cc:336] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
W20260709 13:21:58.877326 22860 consensus_peers.cc:597] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:58.877365 23860 tablet_bootstrap.cc:492] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:58.877441 23663 raft_consensus.cc:2804] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:58.883319 23663 raft_consensus.cc:697] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [term 2 LEADER]: Becoming Leader. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:21:58.887857 23860 tablet_bootstrap.cc:492] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:58.888765 23860 ts_tablet_manager.cc:1403] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.094s	user 0.018s	sys 0.000s
I20260709 13:21:58.900748 22714 catalog_manager.cc:5697] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 1 to 2, leader changed from 91f33bdbdd63438294071eb3776db679 (127.14.23.65) to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 2 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:58.890962 23663 consensus_queue.cc:237] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:58.915226 23860 raft_consensus.cc:359] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:58.915840 23860 raft_consensus.cc:740] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:58.916471 23860 consensus_queue.cc:260] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:58.917909 23860 ts_tablet_manager.cc:1434] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.000s
I20260709 13:21:58.920363 23860 tablet_bootstrap.cc:492] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
W20260709 13:21:58.927130 22860 consensus_peers.cc:597] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:58.927671 22860 consensus_peers.cc:597] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:58.934094 22860 consensus_peers.cc:597] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:58.935035 23008 consensus_peers.cc:597] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:58.936126 22934 consensus_peers.cc:597] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:58.940702 23008 consensus_peers.cc:597] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:58.944321 23008 consensus_peers.cc:597] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:58.944386 22860 consensus_peers.cc:597] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:58.948508 22714 catalog_manager.cc:5697] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 3 to 4, leader changed from 91f33bdbdd63438294071eb3776db679 (127.14.23.65) to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 4 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:58.969075 22860 consensus_peers.cc:597] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:58.973381 22934 consensus_peers.cc:597] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:58.979880 22860 consensus_peers.cc:597] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:58.980377 22860 consensus_peers.cc:597] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:58.983263 23008 consensus_peers.cc:597] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:58.987501 23008 consensus_peers.cc:597] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:58.988276 22714 catalog_manager.cc:5697] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 1 to 2, leader changed from 91f33bdbdd63438294071eb3776db679 (127.14.23.65) to 8ef3887fb3874f8b81079e542d252944 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "8ef3887fb3874f8b81079e542d252944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:58.992451 23860 tablet_bootstrap.cc:492] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:58.993497 23860 tablet_bootstrap.cc:492] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:58.994184 23860 ts_tablet_manager.cc:1403] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.074s	user 0.009s	sys 0.018s
I20260709 13:21:58.996778 23860 raft_consensus.cc:359] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:58.997430 23860 raft_consensus.cc:740] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:58.998231 23860 consensus_queue.cc:260] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:59.001078 23860 ts_tablet_manager.cc:1434] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:21:59.002207 23860 tablet_bootstrap.cc:492] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:59.020856 23860 tablet_bootstrap.cc:492] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.021693 23860 tablet_bootstrap.cc:492] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:59.022264 23860 ts_tablet_manager.cc:1403] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.004s
I20260709 13:21:59.024891 23860 raft_consensus.cc:359] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:59.025543 23860 raft_consensus.cc:740] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.026328 23860 consensus_queue.cc:260] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:59.029078 23860 ts_tablet_manager.cc:1434] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:21:59.029883 23860 tablet_bootstrap.cc:492] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
W20260709 13:21:59.030576 22860 consensus_peers.cc:597] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:59.045509 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.65:43155
I20260709 13:21:59.045593 23958 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.65:43155 every 8 connection(s)
I20260709 13:21:59.064702 23860 tablet_bootstrap.cc:492] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.066067 23860 tablet_bootstrap.cc:492] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:59.066884 23860 ts_tablet_manager.cc:1403] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.037s	user 0.014s	sys 0.019s
I20260709 13:21:59.069960 23860 raft_consensus.cc:359] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:59.070744 23860 raft_consensus.cc:740] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.071702 23860 consensus_queue.cc:260] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:59.079427 23860 ts_tablet_manager.cc:1434] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:21:59.080458 23860 tablet_bootstrap.cc:492] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:59.101661 23040 raft_consensus.cc:1275] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Refusing update from remote peer 8ef3887fb3874f8b81079e542d252944: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:59.103502 23663 consensus_queue.cc:1048] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:59.111197 23860 tablet_bootstrap.cc:492] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:21:59.112356 23860 tablet_bootstrap.cc:492] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:59.113070 23860 ts_tablet_manager.cc:1403] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.033s	user 0.021s	sys 0.004s
I20260709 13:21:59.115916 23860 raft_consensus.cc:359] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:59.116837 23860 raft_consensus.cc:740] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.117827 23860 consensus_queue.cc:260] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:59.120533 23860 ts_tablet_manager.cc:1434] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:21:59.121515 23860 tablet_bootstrap.cc:492] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
W20260709 13:21:59.146102 22934 consensus_peers.cc:597] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:59.158320 22894 raft_consensus.cc:1275] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Refusing update from remote peer 8ef3887fb3874f8b81079e542d252944: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:59.161526 23663 consensus_queue.cc:1048] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:21:59.183717 22934 consensus_peers.cc:597] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:59.191111 23959 heartbeater.cc:344] Connected to a master server at 127.14.23.126:41523
I20260709 13:21:59.191640 23959 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:59.192586 23959 heartbeater.cc:507] Master 127.14.23.126:41523 requested a full tablet report, sending...
I20260709 13:21:59.199146 23930 raft_consensus.cc:3060] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:59.228044 22714 ts_manager.cc:194] Re-registered known tserver with Master: 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
W20260709 13:21:59.247963 23008 consensus_peers.cc:597] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:59.247982 22890 raft_consensus.cc:1275] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Refusing update from remote peer 8ef3887fb3874f8b81079e542d252944: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:21:59.252712 23663 consensus_queue.cc:1048] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:59.257977 23860 tablet_bootstrap.cc:492] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:21:59.259959 23860 tablet_bootstrap.cc:492] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:59.260694 23860 ts_tablet_manager.cc:1403] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.139s	user 0.059s	sys 0.051s
I20260709 13:21:59.263382 23860 raft_consensus.cc:359] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:59.264232 23860 raft_consensus.cc:740] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.264959 23860 consensus_queue.cc:260] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:59.274257 23860 ts_tablet_manager.cc:1434] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:21:59.275275 23860 tablet_bootstrap.cc:492] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
W20260709 13:21:59.289605 22860 consensus_peers.cc:597] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:59.291697 23008 consensus_peers.cc:597] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:59.297027 22934 consensus_peers.cc:597] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:59.306252 23930 raft_consensus.cc:1240] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Rejecting Update request from peer 49cbf11e18ce4268a7fcca76872afb23 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:21:59.306975 23860 tablet_bootstrap.cc:492] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.307878 23878 consensus_queue.cc:1059] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:59.308887 23878 raft_consensus.cc:3055] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:21:59.309238 23878 raft_consensus.cc:740] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:21:59.310070 23860 tablet_bootstrap.cc:492] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:59.310041 23878 consensus_queue.cc:260] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:59.310941 23860 ts_tablet_manager.cc:1403] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.036s	user 0.012s	sys 0.007s
I20260709 13:21:59.313560 23860 raft_consensus.cc:359] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:59.314322 23860 raft_consensus.cc:740] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.314953 23860 consensus_queue.cc:260] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:21:59.316236 22965 raft_consensus.cc:1275] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Refusing update from remote peer 27745ef236cd494684f62a1269411a68: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:21:59.316850 23860 ts_tablet_manager.cc:1434] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:21:59.318893 23896 consensus_queue.cc:1048] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:59.319892 23860 tablet_bootstrap.cc:492] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:59.323076 22890 raft_consensus.cc:1275] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Refusing update from remote peer 8ef3887fb3874f8b81079e542d252944: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:21:59.327543 23880 consensus_queue.cc:1048] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:59.311471 23878 raft_consensus.cc:3060] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:59.359889 23860 tablet_bootstrap.cc:492] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.360944 23860 tablet_bootstrap.cc:492] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:59.361681 23860 ts_tablet_manager.cc:1403] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.042s	user 0.017s	sys 0.001s
I20260709 13:21:59.366811 23860 raft_consensus.cc:359] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:59.367636 23860 raft_consensus.cc:740] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.368481 23860 consensus_queue.cc:260] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:59.370289 22963 raft_consensus.cc:1275] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Refusing update from remote peer 27745ef236cd494684f62a1269411a68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:59.370754 23860 ts_tablet_manager.cc:1434] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:21:59.371680 23860 tablet_bootstrap.cc:492] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:59.377475 23925 consensus_queue.cc:1048] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:21:59.391014 22934 consensus_peers.cc:597] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:59.394948 23860 tablet_bootstrap.cc:492] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.398495 23860 tablet_bootstrap.cc:492] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:59.399333 23860 ts_tablet_manager.cc:1403] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.028s	user 0.018s	sys 0.004s
I20260709 13:21:59.402074 23860 raft_consensus.cc:359] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:59.402848 23860 raft_consensus.cc:740] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.404008 23860 consensus_queue.cc:260] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:59.409578 23860 ts_tablet_manager.cc:1434] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:21:59.410521 23860 tablet_bootstrap.cc:492] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:59.424887 22963 raft_consensus.cc:1275] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Refusing update from remote peer 27745ef236cd494684f62a1269411a68: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:21:59.428537 23925 consensus_queue.cc:1048] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:59.434758 23860 tablet_bootstrap.cc:492] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.435915 23860 tablet_bootstrap.cc:492] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:59.436656 23860 ts_tablet_manager.cc:1403] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.008s
I20260709 13:21:59.439421 23860 raft_consensus.cc:359] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:59.440166 23860 raft_consensus.cc:740] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.441008 23860 consensus_queue.cc:260] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:59.443341 23860 ts_tablet_manager.cc:1434] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:21:59.444449 23860 tablet_bootstrap.cc:492] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
W20260709 13:21:59.443328 22860 consensus_peers.cc:597] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:59.461879 23932 raft_consensus.cc:3060] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:21:59.462570 23860 tablet_bootstrap.cc:492] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.463582 23860 tablet_bootstrap.cc:492] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:59.464286 23860 ts_tablet_manager.cc:1403] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.007s
I20260709 13:21:59.466962 23860 raft_consensus.cc:359] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:59.467718 23860 raft_consensus.cc:740] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.468436 23860 consensus_queue.cc:260] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:59.469875 23932 raft_consensus.cc:1275] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Refusing update from remote peer 27745ef236cd494684f62a1269411a68: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:21:59.470822 23860 ts_tablet_manager.cc:1434] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:21:59.471758 23860 tablet_bootstrap.cc:492] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:59.471798 23869 consensus_queue.cc:1048] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:21:59.479065 23008 consensus_peers.cc:597] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:59.500294 22714 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57776
I20260709 13:21:59.505205 23959 heartbeater.cc:499] Master 127.14.23.126:41523 was elected leader, sending a full tablet report...
I20260709 13:21:59.508116 22890 raft_consensus.cc:1275] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Refusing update from remote peer 8ef3887fb3874f8b81079e542d252944: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:59.513103 23797 consensus_queue.cc:1048] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:59.524982 23860 tablet_bootstrap.cc:492] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.526062 23860 tablet_bootstrap.cc:492] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:59.526762 23860 ts_tablet_manager.cc:1403] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.055s	user 0.020s	sys 0.008s
I20260709 13:21:59.529327 23860 raft_consensus.cc:359] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:59.530057 23860 raft_consensus.cc:740] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.530742 23860 consensus_queue.cc:260] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
W20260709 13:21:59.542333 22934 consensus_peers.cc:597] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:59.544471 23860 ts_tablet_manager.cc:1434] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:21:59.545720 23860 tablet_bootstrap.cc:492] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:59.565743 23930 pending_rounds.cc:85] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679: Aborting all ops after (but not including) 2
I20260709 13:21:59.566221 23930 pending_rounds.cc:107] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:21:59.566538 23930 raft_consensus.cc:2889] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:59.590210 23929 raft_consensus.cc:3060] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:59.611337 23930 raft_consensus.cc:3060] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:59.616541 23860 tablet_bootstrap.cc:492] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.617542 23860 tablet_bootstrap.cc:492] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:59.618224 23860 ts_tablet_manager.cc:1403] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.073s	user 0.033s	sys 0.023s
I20260709 13:21:59.620725 23860 raft_consensus.cc:359] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:59.621381 23860 raft_consensus.cc:740] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.622074 23860 consensus_queue.cc:260] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:59.623549 23860 ts_tablet_manager.cc:1434] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:21:59.624330 23860 tablet_bootstrap.cc:492] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:59.722393 23860 tablet_bootstrap.cc:492] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.727348 23860 tablet_bootstrap.cc:492] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:59.728094 23860 ts_tablet_manager.cc:1403] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.104s	user 0.032s	sys 0.039s
I20260709 13:21:59.730669 23860 raft_consensus.cc:359] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:59.731880 23860 raft_consensus.cc:740] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.732578 23860 consensus_queue.cc:260] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:59.734738 23860 ts_tablet_manager.cc:1434] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:21:59.735575 23860 tablet_bootstrap.cc:492] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:59.749869 23860 tablet_bootstrap.cc:492] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.751184 23860 tablet_bootstrap.cc:492] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:59.751837 23860 ts_tablet_manager.cc:1403] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.016s	user 0.015s	sys 0.000s
I20260709 13:21:59.754859 23860 raft_consensus.cc:359] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:59.755371 23860 raft_consensus.cc:740] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.755942 23860 consensus_queue.cc:260] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:59.757573 23860 ts_tablet_manager.cc:1434] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.000s
I20260709 13:21:59.758749 23860 tablet_bootstrap.cc:492] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:59.791311 23860 tablet_bootstrap.cc:492] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.792021 23860 tablet_bootstrap.cc:492] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:59.792532 23860 ts_tablet_manager.cc:1403] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.006s
I20260709 13:21:59.794497 23860 raft_consensus.cc:359] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:59.795138 23860 raft_consensus.cc:740] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.795809 23860 consensus_queue.cc:260] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:59.797773 23860 ts_tablet_manager.cc:1434] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:21:59.798576 23860 tablet_bootstrap.cc:492] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:59.821171 23860 tablet_bootstrap.cc:492] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.822467 23860 tablet_bootstrap.cc:492] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:59.823163 23860 ts_tablet_manager.cc:1403] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.007s
I20260709 13:21:59.826732 23860 raft_consensus.cc:359] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:59.827270 23860 raft_consensus.cc:740] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.827899 23860 consensus_queue.cc:260] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:59.829960 23860 ts_tablet_manager.cc:1434] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:21:59.830902 23860 tablet_bootstrap.cc:492] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:59.857085 23860 tablet_bootstrap.cc:492] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.858124 23860 tablet_bootstrap.cc:492] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:59.858790 23860 ts_tablet_manager.cc:1403] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.028s	user 0.018s	sys 0.003s
I20260709 13:21:59.858881 23924 raft_consensus.cc:493] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:59.859393 23924 raft_consensus.cc:515] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:59.861835 23860 raft_consensus.cc:359] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:59.862576 23860 raft_consensus.cc:740] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.863338 23860 consensus_queue.cc:260] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:21:59.865521 23860 ts_tablet_manager.cc:1434] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:21:59.866405 23860 tablet_bootstrap.cc:492] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:59.866387 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:21:59.867322 23932 raft_consensus.cc:2410] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:21:59.867260 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:21:59.868060 22890 raft_consensus.cc:2468] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 4.
I20260709 13:21:59.870232 23008 leader_election.cc:304] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:21:59.861728 23924 leader_election.cc:290] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:59.871505 23925 raft_consensus.cc:2804] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:21:59.872051 23925 raft_consensus.cc:493] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:59.872414 23925 raft_consensus.cc:3060] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:21:59.880474 23925 raft_consensus.cc:515] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:59.889022 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:21:59.889707 23932 raft_consensus.cc:3060] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:21:59.897497 23932 raft_consensus.cc:2410] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:21:59.888993 23925 leader_election.cc:290] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 5 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:21:59.904292 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536d31f168a54e19aa9aa819e6818d64" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:21:59.911556 22890 raft_consensus.cc:3060] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:21:59.924840 22890 raft_consensus.cc:2468] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 5.
I20260709 13:21:59.932154 23008 leader_election.cc:304] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:21:59.933010 23869 raft_consensus.cc:2804] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:21:59.933573 23869 raft_consensus.cc:697] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 5 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:21:59.934489 23869 consensus_queue.cc:237] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:59.938333 23860 tablet_bootstrap.cc:492] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=1 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.939636 23860 tablet_bootstrap.cc:492] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:59.940280 23860 ts_tablet_manager.cc:1403] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.074s	user 0.025s	sys 0.027s
I20260709 13:21:59.944191 23860 raft_consensus.cc:359] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:59.944990 23860 raft_consensus.cc:740] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.945938 22715 catalog_manager.cc:5697] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 1 to 5, leader changed from 91f33bdbdd63438294071eb3776db679 (127.14.23.65) to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 5 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:59.946256 23860 consensus_queue.cc:260] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:59.948830 23860 ts_tablet_manager.cc:1434] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:21:59.949687 23860 tablet_bootstrap.cc:492] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:21:59.976330 23860 tablet_bootstrap.cc:492] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.977373 23860 tablet_bootstrap.cc:492] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:21:59.978195 23860 ts_tablet_manager.cc:1403] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.029s	user 0.017s	sys 0.006s
I20260709 13:21:59.980677 23860 raft_consensus.cc:359] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:59.981564 23860 raft_consensus.cc:740] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.982378 23860 consensus_queue.cc:260] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:21:59.985423 23866 raft_consensus.cc:493] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:59.985909 23866 raft_consensus.cc:515] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:21:59.988576 23866 leader_election.cc:290] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:22:00.006031 23860 ts_tablet_manager.cc:1434] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260709 13:22:00.007050 23860 tablet_bootstrap.cc:492] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:22:00.110229 23036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "27745ef236cd494684f62a1269411a68" is_pre_election: true
I20260709 13:22:00.137842 23860 tablet_bootstrap.cc:492] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:00.138845 23860 tablet_bootstrap.cc:492] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:22:00.139509 23860 ts_tablet_manager.cc:1403] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.133s	user 0.015s	sys 0.001s
I20260709 13:22:00.141510 23860 raft_consensus.cc:359] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:00.141999 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:22:00.143020 22892 raft_consensus.cc:2410] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91f33bdbdd63438294071eb3776db679 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:22:00.142200 23860 raft_consensus.cc:740] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:22:00.145026 23860 consensus_queue.cc:260] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:00.146505 23855 leader_election.cc:304] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91f33bdbdd63438294071eb3776db679; no voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:22:00.150861 23860 ts_tablet_manager.cc:1434] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:22:00.151865 23860 tablet_bootstrap.cc:492] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:22:00.161065 24006 raft_consensus.cc:2749] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:22:00.232429 23860 tablet_bootstrap.cc:492] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:00.233412 23860 tablet_bootstrap.cc:492] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:22:00.234099 23860 ts_tablet_manager.cc:1403] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.082s	user 0.028s	sys 0.037s
I20260709 13:22:00.236259 23860 raft_consensus.cc:359] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:00.236732 23860 raft_consensus.cc:740] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:22:00.237380 23860 consensus_queue.cc:260] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:00.239226 23860 ts_tablet_manager.cc:1434] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:22:00.240345 23860 tablet_bootstrap.cc:492] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:22:00.264991 23930 raft_consensus.cc:3060] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:00.291368 23860 tablet_bootstrap.cc:492] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:00.293462 23860 tablet_bootstrap.cc:492] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:22:00.294178 23860 ts_tablet_manager.cc:1403] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.054s	user 0.026s	sys 0.020s
I20260709 13:22:00.296502 23860 raft_consensus.cc:359] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:00.297011 23860 raft_consensus.cc:740] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:22:00.298120 23860 consensus_queue.cc:260] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:00.299866 23860 ts_tablet_manager.cc:1434] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.003s
I20260709 13:22:00.300669 23860 tablet_bootstrap.cc:492] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:22:00.321362 23860 tablet_bootstrap.cc:492] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:22:00.322366 23860 tablet_bootstrap.cc:492] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:22:00.323102 23860 ts_tablet_manager.cc:1403] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.012s
I20260709 13:22:00.325595 23860 raft_consensus.cc:359] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:00.326473 23860 raft_consensus.cc:740] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:22:00.327169 23860 consensus_queue.cc:260] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:00.329368 23860 ts_tablet_manager.cc:1434] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:22:00.330549 23860 tablet_bootstrap.cc:492] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:22:00.347667 23860 tablet_bootstrap.cc:492] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:00.348618 23860 tablet_bootstrap.cc:492] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:22:00.349246 23860 ts_tablet_manager.cc:1403] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.000s
I20260709 13:22:00.353075 23860 raft_consensus.cc:359] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:00.353868 23860 raft_consensus.cc:740] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:22:00.354650 23860 consensus_queue.cc:260] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:00.356526 23860 ts_tablet_manager.cc:1434] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:22:00.359130 23860 tablet_bootstrap.cc:492] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:22:00.376870 23860 tablet_bootstrap.cc:492] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:00.377754 23860 tablet_bootstrap.cc:492] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:22:00.378393 23860 ts_tablet_manager.cc:1403] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.002s
I20260709 13:22:00.381019 23860 raft_consensus.cc:359] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:00.381753 23860 raft_consensus.cc:740] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:22:00.382406 23860 consensus_queue.cc:260] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:00.384325 23860 ts_tablet_manager.cc:1434] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:22:00.385076 23860 tablet_bootstrap.cc:492] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:22:00.403487 22892 raft_consensus.cc:1275] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 5 FOLLOWER]: Refusing update from remote peer 27745ef236cd494684f62a1269411a68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
I20260709 13:22:00.406628 23896 consensus_queue.cc:1048] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:00.424708 23860 tablet_bootstrap.cc:492] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:00.425760 23860 tablet_bootstrap.cc:492] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:22:00.427908 23860 ts_tablet_manager.cc:1403] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.043s	user 0.020s	sys 0.016s
I20260709 13:22:00.430989 23860 raft_consensus.cc:359] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:00.431710 23860 raft_consensus.cc:740] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:22:00.432958 23860 consensus_queue.cc:260] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:00.435251 23860 ts_tablet_manager.cc:1434] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:22:00.436061 23860 tablet_bootstrap.cc:492] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:22:00.475941 23930 raft_consensus.cc:1275] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 5 FOLLOWER]: Refusing update from remote peer 27745ef236cd494684f62a1269411a68: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 2. (term mismatch)
I20260709 13:22:00.476408 23930 pending_rounds.cc:85] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679: Aborting all ops after (but not including) 1
I20260709 13:22:00.476656 23930 pending_rounds.cc:107] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 13:22:00.476912 23930 raft_consensus.cc:2889] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:00.532776 23924 consensus_queue.cc:1048] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:00.563323 23932 pending_rounds.cc:85] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679: Aborting all ops after (but not including) 1
I20260709 13:22:00.563666 23932 pending_rounds.cc:107] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:22:00.563910 23932 raft_consensus.cc:2889] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:00.678395 23860 tablet_bootstrap.cc:492] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:00.681306 23860 tablet_bootstrap.cc:492] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:22:00.681867 23860 ts_tablet_manager.cc:1403] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.246s	user 0.041s	sys 0.071s
I20260709 13:22:00.694384 23860 raft_consensus.cc:359] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:00.695158 23860 raft_consensus.cc:740] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:22:00.696017 23860 consensus_queue.cc:260] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:00.697247 23925 raft_consensus.cc:493] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:22:00.706566 23925 raft_consensus.cc:515] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:00.708645 23925 leader_election.cc:290] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:22:00.706198 23860 ts_tablet_manager.cc:1434] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 13:22:00.743021 23860 tablet_bootstrap.cc:492] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
W20260709 13:22:00.750262 23008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:45323, user_credentials={real_user=slave}} blocked reactor thread for 126050us
I20260709 13:22:00.775518 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679" is_pre_election: true
I20260709 13:22:00.776355 23929 raft_consensus.cc:2393] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 in current term 4: Already voted for candidate 91f33bdbdd63438294071eb3776db679 in this term.
I20260709 13:22:00.776686 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23" is_pre_election: true
I20260709 13:22:00.777411 22890 raft_consensus.cc:2468] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 4.
W20260709 13:22:00.784080 23008 consensus_peers.cc:597] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:00.807590 23008 leader_election.cc:304] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:22:00.808414 23925 raft_consensus.cc:2804] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:22:00.808916 23925 raft_consensus.cc:493] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 49cbf11e18ce4268a7fcca76872afb23)
I20260709 13:22:00.809353 23925 raft_consensus.cc:3060] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:00.824923 23925 raft_consensus.cc:515] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:00.829519 23925 leader_election.cc:290] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 election: Requested vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:22:00.832612 23921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91f33bdbdd63438294071eb3776db679"
I20260709 13:22:00.833534 23921 raft_consensus.cc:2393] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 27745ef236cd494684f62a1269411a68 in current term 4: Already voted for candidate 91f33bdbdd63438294071eb3776db679 in this term.
I20260709 13:22:00.837006 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56df9023b48a4dc5ae4b437c358860ed" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:22:00.846179 23860 tablet_bootstrap.cc:492] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:00.847236 23860 tablet_bootstrap.cc:492] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:22:00.847970 23860 ts_tablet_manager.cc:1403] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.105s	user 0.029s	sys 0.004s
I20260709 13:22:00.850575 23860 raft_consensus.cc:359] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:00.851408 23860 raft_consensus.cc:740] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:22:00.852145 23860 consensus_queue.cc:260] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:00.854701 23860 ts_tablet_manager.cc:1434] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:22:00.855543 23860 tablet_bootstrap.cc:492] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:22:00.855827 22890 raft_consensus.cc:2468] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 4.
I20260709 13:22:00.857461 23008 leader_election.cc:304] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 91f33bdbdd63438294071eb3776db679
I20260709 13:22:00.858749 24025 raft_consensus.cc:2804] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:22:00.860294 24025 raft_consensus.cc:697] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 4 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:22:00.861177 24025 consensus_queue.cc:237] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:00.887508 22715 catalog_manager.cc:5697] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 3 to 4, leader changed from 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66) to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 4 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:00.897197 23860 tablet_bootstrap.cc:492] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:00.898319 23860 tablet_bootstrap.cc:492] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:22:00.899031 23860 ts_tablet_manager.cc:1403] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.044s	user 0.017s	sys 0.001s
I20260709 13:22:00.901623 23860 raft_consensus.cc:359] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:00.902470 23860 raft_consensus.cc:740] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:22:00.903312 23860 consensus_queue.cc:260] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:00.909658 23860 ts_tablet_manager.cc:1434] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:22:00.910560 23860 tablet_bootstrap.cc:492] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:22:00.935129 23860 tablet_bootstrap.cc:492] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:00.936080 23860 tablet_bootstrap.cc:492] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:22:00.936789 23860 ts_tablet_manager.cc:1403] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.007s
I20260709 13:22:00.939304 23860 raft_consensus.cc:359] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:00.939968 23860 raft_consensus.cc:740] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:22:00.940773 23860 consensus_queue.cc:260] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:00.942498 23860 ts_tablet_manager.cc:1434] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:22:00.943300 23860 tablet_bootstrap.cc:492] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:22:00.966921 23860 tablet_bootstrap.cc:492] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:00.967885 23860 tablet_bootstrap.cc:492] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:22:00.968950 23860 ts_tablet_manager.cc:1403] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.000s
I20260709 13:22:00.971693 23860 raft_consensus.cc:359] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:00.972398 23860 raft_consensus.cc:740] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:22:00.973078 23860 consensus_queue.cc:260] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:00.978695 23860 ts_tablet_manager.cc:1434] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:22:00.979688 23860 tablet_bootstrap.cc:492] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:22:01.061173 23860 tablet_bootstrap.cc:492] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:01.062196 23860 tablet_bootstrap.cc:492] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:22:01.062909 23860 ts_tablet_manager.cc:1403] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.083s	user 0.033s	sys 0.028s
I20260709 13:22:01.065435 23860 raft_consensus.cc:359] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:01.066179 23860 raft_consensus.cc:740] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:22:01.066898 23860 consensus_queue.cc:260] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:01.068989 23860 ts_tablet_manager.cc:1434] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:22:01.070441 23860 tablet_bootstrap.cc:492] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:22:01.089900 23860 tablet_bootstrap.cc:492] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:01.090891 23860 tablet_bootstrap.cc:492] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:22:01.091583 23860 ts_tablet_manager.cc:1403] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.004s
I20260709 13:22:01.094072 23860 raft_consensus.cc:359] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:01.094760 23860 raft_consensus.cc:740] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:22:01.095427 23860 consensus_queue.cc:260] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:01.097571 23860 ts_tablet_manager.cc:1434] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:22:01.098596 23860 tablet_bootstrap.cc:492] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:22:01.118713 23860 tablet_bootstrap.cc:492] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:22:01.119645 23860 tablet_bootstrap.cc:492] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:22:01.120289 23860 ts_tablet_manager.cc:1403] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.001s
I20260709 13:22:01.122655 23860 raft_consensus.cc:359] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:01.123260 23860 raft_consensus.cc:740] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:22:01.123957 23860 consensus_queue.cc:260] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:01.126324 23860 ts_tablet_manager.cc:1434] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 13:22:01.127153 23860 tablet_bootstrap.cc:492] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:22:01.142426 23860 tablet_bootstrap.cc:492] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:01.143424 23860 tablet_bootstrap.cc:492] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:22:01.144105 23860 ts_tablet_manager.cc:1403] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.001s
I20260709 13:22:01.146433 23860 raft_consensus.cc:359] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:01.147101 23860 raft_consensus.cc:740] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:22:01.147766 23860 consensus_queue.cc:260] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:01.148316 23921 raft_consensus.cc:3060] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:01.149791 23860 ts_tablet_manager.cc:1434] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:22:01.158916 23860 tablet_bootstrap.cc:492] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
I20260709 13:22:01.171303 23926 raft_consensus.cc:3060] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:01.186273 23860 tablet_bootstrap.cc:492] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:01.187316 23860 tablet_bootstrap.cc:492] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:22:01.187953 23860 ts_tablet_manager.cc:1403] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.008s
I20260709 13:22:01.194824 23860 raft_consensus.cc:359] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:01.195528 23860 raft_consensus.cc:740] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:22:01.196252 23860 consensus_queue.cc:260] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:01.198361 23860 ts_tablet_manager.cc:1434] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:22:01.200242 23860 tablet_bootstrap.cc:492] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679: Bootstrap starting.
W20260709 13:22:01.220496 22934 consensus_peers.cc:597] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:01.222790 23930 raft_consensus.cc:3060] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:01.254019 23926 raft_consensus.cc:3060] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:01.268201 23926 raft_consensus.cc:1275] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Refusing update from remote peer 27745ef236cd494684f62a1269411a68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:01.270674 23860 tablet_bootstrap.cc:492] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:01.271703 23860 tablet_bootstrap.cc:492] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679: Bootstrap complete.
I20260709 13:22:01.272396 23860 ts_tablet_manager.cc:1403] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679: Time spent bootstrapping tablet: real 0.072s	user 0.028s	sys 0.015s
I20260709 13:22:01.273180 23924 consensus_queue.cc:1050] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [LEADER]: Got LMP mismatch error from peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:01.275007 23860 raft_consensus.cc:359] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:01.275771 23860 raft_consensus.cc:740] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Initialized, Role: FOLLOWER
I20260709 13:22:01.276563 23860 consensus_queue.cc:260] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:01.278651 23860 ts_tablet_manager.cc:1434] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:22:01.400705 23930 raft_consensus.cc:1275] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Refusing update from remote peer 27745ef236cd494684f62a1269411a68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:22:01.402427 23925 consensus_queue.cc:1048] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:01.416365 22890 raft_consensus.cc:1275] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Refusing update from remote peer 27745ef236cd494684f62a1269411a68: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:22:01.419039 24025 consensus_queue.cc:1048] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:22:01.597424 23921 raft_consensus.cc:3060] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:09.101542 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:09.150386 23040 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a4c294dbaeb4040b157ad72102a9e56"
dest_uuid: "27745ef236cd494684f62a1269411a68"
mode: GRACEFUL
new_leader_uuid: "8ef3887fb3874f8b81079e542d252944"
 from {username='slave'} at 127.0.0.1:38870
I20260709 13:22:09.151243 23040 raft_consensus.cc:606] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Received request to transfer leadership to TS 8ef3887fb3874f8b81079e542d252944
I20260709 13:22:09.164419 22890 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6e09daa4f2dd474d96ae9c4dfa1a8758"
dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
mode: GRACEFUL
new_leader_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:53152
I20260709 13:22:09.165022 22890 raft_consensus.cc:606] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 LEADER]: Received request to transfer leadership to TS 91f33bdbdd63438294071eb3776db679
I20260709 13:22:09.167655 23040 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "89ecf8a5022f43c787aaff1cf7679264"
dest_uuid: "27745ef236cd494684f62a1269411a68"
mode: GRACEFUL
new_leader_uuid: "8ef3887fb3874f8b81079e542d252944"
 from {username='slave'} at 127.0.0.1:38870
I20260709 13:22:09.168222 23040 raft_consensus.cc:606] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Received request to transfer leadership to TS 8ef3887fb3874f8b81079e542d252944
I20260709 13:22:09.170425 23040 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e230e64c6984b72a2ba543f2a017e25"
dest_uuid: "27745ef236cd494684f62a1269411a68"
mode: GRACEFUL
new_leader_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:38870
I20260709 13:22:09.170907 23040 raft_consensus.cc:606] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 3 LEADER]: Received request to transfer leadership to TS 91f33bdbdd63438294071eb3776db679
I20260709 13:22:09.173007 23040 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d8b119f5c5844d1be439423b333ec60"
dest_uuid: "27745ef236cd494684f62a1269411a68"
mode: GRACEFUL
new_leader_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:38870
I20260709 13:22:09.173457 23040 raft_consensus.cc:606] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 4 LEADER]: Received request to transfer leadership to TS 91f33bdbdd63438294071eb3776db679
I20260709 13:22:09.175663 23040 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad"
dest_uuid: "27745ef236cd494684f62a1269411a68"
mode: GRACEFUL
new_leader_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:38870
I20260709 13:22:09.176116 23040 raft_consensus.cc:606] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 4 LEADER]: Received request to transfer leadership to TS 91f33bdbdd63438294071eb3776db679
I20260709 13:22:09.177117 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:22:09.178313 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:09.289700 24057 raft_consensus.cc:993] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23: : Instructing follower 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:09.290334 24070 raft_consensus.cc:1081] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 LEADER]: Signalling peer 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:09.291939 23921 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6e09daa4f2dd474d96ae9c4dfa1a8758"
dest_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:42318
I20260709 13:22:09.292444 23921 raft_consensus.cc:493] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:09.292850 23921 raft_consensus.cc:3060] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:09.299870 23921 raft_consensus.cc:515] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:09.303352 23921 leader_election.cc:290] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:22:09.303252 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e09daa4f2dd474d96ae9c4dfa1a8758" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:22:09.304057 23041 raft_consensus.cc:3060] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:09.306945 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e09daa4f2dd474d96ae9c4dfa1a8758" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:22:09.307595 22890 raft_consensus.cc:3055] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:22:09.307894 22890 raft_consensus.cc:740] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:22:09.308786 22890 consensus_queue.cc:260] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:09.310098 22890 raft_consensus.cc:3060] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:09.312153 23041 raft_consensus.cc:2468] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 4.
I20260709 13:22:09.313692 23853 leader_election.cc:304] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:22:09.315547 24074 raft_consensus.cc:2804] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:22:09.316833 24074 raft_consensus.cc:697] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 4 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:22:09.316897 22890 raft_consensus.cc:2468] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 4.
I20260709 13:22:09.318372 24074 consensus_queue.cc:237] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:09.332543 22714 catalog_manager.cc:5697] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 3 to 4, leader changed from 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66) to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 4 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:09.398712 24052 raft_consensus.cc:993] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68: : Instructing follower 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:09.399283 24065 raft_consensus.cc:1081] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 4 LEADER]: Signalling peer 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:09.400759 23921 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad"
dest_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:42328
I20260709 13:22:09.401170 23921 raft_consensus.cc:493] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:09.401404 23921 raft_consensus.cc:3060] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:22:09.407037 23921 raft_consensus.cc:515] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:09.408740 23921 leader_election.cc:290] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 5 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:22:09.409873 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:22:09.410764 23041 raft_consensus.cc:3055] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:22:09.411106 23041 raft_consensus.cc:740] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:22:09.411933 23041 consensus_queue.cc:260] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:09.413517 23041 raft_consensus.cc:3060] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:22:09.420992 23041 raft_consensus.cc:2468] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 5.
I20260709 13:22:09.422518 23853 leader_election.cc:304] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:22:09.423280 24074 raft_consensus.cc:2804] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:22:09.423797 24074 raft_consensus.cc:697] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 5 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:22:09.424772 24074 consensus_queue.cc:237] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:09.432757 22961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c005b0f2343140f68e3069174596c1ad" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:22:09.433337 22961 raft_consensus.cc:3060] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:22:09.438689 22714 catalog_manager.cc:5697] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 4 to 5, leader changed from 27745ef236cd494684f62a1269411a68 (127.14.23.68) to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 5 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:09.440363 22961 raft_consensus.cc:2468] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 5.
I20260709 13:22:09.471096 24065 raft_consensus.cc:993] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68: : Instructing follower 8ef3887fb3874f8b81079e542d252944 to start an election
I20260709 13:22:09.471534 24052 raft_consensus.cc:1081] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Signalling peer 8ef3887fb3874f8b81079e542d252944 to start an election
I20260709 13:22:09.473047 22961 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1a4c294dbaeb4040b157ad72102a9e56"
dest_uuid: "8ef3887fb3874f8b81079e542d252944"
 from {username='slave'} at 127.0.0.1:52354
I20260709 13:22:09.473680 22961 raft_consensus.cc:493] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:09.474110 22961 raft_consensus.cc:3060] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:09.476200 24065 raft_consensus.cc:993] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68: : Instructing follower 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:09.476643 24052 raft_consensus.cc:1081] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 3 LEADER]: Signalling peer 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:09.478025 23921 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e230e64c6984b72a2ba543f2a017e25"
dest_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:42328
I20260709 13:22:09.478453 23921 raft_consensus.cc:493] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:09.478749 23921 raft_consensus.cc:3060] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:09.479212 22961 raft_consensus.cc:515] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:22:09.481608 22961 leader_election.cc:290] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:22:09.482206 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4c294dbaeb4040b157ad72102a9e56" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:22:09.482730 22890 raft_consensus.cc:3060] T 1a4c294dbaeb4040b157ad72102a9e56 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:09.486784 23921 raft_consensus.cc:515] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:09.487505 23041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4c294dbaeb4040b157ad72102a9e56" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:22:09.488101 23041 raft_consensus.cc:3055] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:09.488408 23041 raft_consensus.cc:740] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:22:09.489183 23041 consensus_queue.cc:260] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:22:09.490451 23041 raft_consensus.cc:3060] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:09.491547 22890 raft_consensus.cc:2468] T 1a4c294dbaeb4040b157ad72102a9e56 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 2.
I20260709 13:22:09.492084 23921 leader_election.cc:290] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:22:09.492292 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e230e64c6984b72a2ba543f2a017e25" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:22:09.492923 23040 raft_consensus.cc:3055] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:22:09.493218 23040 raft_consensus.cc:740] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:22:09.493316 22934 leader_election.cc:304] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:22:09.494351 24028 raft_consensus.cc:2804] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:09.494058 23040 consensus_queue.cc:260] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:09.495374 23040 raft_consensus.cc:3060] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:09.496212 24028 raft_consensus.cc:697] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [term 2 LEADER]: Becoming Leader. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:22:09.497032 24028 consensus_queue.cc:237] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:22:09.500545 23041 raft_consensus.cc:2468] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 2.
I20260709 13:22:09.501713 22961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e230e64c6984b72a2ba543f2a017e25" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:22:09.502382 22961 raft_consensus.cc:3060] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:09.502806 23040 raft_consensus.cc:2468] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 4.
I20260709 13:22:09.508441 22714 catalog_manager.cc:5697] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 1 to 2, leader changed from 27745ef236cd494684f62a1269411a68 (127.14.23.68) to 8ef3887fb3874f8b81079e542d252944 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "8ef3887fb3874f8b81079e542d252944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:09.508942 22961 raft_consensus.cc:2468] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 4.
I20260709 13:22:09.509701 23853 leader_election.cc:304] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:22:09.510564 24074 raft_consensus.cc:2804] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:22:09.511039 24074 raft_consensus.cc:697] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 4 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:22:09.512054 24074 consensus_queue.cc:237] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:09.523701 22714 catalog_manager.cc:5697] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 3 to 4, leader changed from 27745ef236cd494684f62a1269411a68 (127.14.23.68) to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 4 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:09.554378 24065 raft_consensus.cc:993] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68: : Instructing follower 8ef3887fb3874f8b81079e542d252944 to start an election
I20260709 13:22:09.554867 24065 raft_consensus.cc:1081] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Signalling peer 8ef3887fb3874f8b81079e542d252944 to start an election
I20260709 13:22:09.556653 22963 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "89ecf8a5022f43c787aaff1cf7679264"
dest_uuid: "8ef3887fb3874f8b81079e542d252944"
 from {username='slave'} at 127.0.0.1:52354
I20260709 13:22:09.557226 22963 raft_consensus.cc:493] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:09.557595 22963 raft_consensus.cc:3060] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:09.569314 22963 raft_consensus.cc:515] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:22:09.571395 22963 leader_election.cc:290] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:22:09.571815 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ecf8a5022f43c787aaff1cf7679264" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:22:09.572592 22892 raft_consensus.cc:3060] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:09.571914 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ecf8a5022f43c787aaff1cf7679264" candidate_uuid: "8ef3887fb3874f8b81079e542d252944" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:22:09.573238 23040 raft_consensus.cc:3055] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:09.573539 23040 raft_consensus.cc:740] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:22:09.574542 23040 consensus_queue.cc:260] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:22:09.575913 23040 raft_consensus.cc:3060] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:09.579828 22892 raft_consensus.cc:2468] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 2.
I20260709 13:22:09.581480 22934 leader_election.cc:304] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:22:09.582268 24084 raft_consensus.cc:2804] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:09.582666 24084 raft_consensus.cc:697] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [term 2 LEADER]: Becoming Leader. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:22:09.583484 24084 consensus_queue.cc:237] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:22:09.584465 23040 raft_consensus.cc:2468] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef3887fb3874f8b81079e542d252944 in term 2.
I20260709 13:22:09.601564 22714 catalog_manager.cc:5697] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 reported cstate change: term changed from 1 to 2, leader changed from 27745ef236cd494684f62a1269411a68 (127.14.23.68) to 8ef3887fb3874f8b81079e542d252944 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "8ef3887fb3874f8b81079e542d252944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:09.698450 24052 raft_consensus.cc:993] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68: : Instructing follower 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:09.698975 24052 raft_consensus.cc:1081] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 4 LEADER]: Signalling peer 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:09.701319 23930 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9d8b119f5c5844d1be439423b333ec60"
dest_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:42328
I20260709 13:22:09.701834 23930 raft_consensus.cc:493] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:09.702209 23930 raft_consensus.cc:3060] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:22:09.709246 23930 raft_consensus.cc:515] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:09.711218 23930 leader_election.cc:290] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 5 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:22:09.712637 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8b119f5c5844d1be439423b333ec60" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:22:09.713622 22963 raft_consensus.cc:3060] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:22:09.712985 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d8b119f5c5844d1be439423b333ec60" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:22:09.714522 23040 raft_consensus.cc:3055] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:22:09.714888 23040 raft_consensus.cc:740] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:22:09.715710 23040 consensus_queue.cc:260] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:09.716815 23040 raft_consensus.cc:3060] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:22:09.720474 22963 raft_consensus.cc:2468] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 5.
I20260709 13:22:09.721517 23855 leader_election.cc:304] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:22:09.722344 24074 raft_consensus.cc:2804] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:22:09.722375 23040 raft_consensus.cc:2468] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 5.
I20260709 13:22:09.722901 24074 raft_consensus.cc:697] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 5 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:22:09.723793 24074 consensus_queue.cc:237] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:09.739547 22714 catalog_manager.cc:5697] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 4 to 5, leader changed from 27745ef236cd494684f62a1269411a68 (127.14.23.68) to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 5 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:09.785356 23040 raft_consensus.cc:1275] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:22:09.786681 24074 consensus_queue.cc:1048] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:09.801103 22892 raft_consensus.cc:1275] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:22:09.805441 24074 consensus_queue.cc:1048] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:09.940021 22963 raft_consensus.cc:1275] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 5 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:22:09.941126 24074 consensus_queue.cc:1048] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:09.951104 23041 raft_consensus.cc:1275] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 5 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:22:09.952735 24075 consensus_queue.cc:1048] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:22:09.991353 22890 raft_consensus.cc:1275] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Refusing update from remote peer 8ef3887fb3874f8b81079e542d252944: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:09.992589 24084 consensus_queue.cc:1048] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:10.002058 23041 raft_consensus.cc:1275] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Refusing update from remote peer 8ef3887fb3874f8b81079e542d252944: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:10.003433 24084 consensus_queue.cc:1048] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:10.022735 22892 raft_consensus.cc:1275] T 1a4c294dbaeb4040b157ad72102a9e56 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Refusing update from remote peer 8ef3887fb3874f8b81079e542d252944: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:10.023988 24104 consensus_queue.cc:1048] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:10.034643 23040 raft_consensus.cc:1275] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Refusing update from remote peer 8ef3887fb3874f8b81079e542d252944: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:10.036442 24104 consensus_queue.cc:1048] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:10.042249 22963 raft_consensus.cc:1275] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:22:10.043932 24075 consensus_queue.cc:1048] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:10.077023 23041 raft_consensus.cc:1275] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:22:10.078490 24074 consensus_queue.cc:1048] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:10.179204 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:10.214075 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:10.239913 22961 raft_consensus.cc:1275] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 5 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:22:10.241348 24097 consensus_queue.cc:1048] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:10.250965 22892 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "084ee2ebbf564ed583f84c1227d29bd4"
dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
mode: GRACEFUL
new_leader_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:53152
I20260709 13:22:10.251747 22892 raft_consensus.cc:606] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Received request to transfer leadership to TS 91f33bdbdd63438294071eb3776db679
I20260709 13:22:10.255295 23041 raft_consensus.cc:1275] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 5 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:22:10.257066 24075 consensus_queue.cc:1048] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:10.287184 22963 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0957b803701e45648f538789ec62e507"
dest_uuid: "8ef3887fb3874f8b81079e542d252944"
mode: GRACEFUL
new_leader_uuid: "27745ef236cd494684f62a1269411a68"
 from {username='slave'} at 127.0.0.1:55902
I20260709 13:22:10.287781 22963 raft_consensus.cc:606] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 2 LEADER]: Received request to transfer leadership to TS 27745ef236cd494684f62a1269411a68
I20260709 13:22:10.290449 22892 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087"
dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
mode: GRACEFUL
new_leader_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:53152
I20260709 13:22:10.291014 22892 raft_consensus.cc:606] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 LEADER]: Received request to transfer leadership to TS 91f33bdbdd63438294071eb3776db679
I20260709 13:22:10.293483 22892 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681"
dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
mode: GRACEFUL
new_leader_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:53152
I20260709 13:22:10.293988 22892 raft_consensus.cc:606] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 LEADER]: Received request to transfer leadership to TS 91f33bdbdd63438294071eb3776db679
I20260709 13:22:10.296202 22892 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c52d63f523d24195bc8127113598ff91"
dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
mode: GRACEFUL
new_leader_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:53152
I20260709 13:22:10.296629 22892 raft_consensus.cc:606] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Received request to transfer leadership to TS 91f33bdbdd63438294071eb3776db679
I20260709 13:22:10.298977 22892 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee6a9c5a5fdc42e78dbbe6b411416c1a"
dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
mode: GRACEFUL
new_leader_uuid: "27745ef236cd494684f62a1269411a68"
 from {username='slave'} at 127.0.0.1:53152
I20260709 13:22:10.299583 22892 raft_consensus.cc:606] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Received request to transfer leadership to TS 27745ef236cd494684f62a1269411a68
I20260709 13:22:10.300860 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:22:10.302138 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:10.357602 24070 raft_consensus.cc:993] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23: : Instructing follower 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:10.358304 24057 raft_consensus.cc:1081] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 LEADER]: Signalling peer 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:10.360293 23930 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681"
dest_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:42318
I20260709 13:22:10.360849 23930 raft_consensus.cc:493] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:10.361171 23930 raft_consensus.cc:3060] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:22:10.368399 23930 raft_consensus.cc:515] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:10.370384 23930 leader_election.cc:290] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 5 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:22:10.371876 22961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:22:10.372133 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ecffc3cb18a4a6eb87a67130c728681" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:22:10.372347 22961 raft_consensus.cc:3060] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:22:10.372761 22892 raft_consensus.cc:3055] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:22:10.373090 22892 raft_consensus.cc:740] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:22:10.373940 22892 consensus_queue.cc:260] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:10.375167 22892 raft_consensus.cc:3060] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:22:10.382092 22892 raft_consensus.cc:2468] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 5.
I20260709 13:22:10.385190 22961 raft_consensus.cc:2468] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 5.
I20260709 13:22:10.385468 23855 leader_election.cc:304] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:22:10.387844 24109 raft_consensus.cc:2804] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:22:10.388382 24109 raft_consensus.cc:697] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 5 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:22:10.389293 24109 consensus_queue.cc:237] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:10.394362 24057 raft_consensus.cc:993] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23: : Instructing follower 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:10.395088 24088 raft_consensus.cc:1081] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Signalling peer 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:10.397871 23926 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c52d63f523d24195bc8127113598ff91"
dest_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:42318
I20260709 13:22:10.398499 23926 raft_consensus.cc:493] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:10.398898 23926 raft_consensus.cc:3060] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:10.405803 22717 catalog_manager.cc:5697] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 4 to 5, leader changed from 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66) to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 5 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:10.406863 23926 raft_consensus.cc:515] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:10.410259 23926 leader_election.cc:290] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:22:10.410287 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52d63f523d24195bc8127113598ff91" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:22:10.411024 23040 raft_consensus.cc:3060] T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:10.411288 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52d63f523d24195bc8127113598ff91" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:22:10.411885 22890 raft_consensus.cc:3055] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:10.412168 22890 raft_consensus.cc:740] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:22:10.413022 22890 consensus_queue.cc:260] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:10.414353 22890 raft_consensus.cc:3060] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:10.418216 23040 raft_consensus.cc:2468] T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 2.
I20260709 13:22:10.419607 23853 leader_election.cc:304] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:22:10.420727 24109 raft_consensus.cc:2804] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:10.421167 22890 raft_consensus.cc:2468] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 2.
I20260709 13:22:10.421448 24109 raft_consensus.cc:697] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [term 2 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:22:10.423388 24109 consensus_queue.cc:237] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:10.434813 22717 catalog_manager.cc:5697] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 1 to 2, leader changed from 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66) to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 2 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:10.554342 24088 raft_consensus.cc:993] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23: : Instructing follower 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:10.554774 24057 raft_consensus.cc:1081] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Signalling peer 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:10.556182 23926 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "084ee2ebbf564ed583f84c1227d29bd4"
dest_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:42318
I20260709 13:22:10.556694 23926 raft_consensus.cc:493] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:10.556996 23926 raft_consensus.cc:3060] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:10.562378 23926 raft_consensus.cc:515] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:10.563931 23926 leader_election.cc:290] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:22:10.564643 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "084ee2ebbf564ed583f84c1227d29bd4" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:22:10.564868 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "084ee2ebbf564ed583f84c1227d29bd4" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:22:10.565233 23040 raft_consensus.cc:3060] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:10.565442 22890 raft_consensus.cc:3055] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:10.565788 22890 raft_consensus.cc:740] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:22:10.566500 22890 consensus_queue.cc:260] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:10.568217 22890 raft_consensus.cc:3060] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:10.571482 23040 raft_consensus.cc:2468] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 3.
I20260709 13:22:10.572476 23853 leader_election.cc:304] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:22:10.573132 24109 raft_consensus.cc:2804] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:10.573614 24109 raft_consensus.cc:697] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 3 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:22:10.574493 22890 raft_consensus.cc:2468] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 3.
I20260709 13:22:10.574535 24109 consensus_queue.cc:237] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:10.583223 22717 catalog_manager.cc:5697] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 2 to 3, leader changed from 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66) to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 3 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:10.663439 24088 raft_consensus.cc:993] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23: : Instructing follower 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:10.663954 24088 raft_consensus.cc:1081] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 LEADER]: Signalling peer 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:10.665416 23926 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087"
dest_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:42318
I20260709 13:22:10.665881 23926 raft_consensus.cc:493] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:10.666217 23926 raft_consensus.cc:3060] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:10.671730 23926 raft_consensus.cc:515] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:10.674571 23926 leader_election.cc:290] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:22:10.675521 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:22:10.676215 22890 raft_consensus.cc:3055] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:22:10.676512 22890 raft_consensus.cc:740] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:22:10.675457 22961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "206c4910f2f9410d9c27ed152cb96087" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:22:10.677177 22961 raft_consensus.cc:3060] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:10.677431 22890 consensus_queue.cc:260] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:10.678805 22890 raft_consensus.cc:3060] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:10.682713 24084 raft_consensus.cc:993] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944: : Instructing follower 27745ef236cd494684f62a1269411a68 to start an election
I20260709 13:22:10.683269 24103 raft_consensus.cc:1081] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 2 LEADER]: Signalling peer 27745ef236cd494684f62a1269411a68 to start an election
I20260709 13:22:10.685040 22890 raft_consensus.cc:2468] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 4.
I20260709 13:22:10.683050 22961 raft_consensus.cc:2468] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 4.
I20260709 13:22:10.686407 23855 leader_election.cc:304] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:22:10.687638 23040 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0957b803701e45648f538789ec62e507"
dest_uuid: "27745ef236cd494684f62a1269411a68"
 from {username='slave'} at 127.0.0.1:36850
I20260709 13:22:10.688057 24109 raft_consensus.cc:2804] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:22:10.688230 23040 raft_consensus.cc:493] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:10.688827 23040 raft_consensus.cc:3060] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:10.688625 24109 raft_consensus.cc:697] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 4 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:22:10.689851 24109 consensus_queue.cc:237] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:10.692258 24057 raft_consensus.cc:993] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23: : Instructing follower 27745ef236cd494684f62a1269411a68 to start an election
I20260709 13:22:10.692775 24057 raft_consensus.cc:1081] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Signalling peer 27745ef236cd494684f62a1269411a68 to start an election
I20260709 13:22:10.694360 23041 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee6a9c5a5fdc42e78dbbe6b411416c1a"
dest_uuid: "27745ef236cd494684f62a1269411a68"
 from {username='slave'} at 127.0.0.1:36848
I20260709 13:22:10.694868 23041 raft_consensus.cc:493] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:10.695174 23041 raft_consensus.cc:3060] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:10.698450 23040 raft_consensus.cc:515] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:22:10.701251 23041 raft_consensus.cc:515] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:22:10.701732 23040 leader_election.cc:290] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 3 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:22:10.701988 22717 catalog_manager.cc:5697] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 3 to 4, leader changed from 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66) to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 4 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:10.703320 23041 leader_election.cc:290] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 3 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:22:10.703104 22890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0957b803701e45648f538789ec62e507" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:22:10.704420 22890 raft_consensus.cc:3060] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:10.704883 22961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0957b803701e45648f538789ec62e507" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:22:10.705050 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6a9c5a5fdc42e78dbbe6b411416c1a" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:22:10.705626 22892 raft_consensus.cc:3055] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:10.705994 22892 raft_consensus.cc:740] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:22:10.706431 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6a9c5a5fdc42e78dbbe6b411416c1a" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:22:10.705480 22961 raft_consensus.cc:3055] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:10.706966 22961 raft_consensus.cc:740] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:22:10.707696 22961 consensus_queue.cc:260] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:22:10.706749 22892 consensus_queue.cc:260] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:22:10.708878 22961 raft_consensus.cc:3060] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:10.709093 22892 raft_consensus.cc:3060] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:10.711954 22963 raft_consensus.cc:3060] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:10.712911 22890 raft_consensus.cc:2468] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 3.
I20260709 13:22:10.714349 23008 leader_election.cc:304] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:22:10.715440 24052 raft_consensus.cc:2804] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:10.715873 24052 raft_consensus.cc:697] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 3 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:22:10.716604 24052 consensus_queue.cc:237] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:22:10.719283 22963 raft_consensus.cc:2468] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 3.
I20260709 13:22:10.721117 23008 leader_election.cc:304] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:22:10.722764 24115 raft_consensus.cc:2804] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:10.723251 24115 raft_consensus.cc:697] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 3 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:22:10.723531 22961 raft_consensus.cc:2468] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 3.
I20260709 13:22:10.724033 24115 consensus_queue.cc:237] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:22:10.724602 22892 raft_consensus.cc:2468] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 3.
I20260709 13:22:10.734875 22717 catalog_manager.cc:5697] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 2 to 3, leader changed from 8ef3887fb3874f8b81079e542d252944 (127.14.23.67) to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 3 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:10.749064 22717 catalog_manager.cc:5697] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 2 to 3, leader changed from 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66) to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 3 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:10.889410 22892 raft_consensus.cc:1275] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:10.890661 24109 consensus_queue.cc:1048] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:10.897730 22963 raft_consensus.cc:1275] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 5 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:22:10.899828 24109 consensus_queue.cc:1048] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:10.915397 22894 raft_consensus.cc:1275] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 5 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:22:10.917428 24075 consensus_queue.cc:1048] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:10.921080 23041 raft_consensus.cc:1275] T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:10.926970 24075 consensus_queue.cc:1048] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:11.013357 23041 raft_consensus.cc:1275] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:11.014601 24131 consensus_queue.cc:1048] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:11.032169 22894 raft_consensus.cc:1275] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:11.034171 24109 consensus_queue.cc:1048] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:11.143193 22894 raft_consensus.cc:1275] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Refusing update from remote peer 27745ef236cd494684f62a1269411a68: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:22:11.144364 24065 consensus_queue.cc:1048] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:11.149363 22963 raft_consensus.cc:1275] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Refusing update from remote peer 27745ef236cd494684f62a1269411a68: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:22:11.150859 24065 consensus_queue.cc:1048] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:11.156926 22961 raft_consensus.cc:1275] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Refusing update from remote peer 27745ef236cd494684f62a1269411a68: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:22:11.164188 24115 consensus_queue.cc:1048] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:11.174027 22894 raft_consensus.cc:1275] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Refusing update from remote peer 27745ef236cd494684f62a1269411a68: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:22:11.186760 24065 consensus_queue.cc:1048] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:11.194188 22892 raft_consensus.cc:1275] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:22:11.195614 24109 consensus_queue.cc:1048] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:11.210417 22963 raft_consensus.cc:1275] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:22:11.217407 24075 consensus_queue.cc:1048] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:11.303005 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:11.343711 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:11.380007 22894 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "745e7e3bed5b4d59949aeddcede543f3"
dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
mode: GRACEFUL
new_leader_uuid: "27745ef236cd494684f62a1269411a68"
 from {username='slave'} at 127.0.0.1:53152
I20260709 13:22:11.380609 22894 raft_consensus.cc:606] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Received request to transfer leadership to TS 27745ef236cd494684f62a1269411a68
I20260709 13:22:11.384083 23041 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f631a34c02c4de8ae5bd70760fba972"
dest_uuid: "27745ef236cd494684f62a1269411a68"
mode: GRACEFUL
new_leader_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:38870
I20260709 13:22:11.384737 23041 raft_consensus.cc:606] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Received request to transfer leadership to TS 91f33bdbdd63438294071eb3776db679
I20260709 13:22:11.387403 22963 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6"
dest_uuid: "8ef3887fb3874f8b81079e542d252944"
mode: GRACEFUL
new_leader_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:55902
I20260709 13:22:11.387986 22963 raft_consensus.cc:606] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 4 LEADER]: Received request to transfer leadership to TS 91f33bdbdd63438294071eb3776db679
I20260709 13:22:11.390443 22894 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac141e204e3f4d3c84d6651893bdfc6f"
dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
mode: GRACEFUL
new_leader_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:53152
I20260709 13:22:11.390915 22894 raft_consensus.cc:606] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Received request to transfer leadership to TS 91f33bdbdd63438294071eb3776db679
I20260709 13:22:11.393126 23041 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1934129c8764c018495eff97d43a3b5"
dest_uuid: "27745ef236cd494684f62a1269411a68"
mode: GRACEFUL
new_leader_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:38870
I20260709 13:22:11.393675 23041 raft_consensus.cc:606] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Received request to transfer leadership to TS 91f33bdbdd63438294071eb3776db679
I20260709 13:22:11.396109 22894 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d217f95123f542e886aa049a608a88d4"
dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
mode: GRACEFUL
new_leader_uuid: "27745ef236cd494684f62a1269411a68"
 from {username='slave'} at 127.0.0.1:53152
I20260709 13:22:11.396620 22894 raft_consensus.cc:606] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Received request to transfer leadership to TS 27745ef236cd494684f62a1269411a68
I20260709 13:22:11.397509 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:22:11.398809 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:11.517982 24057 raft_consensus.cc:993] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23: : Instructing follower 27745ef236cd494684f62a1269411a68 to start an election
I20260709 13:22:11.518481 24136 raft_consensus.cc:1081] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Signalling peer 27745ef236cd494684f62a1269411a68 to start an election
I20260709 13:22:11.519857 23040 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d217f95123f542e886aa049a608a88d4"
dest_uuid: "27745ef236cd494684f62a1269411a68"
 from {username='slave'} at 127.0.0.1:36848
I20260709 13:22:11.520365 23040 raft_consensus.cc:493] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:11.520678 23040 raft_consensus.cc:3060] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:11.526363 23040 raft_consensus.cc:515] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:22:11.528069 23040 leader_election.cc:290] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 3 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:22:11.528607 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d217f95123f542e886aa049a608a88d4" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:22:11.529250 22892 raft_consensus.cc:3055] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:11.529562 22892 raft_consensus.cc:740] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:22:11.529879 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d217f95123f542e886aa049a608a88d4" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:22:11.530465 22963 raft_consensus.cc:3060] T d217f95123f542e886aa049a608a88d4 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:11.530304 22892 consensus_queue.cc:260] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:22:11.531526 22892 raft_consensus.cc:3060] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:11.537256 22892 raft_consensus.cc:2468] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 3.
I20260709 13:22:11.537884 22963 raft_consensus.cc:2468] T d217f95123f542e886aa049a608a88d4 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 3.
I20260709 13:22:11.538319 23008 leader_election.cc:304] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:22:11.539011 24140 raft_consensus.cc:2804] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:11.539389 24140 raft_consensus.cc:697] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [term 3 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:22:11.540225 24140 consensus_queue.cc:237] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:22:11.552230 22715 catalog_manager.cc:5697] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 2 to 3, leader changed from 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66) to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 3 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:11.555524 24115 raft_consensus.cc:993] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68: : Instructing follower 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:11.556178 24115 raft_consensus.cc:1081] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Signalling peer 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:11.558981 23930 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f631a34c02c4de8ae5bd70760fba972"
dest_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:42328
I20260709 13:22:11.559526 23930 raft_consensus.cc:493] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:11.559885 23930 raft_consensus.cc:3060] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:11.566744 23930 raft_consensus.cc:515] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:11.568787 23930 leader_election.cc:290] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:22:11.569828 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f631a34c02c4de8ae5bd70760fba972" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:22:11.570562 23040 raft_consensus.cc:3055] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:11.570859 23040 raft_consensus.cc:740] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:22:11.569823 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f631a34c02c4de8ae5bd70760fba972" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:22:11.571614 22892 raft_consensus.cc:3060] T 8f631a34c02c4de8ae5bd70760fba972 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:11.571637 23040 consensus_queue.cc:260] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:11.573143 23040 raft_consensus.cc:3060] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:11.578656 22892 raft_consensus.cc:2468] T 8f631a34c02c4de8ae5bd70760fba972 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 2.
I20260709 13:22:11.579793 23040 raft_consensus.cc:2468] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 2.
I20260709 13:22:11.581022 23855 leader_election.cc:304] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:22:11.581875 24146 raft_consensus.cc:2804] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:11.582377 24146 raft_consensus.cc:697] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 2 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:22:11.582790 24052 raft_consensus.cc:993] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68: : Instructing follower 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:11.583348 24140 raft_consensus.cc:1081] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Signalling peer 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:11.583284 24146 consensus_queue.cc:237] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:11.585353 23930 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d1934129c8764c018495eff97d43a3b5"
dest_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:42328
I20260709 13:22:11.585780 23930 raft_consensus.cc:493] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:11.586167 23930 raft_consensus.cc:3060] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:11.592885 23930 raft_consensus.cc:515] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:11.596290 23930 leader_election.cc:290] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 election: Requested vote from peers 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:22:11.596166 22715 catalog_manager.cc:5697] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 1 to 2, leader changed from 27745ef236cd494684f62a1269411a68 (127.14.23.68) to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 2 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:11.597294 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1934129c8764c018495eff97d43a3b5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:22:11.597872 22892 raft_consensus.cc:3060] T d1934129c8764c018495eff97d43a3b5 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:11.598904 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1934129c8764c018495eff97d43a3b5" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:22:11.603762 24057 raft_consensus.cc:993] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23: : Instructing follower 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:11.604266 24057 raft_consensus.cc:1081] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Signalling peer 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:11.604763 22892 raft_consensus.cc:2468] T d1934129c8764c018495eff97d43a3b5 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 2.
I20260709 13:22:11.600455 23040 raft_consensus.cc:3055] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:11.606020 23040 raft_consensus.cc:740] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:22:11.606591 23926 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ac141e204e3f4d3c84d6651893bdfc6f"
dest_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:42318
I20260709 13:22:11.607182 23855 leader_election.cc:304] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:22:11.608448 24075 raft_consensus.cc:2804] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:11.607180 23926 raft_consensus.cc:493] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:11.611779 23040 consensus_queue.cc:260] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:11.612920 23040 raft_consensus.cc:3060] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:11.610424 23926 raft_consensus.cc:3060] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:11.622064 24075 raft_consensus.cc:697] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [term 2 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:22:11.622493 23040 raft_consensus.cc:2468] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 2.
I20260709 13:22:11.622876 23926 raft_consensus.cc:515] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:11.623167 24075 consensus_queue.cc:237] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:11.625653 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac141e204e3f4d3c84d6651893bdfc6f" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:22:11.626245 22892 raft_consensus.cc:3055] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:11.626560 22892 raft_consensus.cc:740] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:22:11.627259 22892 consensus_queue.cc:260] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:11.628247 22892 raft_consensus.cc:3060] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:11.631387 23926 leader_election.cc:290] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:22:11.631903 22965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac141e204e3f4d3c84d6651893bdfc6f" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:22:11.632504 22965 raft_consensus.cc:3060] T ac141e204e3f4d3c84d6651893bdfc6f P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:11.635249 22892 raft_consensus.cc:2468] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 2.
I20260709 13:22:11.636428 23855 leader_election.cc:304] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:22:11.637236 24075 raft_consensus.cc:2804] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:11.637706 24075 raft_consensus.cc:697] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [term 2 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:22:11.637820 22715 catalog_manager.cc:5697] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 1 to 2, leader changed from 27745ef236cd494684f62a1269411a68 (127.14.23.68) to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 2 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:11.638697 22965 raft_consensus.cc:2468] T ac141e204e3f4d3c84d6651893bdfc6f P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 2.
I20260709 13:22:11.638598 24075 consensus_queue.cc:237] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:11.656101 22715 catalog_manager.cc:5697] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 1 to 2, leader changed from 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66) to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 2 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:11.752440 24136 raft_consensus.cc:993] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23: : Instructing follower 27745ef236cd494684f62a1269411a68 to start an election
I20260709 13:22:11.752895 24057 raft_consensus.cc:1081] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Signalling peer 27745ef236cd494684f62a1269411a68 to start an election
I20260709 13:22:11.754405 23040 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "745e7e3bed5b4d59949aeddcede543f3"
dest_uuid: "27745ef236cd494684f62a1269411a68"
 from {username='slave'} at 127.0.0.1:36848
I20260709 13:22:11.754940 23040 raft_consensus.cc:493] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:11.755281 23040 raft_consensus.cc:3060] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:11.762305 23040 raft_consensus.cc:515] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:22:11.764174 23040 leader_election.cc:290] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:22:11.764691 24102 raft_consensus.cc:993] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944: : Instructing follower 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:11.765327 24102 raft_consensus.cc:1081] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 4 LEADER]: Signalling peer 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:11.765606 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "745e7e3bed5b4d59949aeddcede543f3" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:22:11.766052 22892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "745e7e3bed5b4d59949aeddcede543f3" candidate_uuid: "27745ef236cd494684f62a1269411a68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:22:11.766243 22963 raft_consensus.cc:3060] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:11.766664 22892 raft_consensus.cc:3055] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:11.766968 22892 raft_consensus.cc:740] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:22:11.767220 23926 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6"
dest_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:42302
I20260709 13:22:11.767771 23926 raft_consensus.cc:493] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:11.767729 22892 consensus_queue.cc:260] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:22:11.768304 23926 raft_consensus.cc:3060] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:22:11.769130 22892 raft_consensus.cc:3060] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:11.774003 23926 raft_consensus.cc:515] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:11.774370 22963 raft_consensus.cc:2468] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 2.
I20260709 13:22:11.775779 23926 leader_election.cc:290] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 5 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:22:11.775585 22892 raft_consensus.cc:2468] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27745ef236cd494684f62a1269411a68 in term 2.
I20260709 13:22:11.775566 23008 leader_election.cc:304] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 8ef3887fb3874f8b81079e542d252944; no voters: 
I20260709 13:22:11.776696 24115 raft_consensus.cc:2804] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:11.776619 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:22:11.777357 22894 raft_consensus.cc:3060] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:22:11.777896 24115 raft_consensus.cc:697] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [term 2 LEADER]: Becoming Leader. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:22:11.778292 22963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28cd6c417f74c9ea3a4172ad8f3c5e6" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:22:11.779000 22963 raft_consensus.cc:3055] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:22:11.778901 24115 consensus_queue.cc:237] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:22:11.779378 22963 raft_consensus.cc:740] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8ef3887fb3874f8b81079e542d252944, State: Running, Role: LEADER
I20260709 13:22:11.781065 22963 consensus_queue.cc:260] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:11.782419 22963 raft_consensus.cc:3060] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:22:11.788776 22894 raft_consensus.cc:2468] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 5.
I20260709 13:22:11.789969 23855 leader_election.cc:304] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:22:11.790864 22963 raft_consensus.cc:2468] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 5.
I20260709 13:22:11.791451 24075 raft_consensus.cc:2804] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:22:11.792013 24075 raft_consensus.cc:697] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 5 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:22:11.792826 24075 consensus_queue.cc:237] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:11.800647 22715 catalog_manager.cc:5697] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 reported cstate change: term changed from 1 to 2, leader changed from 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66) to 27745ef236cd494684f62a1269411a68 (127.14.23.68). New cstate: current_term: 2 leader_uuid: "27745ef236cd494684f62a1269411a68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:11.803406 22717 catalog_manager.cc:5697] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 4 to 5, leader changed from 8ef3887fb3874f8b81079e542d252944 (127.14.23.67) to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 5 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:11.945515 22963 raft_consensus.cc:1275] T d217f95123f542e886aa049a608a88d4 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Refusing update from remote peer 27745ef236cd494684f62a1269411a68: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:22:11.946866 24115 consensus_queue.cc:1048] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:11.967566 22894 raft_consensus.cc:1275] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Refusing update from remote peer 27745ef236cd494684f62a1269411a68: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:22:11.970199 24052 consensus_queue.cc:1048] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:12.086907 23036 raft_consensus.cc:1275] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:12.088150 24075 consensus_queue.cc:1048] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:12.099083 22894 raft_consensus.cc:1275] T d1934129c8764c018495eff97d43a3b5 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:12.101357 24097 consensus_queue.cc:1048] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:12.110610 22892 raft_consensus.cc:1275] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:12.112700 24097 consensus_queue.cc:1048] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:12.132273 22965 raft_consensus.cc:1275] T ac141e204e3f4d3c84d6651893bdfc6f P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:12.133780 24152 consensus_queue.cc:1048] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:12.151947 22892 raft_consensus.cc:1275] T 8f631a34c02c4de8ae5bd70760fba972 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:12.153375 24097 consensus_queue.cc:1048] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:12.169018 23036 raft_consensus.cc:1275] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:12.170782 24097 consensus_queue.cc:1048] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:12.320461 22892 raft_consensus.cc:1275] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Refusing update from remote peer 27745ef236cd494684f62a1269411a68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:12.321610 24052 consensus_queue.cc:1048] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:12.330814 22965 raft_consensus.cc:1275] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Refusing update from remote peer 27745ef236cd494684f62a1269411a68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:12.332427 24115 consensus_queue.cc:1048] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:12.394971 22890 raft_consensus.cc:1275] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 5 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:22:12.396456 24097 consensus_queue.cc:1048] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:12.399585 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:12.407732 22961 raft_consensus.cc:1275] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 5 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:22:12.409965 24075 consensus_queue.cc:1048] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:12.471134 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:12.507614 23036 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0957b803701e45648f538789ec62e507"
dest_uuid: "27745ef236cd494684f62a1269411a68"
mode: GRACEFUL
new_leader_uuid: "49cbf11e18ce4268a7fcca76872afb23"
 from {username='slave'} at 127.0.0.1:38870
I20260709 13:22:12.508276 23036 raft_consensus.cc:606] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 3 LEADER]: Received request to transfer leadership to TS 49cbf11e18ce4268a7fcca76872afb23
I20260709 13:22:12.511545 23036 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0e55b4e727be4089ba7454b662d03b9c"
dest_uuid: "27745ef236cd494684f62a1269411a68"
mode: GRACEFUL
new_leader_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:38870
I20260709 13:22:12.512099 23036 raft_consensus.cc:606] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 2 LEADER]: Received request to transfer leadership to TS 91f33bdbdd63438294071eb3776db679
I20260709 13:22:12.513110 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:22:12.514824 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:12.800977 24115 raft_consensus.cc:993] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68: : Instructing follower 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:12.801411 24065 raft_consensus.cc:1081] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 2 LEADER]: Signalling peer 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:12.802845 23930 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0e55b4e727be4089ba7454b662d03b9c"
dest_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:42328
I20260709 13:22:12.803328 23930 raft_consensus.cc:493] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:12.803654 23930 raft_consensus.cc:3060] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:12.812232 23930 raft_consensus.cc:515] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:12.814680 23930 leader_election.cc:290] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:22:12.814913 22965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e55b4e727be4089ba7454b662d03b9c" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:22:12.815557 22965 raft_consensus.cc:3060] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:12.815528 23036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e55b4e727be4089ba7454b662d03b9c" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:22:12.816570 23036 raft_consensus.cc:3055] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:12.816820 23036 raft_consensus.cc:740] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:22:12.817478 23036 consensus_queue.cc:260] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:12.818387 23036 raft_consensus.cc:3060] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:12.820719 22965 raft_consensus.cc:2468] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 3.
I20260709 13:22:12.821857 23855 leader_election.cc:304] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:22:12.822676 24146 raft_consensus.cc:2804] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:12.823230 24146 raft_consensus.cc:697] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [term 3 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:22:12.823967 23036 raft_consensus.cc:2468] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 3.
I20260709 13:22:12.824103 24146 consensus_queue.cc:237] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:12.834308 22717 catalog_manager.cc:5697] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 2 to 3, leader changed from 27745ef236cd494684f62a1269411a68 (127.14.23.68) to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 3 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:13.034093 24065 raft_consensus.cc:993] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68: : Instructing follower 49cbf11e18ce4268a7fcca76872afb23 to start an election
I20260709 13:22:13.034531 24115 raft_consensus.cc:1081] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 3 LEADER]: Signalling peer 49cbf11e18ce4268a7fcca76872afb23 to start an election
I20260709 13:22:13.035969 22894 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0957b803701e45648f538789ec62e507"
dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
 from {username='slave'} at 127.0.0.1:43422
I20260709 13:22:13.036422 22894 raft_consensus.cc:493] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:13.036713 22894 raft_consensus.cc:3060] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:13.041687 22894 raft_consensus.cc:515] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:22:13.043640 22894 leader_election.cc:290] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 4 election: Requested vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 27745ef236cd494684f62a1269411a68 (127.14.23.68:44509)
I20260709 13:22:13.044446 22965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0957b803701e45648f538789ec62e507" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:22:13.044605 23040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0957b803701e45648f538789ec62e507" candidate_uuid: "49cbf11e18ce4268a7fcca76872afb23" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "27745ef236cd494684f62a1269411a68"
I20260709 13:22:13.045185 23040 raft_consensus.cc:3055] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:22:13.045488 23040 raft_consensus.cc:740] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 27745ef236cd494684f62a1269411a68, State: Running, Role: LEADER
I20260709 13:22:13.045639 22965 raft_consensus.cc:3060] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:13.046295 23040 consensus_queue.cc:260] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:22:13.047520 23040 raft_consensus.cc:3060] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:13.053254 23040 raft_consensus.cc:2468] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 4.
I20260709 13:22:13.053790 22965 raft_consensus.cc:2468] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49cbf11e18ce4268a7fcca76872afb23 in term 4.
I20260709 13:22:13.054495 22858 leader_election.cc:304] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27745ef236cd494684f62a1269411a68, 49cbf11e18ce4268a7fcca76872afb23; no voters: 
I20260709 13:22:13.055369 24057 raft_consensus.cc:2804] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:22:13.055825 24057 raft_consensus.cc:697] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 LEADER]: Becoming Leader. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:22:13.056615 24057 consensus_queue.cc:237] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } }
I20260709 13:22:13.071501 22717 catalog_manager.cc:5697] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 reported cstate change: term changed from 3 to 4, leader changed from 27745ef236cd494684f62a1269411a68 (127.14.23.68) to 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66). New cstate: current_term: 4 leader_uuid: "49cbf11e18ce4268a7fcca76872afb23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:13.334444 23036 raft_consensus.cc:1275] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:13.335608 24146 consensus_queue.cc:1048] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:13.356796 22961 raft_consensus.cc:1275] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:13.358338 24152 consensus_queue.cc:1048] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:13.515906 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:13.518724 22961 raft_consensus.cc:1275] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:22:13.519919 24174 consensus_queue.cc:1048] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:13.543896 23036 raft_consensus.cc:1275] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Refusing update from remote peer 49cbf11e18ce4268a7fcca76872afb23: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:22:13.547236 24190 consensus_queue.cc:1048] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:13.588227 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:13.630333 22894 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb"
dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
mode: GRACEFUL
new_leader_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:53152
I20260709 13:22:13.630803 22894 raft_consensus.cc:606] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Received request to transfer leadership to TS 91f33bdbdd63438294071eb3776db679
I20260709 13:22:13.631816 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:22:13.632936 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:13.912109 24174 raft_consensus.cc:993] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23: : Instructing follower 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:13.912616 24190 raft_consensus.cc:1081] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Signalling peer 91f33bdbdd63438294071eb3776db679 to start an election
I20260709 13:22:13.913975 23926 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb"
dest_uuid: "91f33bdbdd63438294071eb3776db679"
 from {username='slave'} at 127.0.0.1:42318
I20260709 13:22:13.914443 23926 raft_consensus.cc:493] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:13.914695 23926 raft_consensus.cc:3060] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:13.919188 23926 raft_consensus.cc:515] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:13.920672 23926 leader_election.cc:290] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 election: Requested vote from peers 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323), 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011)
I20260709 13:22:13.921447 22894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "49cbf11e18ce4268a7fcca76872afb23"
I20260709 13:22:13.922034 22965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2b82eb33bf425d94d0d14dfba05ecb" candidate_uuid: "91f33bdbdd63438294071eb3776db679" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "8ef3887fb3874f8b81079e542d252944"
I20260709 13:22:13.922097 22894 raft_consensus.cc:3055] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:13.922627 22965 raft_consensus.cc:3060] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:13.922623 22894 raft_consensus.cc:740] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49cbf11e18ce4268a7fcca76872afb23, State: Running, Role: LEADER
I20260709 13:22:13.923557 22894 consensus_queue.cc:260] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:13.924780 22894 raft_consensus.cc:3060] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:13.929466 22965 raft_consensus.cc:2468] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 3.
I20260709 13:22:13.930806 23855 leader_election.cc:304] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679; no voters: 
I20260709 13:22:13.931320 22894 raft_consensus.cc:2468] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f33bdbdd63438294071eb3776db679 in term 3.
I20260709 13:22:13.931640 24146 raft_consensus.cc:2804] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:13.932209 24146 raft_consensus.cc:697] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 3 LEADER]: Becoming Leader. State: Replica: 91f33bdbdd63438294071eb3776db679, State: Running, Role: LEADER
I20260709 13:22:13.933285 24146 consensus_queue.cc:237] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } }
I20260709 13:22:13.943617 22717 catalog_manager.cc:5697] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 reported cstate change: term changed from 2 to 3, leader changed from 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66) to 91f33bdbdd63438294071eb3776db679 (127.14.23.65). New cstate: current_term: 3 leader_uuid: "91f33bdbdd63438294071eb3776db679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:14.464593 22894 raft_consensus.cc:1275] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:22:14.465859 24146 consensus_queue.cc:1048] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:14.480086 22965 raft_consensus.cc:1275] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Refusing update from remote peer 91f33bdbdd63438294071eb3776db679: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:22:14.485145 24205 consensus_queue.cc:1048] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:14.633697 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:14.671305 14429 tablet_server.cc:179] TabletServer@127.14.23.65:43155 shutting down...
W20260709 13:22:14.690896 22860 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 228 similar messages]
W20260709 13:22:14.702098 22860 consensus_peers.cc:597] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:14.709467 23008 consensus_peers.cc:597] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:14.712010 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:14.713091 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.713855 14429 raft_consensus.cc:2243] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.714372 14429 raft_consensus.cc:2272] T 21e1e18beffe450f87ff940a4e9165ea P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.716887 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.717504 14429 raft_consensus.cc:2243] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.717882 14429 raft_consensus.cc:2272] T 03a064be2ec640a4a8283957d0b9b98c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.720046 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.720938 14429 raft_consensus.cc:2243] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.721737 14429 raft_consensus.cc:2272] T 9d8b119f5c5844d1be439423b333ec60 P 91f33bdbdd63438294071eb3776db679 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:14.723855 22860 consensus_peers.cc:597] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:14.724635 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.725653 14429 raft_consensus.cc:2243] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.726804 14429 raft_consensus.cc:2272] T c52d63f523d24195bc8127113598ff91 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:14.727144 22860 consensus_peers.cc:597] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:14.729970 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.730965 14429 raft_consensus.cc:2243] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.731894 14429 raft_consensus.cc:2272] T d1934129c8764c018495eff97d43a3b5 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.734879 14429 tablet_replica.cc:333] stopping tablet replica
W20260709 13:22:14.735512 22934 consensus_peers.cc:597] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:14.735899 14429 raft_consensus.cc:2243] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.736387 14429 raft_consensus.cc:2272] T 0abfe80c1fa642ec8a0d075f75264816 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.738811 14429 tablet_replica.cc:333] stopping tablet replica
W20260709 13:22:14.739379 22934 consensus_peers.cc:597] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:14.739588 14429 raft_consensus.cc:2243] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.740432 14429 raft_consensus.cc:2272] T 8e230e64c6984b72a2ba543f2a017e25 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.742776 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.743383 14429 raft_consensus.cc:2243] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.743816 14429 raft_consensus.cc:2272] T 69e638dc505d4b04b802e91b8863d833 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:14.745272 23008 consensus_peers.cc:597] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:14.746189 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.746884 14429 raft_consensus.cc:2243] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.747320 14429 raft_consensus.cc:2272] T 49b752c38e1b47a489a3d720801661d1 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.749442 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.750049 14429 raft_consensus.cc:2243] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.750479 14429 raft_consensus.cc:2272] T 014651e80f9f44e599d2d8e4d308d2a6 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.753058 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.753868 14429 raft_consensus.cc:2243] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.754712 14429 raft_consensus.cc:2272] T 0e55b4e727be4089ba7454b662d03b9c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.756750 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.757340 14429 raft_consensus.cc:2243] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.757740 14429 raft_consensus.cc:2272] T b1394978547a410199b4f798a10edb2d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.759819 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.760361 14429 raft_consensus.cc:2243] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.760751 14429 raft_consensus.cc:2272] T 7c3139446c554b419fc28cad31653600 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.762944 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.763635 14429 raft_consensus.cc:2243] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.764050 14429 raft_consensus.cc:2272] T 5e7b082dcf344fbf83e0ddec4b08b6de P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.766494 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.767252 14429 raft_consensus.cc:2243] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.768006 14429 raft_consensus.cc:2272] T c005b0f2343140f68e3069174596c1ad P 91f33bdbdd63438294071eb3776db679 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.769989 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.770522 14429 raft_consensus.cc:2243] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.770928 14429 raft_consensus.cc:2272] T 505eab84b06f46ee9d336abf01353dee P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.773010 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.773546 14429 raft_consensus.cc:2243] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.774335 14429 raft_consensus.cc:2272] T ac141e204e3f4d3c84d6651893bdfc6f P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.776306 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.776849 14429 raft_consensus.cc:2243] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.777637 14429 raft_consensus.cc:2272] T bb2b82eb33bf425d94d0d14dfba05ecb P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.779927 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.780721 14429 raft_consensus.cc:2243] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.781509 14429 raft_consensus.cc:2272] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 91f33bdbdd63438294071eb3776db679 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.784138 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.785034 14429 raft_consensus.cc:2243] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:22:14.785035 23008 consensus_peers.cc:597] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:14.785956 14429 raft_consensus.cc:2272] T 8f631a34c02c4de8ae5bd70760fba972 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.788883 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.789670 14429 raft_consensus.cc:2243] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.790181 14429 raft_consensus.cc:2272] T 086a7123d3ad43d79101a7c5de5158fe P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.792965 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.793637 14429 raft_consensus.cc:2243] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.794252 14429 raft_consensus.cc:2272] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.796459 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.797015 14429 raft_consensus.cc:2243] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.797713 14429 raft_consensus.cc:2272] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.799741 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.800294 14429 raft_consensus.cc:2243] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.800690 14429 raft_consensus.cc:2272] T 536d31f168a54e19aa9aa819e6818d64 P 91f33bdbdd63438294071eb3776db679 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.803040 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.803771 14429 raft_consensus.cc:2243] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.804324 14429 raft_consensus.cc:2272] T 050e9ef11f664c849a40cbb6acb25f19 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.807312 14429 tablet_replica.cc:333] stopping tablet replica
W20260709 13:22:14.807858 23008 consensus_peers.cc:597] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:14.808110 14429 raft_consensus.cc:2243] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.808663 14429 raft_consensus.cc:2272] T 7802e4d0fd394596b42b661c9eb4c7ce P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.810715 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.811290 14429 raft_consensus.cc:2243] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.812194 14429 raft_consensus.cc:2272] T 084ee2ebbf564ed583f84c1227d29bd4 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.814766 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.815377 14429 raft_consensus.cc:2243] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.816087 14429 raft_consensus.cc:2272] T 206c4910f2f9410d9c27ed152cb96087 P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.818092 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.818656 14429 raft_consensus.cc:2243] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.819037 14429 raft_consensus.cc:2272] T 10066ccf8d174f4a9dd69253613857b8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.821321 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.822093 14429 raft_consensus.cc:2243] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.822574 14429 raft_consensus.cc:2272] T 98ea5c83aada45d3a98027069e96658c P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.825237 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.825979 14429 raft_consensus.cc:2243] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.826444 14429 raft_consensus.cc:2272] T cd78248be3954373ad4fd60b8ae7593d P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:14.826831 22860 consensus_peers.cc:597] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:14.829203 14429 tablet_replica.cc:333] stopping tablet replica
W20260709 13:22:14.829241 22860 consensus_peers.cc:597] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:14.829427 22934 consensus_peers.cc:597] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:14.830039 14429 raft_consensus.cc:2243] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.830497 14429 raft_consensus.cc:2272] T be220d19020b428a8b1cc356f32ff5ad P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.832818 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.833431 14429 raft_consensus.cc:2243] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.834044 14429 raft_consensus.cc:2272] T 989cd1a042864c419d8c10665cb8afd0 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.836596 14429 tablet_replica.cc:333] stopping tablet replica
W20260709 13:22:14.837044 23008 consensus_peers.cc:597] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:14.837393 14429 raft_consensus.cc:2243] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.837949 14429 raft_consensus.cc:2272] T b154956cc17f4ba68f906daffbcedff6 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:14.840175 22934 consensus_peers.cc:597] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:14.840523 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.841147 14429 raft_consensus.cc:2243] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.841473 14429 raft_consensus.cc:2272] T 630a5e60917545b3a8072fc8358207c5 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.843834 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.844488 14429 raft_consensus.cc:2243] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.845197 14429 raft_consensus.cc:2272] T 9ecffc3cb18a4a6eb87a67130c728681 P 91f33bdbdd63438294071eb3776db679 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.847560 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.848264 14429 raft_consensus.cc:2243] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.848721 14429 raft_consensus.cc:2272] T 78234c90229d4615ab9e2c2a811bd0f8 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:14.849536 23008 consensus_peers.cc:597] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:14.851131 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.851776 14429 raft_consensus.cc:2243] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.852221 14429 raft_consensus.cc:2272] T abacd5d0d1e54db38a397e3c35c0cb81 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.854544 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.855268 14429 raft_consensus.cc:2243] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.855834 14429 raft_consensus.cc:2272] T 6972fb562de24785b0feffe6bcd061c5 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.858794 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.859547 14429 raft_consensus.cc:2243] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.860054 14429 raft_consensus.cc:2272] T 39611d2ff3524bc0840e4fd222d96a84 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.862978 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.863744 14429 raft_consensus.cc:2243] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.864311 14429 raft_consensus.cc:2272] T 1e4148bbaf3347bb942721170efe1ecc P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.866839 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.867391 14429 raft_consensus.cc:2243] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.867779 14429 raft_consensus.cc:2272] T c8be856e3d604f8d8a1c56a88ea18643 P 91f33bdbdd63438294071eb3776db679 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.870118 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.870625 14429 raft_consensus.cc:2243] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.871009 14429 raft_consensus.cc:2272] T 50b783c62bd54d4c8f4f1795b7409958 P 91f33bdbdd63438294071eb3776db679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.873144 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.873678 14429 raft_consensus.cc:2243] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.874150 14429 raft_consensus.cc:2272] T 087d64074b5943f0a7d789ead06f4783 P 91f33bdbdd63438294071eb3776db679 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.876529 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:14.877167 14429 raft_consensus.cc:2243] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.877679 14429 raft_consensus.cc:2272] T 56df9023b48a4dc5ae4b437c358860ed P 91f33bdbdd63438294071eb3776db679 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:14.879315 22934 consensus_peers.cc:597] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:14.884505 22934 consensus_peers.cc:597] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:14.894167 22860 consensus_peers.cc:597] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:14.896595 22934 consensus_peers.cc:597] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:14.913368 22860 consensus_peers.cc:597] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:14.974035 22860 consensus_peers.cc:597] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:14.999480 23008 consensus_peers.cc:597] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:15.036165 14429 tablet_server.cc:196] TabletServer@127.14.23.65:43155 shutdown complete.
W20260709 13:22:15.066915 22934 consensus_peers.cc:597] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:15.076308 22860 consensus_peers.cc:597] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:15.102433 22860 consensus_peers.cc:597] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:15.109023 23008 consensus_peers.cc:597] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:15.120028 22934 consensus_peers.cc:597] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:15.158463 23008 consensus_peers.cc:597] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:15.166937 22934 consensus_peers.cc:597] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:15.173588 22860 consensus_peers.cc:597] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:15.223301 22934 consensus_peers.cc:597] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 -> Peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Couldn't send request to peer 91f33bdbdd63438294071eb3776db679. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:15.231541 14429 tablet_server.cc:179] TabletServer@127.14.23.66:0 shutting down...
W20260709 13:22:15.237439 23008 consensus_peers.cc:597] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 -> Peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Couldn't send request to peer 49cbf11e18ce4268a7fcca76872afb23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:15.255512 22934 consensus_peers.cc:597] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 -> Peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Couldn't send request to peer 49cbf11e18ce4268a7fcca76872afb23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:15.292281 22934 consensus_peers.cc:597] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 -> Peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Couldn't send request to peer 49cbf11e18ce4268a7fcca76872afb23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:15.296733 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:15.298374 14429 tablet_replica.cc:333] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.299485 14429 raft_consensus.cc:2243] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.301069 14429 raft_consensus.cc:2272] T b1394978547a410199b4f798a10edb2d P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.304531 14429 tablet_replica.cc:333] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.305400 14429 raft_consensus.cc:2243] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.306564 14429 raft_consensus.cc:2272] T 014651e80f9f44e599d2d8e4d308d2a6 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.311650 14429 tablet_replica.cc:333] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.312928 14429 raft_consensus.cc:2243] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.314311 14429 raft_consensus.cc:2272] T 630a5e60917545b3a8072fc8358207c5 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:15.316749 22934 consensus_peers.cc:597] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 -> Peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Couldn't send request to peer 49cbf11e18ce4268a7fcca76872afb23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:15.317111 14429 tablet_replica.cc:333] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.317866 14429 raft_consensus.cc:2243] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.318547 14429 raft_consensus.cc:2272] T b74a5704470044c1bb9cd7c18e3b6000 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.321516 14429 tablet_replica.cc:333] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.322417 14429 raft_consensus.cc:2243] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.323807 14429 raft_consensus.cc:2272] T 8ad9b8807851437d9a22c0a33f567c15 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:15.323906 22934 consensus_peers.cc:597] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 -> Peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Couldn't send request to peer 49cbf11e18ce4268a7fcca76872afb23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:15.324695 22934 consensus_peers.cc:597] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 -> Peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Couldn't send request to peer 49cbf11e18ce4268a7fcca76872afb23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:15.327214 14429 tablet_replica.cc:333] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.328328 14429 raft_consensus.cc:2243] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.328996 14429 raft_consensus.cc:2272] T c52d63f523d24195bc8127113598ff91 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.332327 14429 tablet_replica.cc:333] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.333271 14429 raft_consensus.cc:2243] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.333971 14429 raft_consensus.cc:2272] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:15.334744 23008 consensus_peers.cc:597] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 -> Peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Couldn't send request to peer 49cbf11e18ce4268a7fcca76872afb23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:15.337239 14429 tablet_replica.cc:333] T 21e1e18beffe450f87ff940a4e9165ea P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.338218 14429 raft_consensus.cc:2243] T 21e1e18beffe450f87ff940a4e9165ea P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.338943 14429 raft_consensus.cc:2272] T 21e1e18beffe450f87ff940a4e9165ea P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.342111 14429 tablet_replica.cc:333] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.342931 14429 raft_consensus.cc:2243] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.343621 14429 raft_consensus.cc:2272] T d217f95123f542e886aa049a608a88d4 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:15.345978 22934 consensus_peers.cc:597] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 -> Peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Couldn't send request to peer 49cbf11e18ce4268a7fcca76872afb23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:15.346613 14429 tablet_replica.cc:333] T d1934129c8764c018495eff97d43a3b5 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.347252 14429 raft_consensus.cc:2243] T d1934129c8764c018495eff97d43a3b5 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.347823 14429 raft_consensus.cc:2272] T d1934129c8764c018495eff97d43a3b5 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.350041 14429 tablet_replica.cc:333] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.350647 14429 raft_consensus.cc:2243] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.351122 14429 raft_consensus.cc:2272] T 745e7e3bed5b4d59949aeddcede543f3 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.353364 14429 tablet_replica.cc:333] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.353983 14429 raft_consensus.cc:2243] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.354496 14429 raft_consensus.cc:2272] T ac141e204e3f4d3c84d6651893bdfc6f P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.356593 14429 tablet_replica.cc:333] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.357266 14429 raft_consensus.cc:2243] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.357806 14429 raft_consensus.cc:2272] T bb2b82eb33bf425d94d0d14dfba05ecb P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.361982 14429 tablet_replica.cc:333] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.363060 14429 raft_consensus.cc:2243] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:22:15.363340 23008 consensus_peers.cc:597] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 -> Peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Couldn't send request to peer 49cbf11e18ce4268a7fcca76872afb23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:15.364545 14429 raft_consensus.cc:2272] T 171526d2fe264ee8b1168192bec31c7a P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.367820 14429 tablet_replica.cc:333] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.369016 14429 raft_consensus.cc:2243] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.369815 14429 raft_consensus.cc:2272] T 536d31f168a54e19aa9aa819e6818d64 P 49cbf11e18ce4268a7fcca76872afb23 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.373109 14429 tablet_replica.cc:333] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.373827 14429 raft_consensus.cc:2243] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.374476 14429 raft_consensus.cc:2272] T 206c4910f2f9410d9c27ed152cb96087 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.376865 14429 tablet_replica.cc:333] T 1a4c294dbaeb4040b157ad72102a9e56 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.377660 14429 raft_consensus.cc:2243] T 1a4c294dbaeb4040b157ad72102a9e56 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.378305 14429 raft_consensus.cc:2272] T 1a4c294dbaeb4040b157ad72102a9e56 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.381311 14429 tablet_replica.cc:333] T 7802e4d0fd394596b42b661c9eb4c7ce P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
W20260709 13:22:15.381588 22934 consensus_peers.cc:597] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944 -> Peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Couldn't send request to peer 49cbf11e18ce4268a7fcca76872afb23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:15.382151 14429 raft_consensus.cc:2243] T 7802e4d0fd394596b42b661c9eb4c7ce P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:22:15.382709 23008 consensus_peers.cc:597] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 -> Peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Couldn't send request to peer 49cbf11e18ce4268a7fcca76872afb23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:15.382859 14429 raft_consensus.cc:2272] T 7802e4d0fd394596b42b661c9eb4c7ce P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.386336 14429 tablet_replica.cc:333] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.390290 14429 raft_consensus.cc:2243] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.390939 14429 raft_consensus.cc:2272] T 050e9ef11f664c849a40cbb6acb25f19 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.395457 14429 tablet_replica.cc:333] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.396356 14429 raft_consensus.cc:2243] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:22:15.396950 23008 consensus_peers.cc:597] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 -> Peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Couldn't send request to peer 49cbf11e18ce4268a7fcca76872afb23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:15.397291 14429 raft_consensus.cc:2272] T 989cd1a042864c419d8c10665cb8afd0 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.399567 14429 tablet_replica.cc:333] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.400185 14429 raft_consensus.cc:2243] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.400684 14429 raft_consensus.cc:2272] T 44841295bfd54159a48e285582824bad P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.402900 14429 tablet_replica.cc:333] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.403509 14429 raft_consensus.cc:2243] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.404929 14429 raft_consensus.cc:2272] T be220d19020b428a8b1cc356f32ff5ad P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.407992 14429 tablet_replica.cc:333] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.408783 14429 raft_consensus.cc:2243] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.409849 14429 raft_consensus.cc:2272] T b154956cc17f4ba68f906daffbcedff6 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.412726 14429 tablet_replica.cc:333] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.413466 14429 raft_consensus.cc:2243] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.414108 14429 raft_consensus.cc:2272] T 968e16cef40146509d654274b9fd2687 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.416383 14429 tablet_replica.cc:333] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.416932 14429 raft_consensus.cc:2243] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.417775 14429 raft_consensus.cc:2272] T 98ea5c83aada45d3a98027069e96658c P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.419827 14429 tablet_replica.cc:333] T 10066ccf8d174f4a9dd69253613857b8 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.420338 14429 raft_consensus.cc:2243] T 10066ccf8d174f4a9dd69253613857b8 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.420790 14429 raft_consensus.cc:2272] T 10066ccf8d174f4a9dd69253613857b8 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.423281 14429 tablet_replica.cc:333] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.424067 14429 raft_consensus.cc:2243] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.424993 14429 raft_consensus.cc:2272] T 78234c90229d4615ab9e2c2a811bd0f8 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.427098 14429 tablet_replica.cc:333] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.427623 14429 raft_consensus.cc:2243] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.428076 14429 raft_consensus.cc:2272] T 9ecffc3cb18a4a6eb87a67130c728681 P 49cbf11e18ce4268a7fcca76872afb23 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.430085 14429 tablet_replica.cc:333] T cabd9adbf414496bbaab3946b413d313 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.430570 14429 raft_consensus.cc:2243] T cabd9adbf414496bbaab3946b413d313 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.431143 14429 raft_consensus.cc:2272] T cabd9adbf414496bbaab3946b413d313 P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.433707 14429 tablet_replica.cc:333] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.434517 14429 raft_consensus.cc:2243] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.435045 14429 raft_consensus.cc:2272] T 89ecf8a5022f43c787aaff1cf7679264 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.437047 14429 tablet_replica.cc:333] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.437625 14429 raft_consensus.cc:2243] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.438905 14429 raft_consensus.cc:2272] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.441821 14429 tablet_replica.cc:333] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.442489 14429 raft_consensus.cc:2243] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.443097 14429 raft_consensus.cc:2272] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.445614 14429 tablet_replica.cc:333] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.446368 14429 raft_consensus.cc:2243] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.447365 14429 raft_consensus.cc:2272] T 03a064be2ec640a4a8283957d0b9b98c P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.449502 14429 tablet_replica.cc:333] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.450043 14429 raft_consensus.cc:2243] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.450560 14429 raft_consensus.cc:2272] T 084ee2ebbf564ed583f84c1227d29bd4 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.452569 14429 tablet_replica.cc:333] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.453204 14429 raft_consensus.cc:2243] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.453796 14429 raft_consensus.cc:2272] T 02259962b5504debb700dd6b7c12915b P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.457036 14429 tablet_replica.cc:333] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.457813 14429 raft_consensus.cc:2243] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.458899 14429 raft_consensus.cc:2272] T 5e7b082dcf344fbf83e0ddec4b08b6de P 49cbf11e18ce4268a7fcca76872afb23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.461750 14429 tablet_replica.cc:333] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.462486 14429 raft_consensus.cc:2243] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.463690 14429 raft_consensus.cc:2272] T 94800e302e4641bb857f2d160f1f60ea P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:15.464120 23008 consensus_peers.cc:597] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 -> Peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Couldn't send request to peer 49cbf11e18ce4268a7fcca76872afb23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:15.466414 14429 tablet_replica.cc:333] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.467212 14429 raft_consensus.cc:2243] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.468371 14429 raft_consensus.cc:2272] T 505eab84b06f46ee9d336abf01353dee P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.470934 14429 tablet_replica.cc:333] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.471515 14429 raft_consensus.cc:2243] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.472000 14429 raft_consensus.cc:2272] T 7c3139446c554b419fc28cad31653600 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.474411 14429 tablet_replica.cc:333] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.474938 14429 raft_consensus.cc:2243] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.475991 14429 raft_consensus.cc:2272] T 087d64074b5943f0a7d789ead06f4783 P 49cbf11e18ce4268a7fcca76872afb23 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:15.477249 22934 consensus_peers.cc:597] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 -> Peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Couldn't send request to peer 49cbf11e18ce4268a7fcca76872afb23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:15.478933 14429 tablet_replica.cc:333] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.479637 14429 raft_consensus.cc:2243] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.480229 14429 raft_consensus.cc:2272] T 56df9023b48a4dc5ae4b437c358860ed P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.483135 14429 tablet_replica.cc:333] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.483843 14429 raft_consensus.cc:2243] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.484565 14429 raft_consensus.cc:2272] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 49cbf11e18ce4268a7fcca76872afb23 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:15.487237 23008 consensus_peers.cc:597] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 -> Peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Couldn't send request to peer 49cbf11e18ce4268a7fcca76872afb23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:15.487771 14429 tablet_replica.cc:333] T 8f631a34c02c4de8ae5bd70760fba972 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.488354 14429 raft_consensus.cc:2243] T 8f631a34c02c4de8ae5bd70760fba972 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.488883 14429 raft_consensus.cc:2272] T 8f631a34c02c4de8ae5bd70760fba972 P 49cbf11e18ce4268a7fcca76872afb23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.491137 14429 tablet_replica.cc:333] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23: stopping tablet replica
I20260709 13:22:15.491604 14429 raft_consensus.cc:2243] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.492408 14429 raft_consensus.cc:2272] T 0957b803701e45648f538789ec62e507 P 49cbf11e18ce4268a7fcca76872afb23 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:15.523047 23008 consensus_peers.cc:597] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 -> Peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Couldn't send request to peer 49cbf11e18ce4268a7fcca76872afb23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:15.532212 23008 consensus_peers.cc:597] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 -> Peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Couldn't send request to peer 49cbf11e18ce4268a7fcca76872afb23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:15.567162 14429 tablet_server.cc:196] TabletServer@127.14.23.66:0 shutdown complete.
W20260709 13:22:15.602350 23008 consensus_peers.cc:597] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 -> Peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Couldn't send request to peer 49cbf11e18ce4268a7fcca76872afb23. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.66:45323: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:15.648505 14429 tablet_server.cc:179] TabletServer@127.14.23.67:0 shutting down...
I20260709 13:22:15.690828 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:15.691607 14429 tablet_replica.cc:333] T d217f95123f542e886aa049a608a88d4 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.692610 14429 raft_consensus.cc:2243] T d217f95123f542e886aa049a608a88d4 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.693270 14429 raft_consensus.cc:2272] T d217f95123f542e886aa049a608a88d4 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.697170 14429 tablet_replica.cc:333] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.698259 14429 raft_consensus.cc:2243] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.699008 14429 raft_consensus.cc:2272] T 9d8b119f5c5844d1be439423b333ec60 P 8ef3887fb3874f8b81079e542d252944 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.702471 14429 tablet_replica.cc:333] T b1394978547a410199b4f798a10edb2d P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.703492 14429 raft_consensus.cc:2243] T b1394978547a410199b4f798a10edb2d P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.704195 14429 raft_consensus.cc:2272] T b1394978547a410199b4f798a10edb2d P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.706705 14429 tablet_replica.cc:333] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.707580 14429 raft_consensus.cc:2243] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.708271 14429 raft_consensus.cc:2272] T 0e55b4e727be4089ba7454b662d03b9c P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.711323 14429 tablet_replica.cc:333] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.712112 14429 raft_consensus.cc:2243] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.712790 14429 raft_consensus.cc:2272] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.715626 14429 tablet_replica.cc:333] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.716207 14429 raft_consensus.cc:2243] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.717108 14429 raft_consensus.cc:2272] T b74a5704470044c1bb9cd7c18e3b6000 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.719323 14429 tablet_replica.cc:333] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.719930 14429 raft_consensus.cc:2243] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.720764 14429 raft_consensus.cc:2272] T 49b752c38e1b47a489a3d720801661d1 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.722738 14429 tablet_replica.cc:333] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.723268 14429 raft_consensus.cc:2243] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.724006 14429 raft_consensus.cc:2272] T 989cd1a042864c419d8c10665cb8afd0 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.726002 14429 tablet_replica.cc:333] T 44841295bfd54159a48e285582824bad P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.726557 14429 raft_consensus.cc:2243] T 44841295bfd54159a48e285582824bad P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.726966 14429 raft_consensus.cc:2272] T 44841295bfd54159a48e285582824bad P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.728919 14429 tablet_replica.cc:333] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.729430 14429 raft_consensus.cc:2243] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.730283 14429 raft_consensus.cc:2272] T 21e1e18beffe450f87ff940a4e9165ea P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.732577 14429 tablet_replica.cc:333] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.733351 14429 raft_consensus.cc:2243] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.733954 14429 raft_consensus.cc:2272] T 087d64074b5943f0a7d789ead06f4783 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.736585 14429 tablet_replica.cc:333] T 171526d2fe264ee8b1168192bec31c7a P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.737407 14429 raft_consensus.cc:2243] T 171526d2fe264ee8b1168192bec31c7a P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.738008 14429 raft_consensus.cc:2272] T 171526d2fe264ee8b1168192bec31c7a P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.741001 14429 tablet_replica.cc:333] T ac141e204e3f4d3c84d6651893bdfc6f P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.741609 14429 raft_consensus.cc:2243] T ac141e204e3f4d3c84d6651893bdfc6f P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.742113 14429 raft_consensus.cc:2272] T ac141e204e3f4d3c84d6651893bdfc6f P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.743963 14429 tablet_replica.cc:333] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.744475 14429 raft_consensus.cc:2243] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.744913 14429 raft_consensus.cc:2272] T bb2b82eb33bf425d94d0d14dfba05ecb P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.746872 14429 tablet_replica.cc:333] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.747457 14429 raft_consensus.cc:2243] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.747912 14429 raft_consensus.cc:2272] T 1e4148bbaf3347bb942721170efe1ecc P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.749815 14429 tablet_replica.cc:333] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.750409 14429 raft_consensus.cc:2243] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.751211 14429 raft_consensus.cc:2272] T c8be856e3d604f8d8a1c56a88ea18643 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.753211 14429 tablet_replica.cc:333] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.753743 14429 raft_consensus.cc:2243] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.754235 14429 raft_consensus.cc:2272] T 9ecffc3cb18a4a6eb87a67130c728681 P 8ef3887fb3874f8b81079e542d252944 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.756170 14429 tablet_replica.cc:333] T 8ad9b8807851437d9a22c0a33f567c15 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.756721 14429 raft_consensus.cc:2243] T 8ad9b8807851437d9a22c0a33f567c15 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.757102 14429 raft_consensus.cc:2272] T 8ad9b8807851437d9a22c0a33f567c15 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.758944 14429 tablet_replica.cc:333] T 50b783c62bd54d4c8f4f1795b7409958 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.759467 14429 raft_consensus.cc:2243] T 50b783c62bd54d4c8f4f1795b7409958 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.759891 14429 raft_consensus.cc:2272] T 50b783c62bd54d4c8f4f1795b7409958 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.761709 14429 tablet_replica.cc:333] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.762243 14429 raft_consensus.cc:2243] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.763037 14429 raft_consensus.cc:2272] T 39611d2ff3524bc0840e4fd222d96a84 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.765331 14429 tablet_replica.cc:333] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.766026 14429 raft_consensus.cc:2243] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.767102 14429 raft_consensus.cc:2272] T 6972fb562de24785b0feffe6bcd061c5 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.769824 14429 tablet_replica.cc:333] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.770609 14429 raft_consensus.cc:2243] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.771098 14429 raft_consensus.cc:2272] T c005b0f2343140f68e3069174596c1ad P 8ef3887fb3874f8b81079e542d252944 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.773599 14429 tablet_replica.cc:333] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.774348 14429 raft_consensus.cc:2243] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.774893 14429 raft_consensus.cc:2272] T 02259962b5504debb700dd6b7c12915b P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:15.775604 23008 consensus_peers.cc:597] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 -> Peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Couldn't send request to peer 8ef3887fb3874f8b81079e542d252944. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:15.777451 14429 tablet_replica.cc:333] T 5e7b082dcf344fbf83e0ddec4b08b6de P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.778290 14429 raft_consensus.cc:2243] T 5e7b082dcf344fbf83e0ddec4b08b6de P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.778941 14429 raft_consensus.cc:2272] T 5e7b082dcf344fbf83e0ddec4b08b6de P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.781939 14429 tablet_replica.cc:333] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.782588 14429 raft_consensus.cc:2243] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.783367 14429 raft_consensus.cc:2272] T abacd5d0d1e54db38a397e3c35c0cb81 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.785650 14429 tablet_replica.cc:333] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.786392 14429 raft_consensus.cc:2243] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.787555 14429 raft_consensus.cc:2272] T 89ecf8a5022f43c787aaff1cf7679264 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.790118 14429 tablet_replica.cc:333] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
W20260709 13:22:15.790361 23008 consensus_peers.cc:597] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 -> Peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Couldn't send request to peer 8ef3887fb3874f8b81079e542d252944. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:15.791014 14429 raft_consensus.cc:2243] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.791566 14429 raft_consensus.cc:2272] T 745e7e3bed5b4d59949aeddcede543f3 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.793802 14429 tablet_replica.cc:333] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.794581 14429 raft_consensus.cc:2243] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.795096 14429 raft_consensus.cc:2272] T 69e638dc505d4b04b802e91b8863d833 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.797796 14429 tablet_replica.cc:333] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.798458 14429 raft_consensus.cc:2243] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.798872 14429 raft_consensus.cc:2272] T 505eab84b06f46ee9d336abf01353dee P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.800829 14429 tablet_replica.cc:333] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.801488 14429 raft_consensus.cc:2243] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.802044 14429 raft_consensus.cc:2272] T a28cd6c417f74c9ea3a4172ad8f3c5e6 P 8ef3887fb3874f8b81079e542d252944 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.803990 14429 tablet_replica.cc:333] T 086a7123d3ad43d79101a7c5de5158fe P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.804513 14429 raft_consensus.cc:2243] T 086a7123d3ad43d79101a7c5de5158fe P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.804948 14429 raft_consensus.cc:2272] T 086a7123d3ad43d79101a7c5de5158fe P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.806941 14429 tablet_replica.cc:333] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.807478 14429 raft_consensus.cc:2243] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.807986 14429 raft_consensus.cc:2272] T 206c4910f2f9410d9c27ed152cb96087 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.810606 14429 tablet_replica.cc:333] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.811213 14429 raft_consensus.cc:2243] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.812080 14429 raft_consensus.cc:2272] T 1a4c294dbaeb4040b157ad72102a9e56 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.814756 14429 tablet_replica.cc:333] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.815258 14429 raft_consensus.cc:2243] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.816185 14429 raft_consensus.cc:2272] T 968e16cef40146509d654274b9fd2687 P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.818826 14429 tablet_replica.cc:333] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.819537 14429 raft_consensus.cc:2243] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.820601 14429 raft_consensus.cc:2272] T 050e9ef11f664c849a40cbb6acb25f19 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.823230 14429 tablet_replica.cc:333] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.823849 14429 raft_consensus.cc:2243] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 13:22:15.823926 23008 consensus_peers.cc:597] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 -> Peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Couldn't send request to peer 8ef3887fb3874f8b81079e542d252944. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:15.824489 14429 raft_consensus.cc:2272] T 98ea5c83aada45d3a98027069e96658c P 8ef3887fb3874f8b81079e542d252944 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.826915 14429 tablet_replica.cc:333] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.827448 14429 raft_consensus.cc:2243] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.828220 14429 raft_consensus.cc:2272] T cd78248be3954373ad4fd60b8ae7593d P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.831539 14429 tablet_replica.cc:333] T 0abfe80c1fa642ec8a0d075f75264816 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.832083 14429 raft_consensus.cc:2243] T 0abfe80c1fa642ec8a0d075f75264816 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.832512 14429 raft_consensus.cc:2272] T 0abfe80c1fa642ec8a0d075f75264816 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.835373 14429 tablet_replica.cc:333] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.835975 14429 raft_consensus.cc:2243] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.836443 14429 raft_consensus.cc:2272] T 8e230e64c6984b72a2ba543f2a017e25 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.838549 14429 tablet_replica.cc:333] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.839031 14429 raft_consensus.cc:2243] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.839429 14429 raft_consensus.cc:2272] T 94800e302e4641bb857f2d160f1f60ea P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.841229 14429 tablet_replica.cc:333] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.841696 14429 raft_consensus.cc:2243] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.842109 14429 raft_consensus.cc:2272] T cabd9adbf414496bbaab3946b413d313 P 8ef3887fb3874f8b81079e542d252944 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.843909 14429 tablet_replica.cc:333] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.844338 14429 raft_consensus.cc:2243] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:15.844766 14429 raft_consensus.cc:2272] T 0957b803701e45648f538789ec62e507 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.846779 14429 tablet_replica.cc:333] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.847223 14429 raft_consensus.cc:2243] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.847992 14429 raft_consensus.cc:2272] T 7c3139446c554b419fc28cad31653600 P 8ef3887fb3874f8b81079e542d252944 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.849828 14429 tablet_replica.cc:333] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944: stopping tablet replica
I20260709 13:22:15.850261 14429 raft_consensus.cc:2243] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:22:15.851024 14429 raft_consensus.cc:2272] T 955a6d8dbbe94ef9be9c1f69b10675d8 P 8ef3887fb3874f8b81079e542d252944 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:15.880738 24222 raft_consensus.cc:493] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:22:15.881371 24222 raft_consensus.cc:515] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:15.883709 24223 raft_consensus.cc:493] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:22:15.884160 24222 leader_election.cc:290] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:22:15.884356 24223 raft_consensus.cc:515] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:15.887013 24223 leader_election.cc:290] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:22:15.902083 24222 raft_consensus.cc:493] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:22:15.902788 24222 raft_consensus.cc:515] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
W20260709 13:22:15.904477 23008 leader_election.cc:336] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
W20260709 13:22:15.905082 23008 leader_election.cc:336] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
W20260709 13:22:15.905723 23008 leader_election.cc:336] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Network error: Client connection negotiation failed: client connection to 127.14.23.66:45323: connect: Connection refused (error 111)
I20260709 13:22:15.906162 23008 leader_election.cc:304] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:22:15.907032 24201 raft_consensus.cc:2749] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:22:15.906450 24222 leader_election.cc:290] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
W20260709 13:22:15.912302 23008 leader_election.cc:336] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:15.912782 23008 leader_election.cc:304] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:22:15.913614 24222 raft_consensus.cc:2749] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:22:15.915693 23008 consensus_peers.cc:597] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 -> Peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Couldn't send request to peer 8ef3887fb3874f8b81079e542d252944. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:15.919970 23008 leader_election.cc:336] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Network error: recv got EOF from unknown peer (error 108)
W20260709 13:22:15.922878 23008 leader_election.cc:336] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
I20260709 13:22:15.923266 23008 leader_election.cc:304] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:22:15.923856 24222 raft_consensus.cc:2749] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:22:15.936468 14429 tablet_server.cc:196] TabletServer@127.14.23.67:0 shutdown complete.
W20260709 13:22:15.952328 23008 consensus_peers.cc:597] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 -> Peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Couldn't send request to peer 8ef3887fb3874f8b81079e542d252944. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.67:43011: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:15.957108 24222 raft_consensus.cc:493] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:22:15.957710 24222 raft_consensus.cc:515] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:15.959744 24222 leader_election.cc:290] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
W20260709 13:22:15.966652 23008 leader_election.cc:336] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
W20260709 13:22:15.967754 23008 leader_election.cc:336] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Network error: Client connection negotiation failed: client connection to 127.14.23.67:43011: connect: Connection refused (error 111)
I20260709 13:22:15.968199 23008 leader_election.cc:304] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:22:15.968901 24222 raft_consensus.cc:2749] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:22:15.970892 23008 consensus_peers.cc:597] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 -> Peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Couldn't send request to peer 8ef3887fb3874f8b81079e542d252944. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.67:43011: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:16.018666 24201 raft_consensus.cc:493] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:22:16.019321 24201 raft_consensus.cc:515] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef3887fb3874f8b81079e542d252944" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 43011 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } }
I20260709 13:22:16.021667 24201 leader_election.cc:290] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011), 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155)
I20260709 13:22:16.022624 14429 tablet_server.cc:179] TabletServer@127.14.23.68:0 shutting down...
W20260709 13:22:16.033484 23008 consensus_peers.cc:597] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 -> Peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Couldn't send request to peer 8ef3887fb3874f8b81079e542d252944. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.67:43011: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:16.034065 23008 consensus_peers.cc:597] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 -> Peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Couldn't send request to peer 8ef3887fb3874f8b81079e542d252944. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.67:43011: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:16.034394 23008 leader_election.cc:336] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Network error: Client connection negotiation failed: client connection to 127.14.23.67:43011: connect: Connection refused (error 111)
W20260709 13:22:16.035040 23008 leader_election.cc:336] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
I20260709 13:22:16.035437 23008 leader_election.cc:304] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 8ef3887fb3874f8b81079e542d252944, 91f33bdbdd63438294071eb3776db679
I20260709 13:22:16.036420 24201 raft_consensus.cc:2749] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:22:16.062440 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:16.063259 14429 tablet_replica.cc:333] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.063941 14429 raft_consensus.cc:2243] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.064452 14429 raft_consensus.cc:2272] T 014651e80f9f44e599d2d8e4d308d2a6 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.066562 14429 tablet_replica.cc:333] T b74a5704470044c1bb9cd7c18e3b6000 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.067314 14429 raft_consensus.cc:2243] T b74a5704470044c1bb9cd7c18e3b6000 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.067807 14429 raft_consensus.cc:2272] T b74a5704470044c1bb9cd7c18e3b6000 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.070283 14429 tablet_replica.cc:333] T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.070879 14429 raft_consensus.cc:2243] T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:22:16.071218 23008 consensus_peers.cc:597] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 -> Peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Couldn't send request to peer 8ef3887fb3874f8b81079e542d252944. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.67:43011: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:16.071358 14429 raft_consensus.cc:2272] T c52d63f523d24195bc8127113598ff91 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.073441 14429 tablet_replica.cc:333] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.074035 14429 raft_consensus.cc:2243] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:16.074820 14429 raft_consensus.cc:2272] T ee6a9c5a5fdc42e78dbbe6b411416c1a P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.076781 14429 tablet_replica.cc:333] T 630a5e60917545b3a8072fc8358207c5 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.077355 14429 raft_consensus.cc:2243] T 630a5e60917545b3a8072fc8358207c5 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.077768 14429 raft_consensus.cc:2272] T 630a5e60917545b3a8072fc8358207c5 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.079799 14429 tablet_replica.cc:333] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.080375 14429 raft_consensus.cc:2243] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.080793 14429 raft_consensus.cc:2272] T 9d8b119f5c5844d1be439423b333ec60 P 27745ef236cd494684f62a1269411a68 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.083624 14429 tablet_replica.cc:333] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.084316 14429 raft_consensus.cc:2243] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.084926 14429 raft_consensus.cc:2272] T 0e55b4e727be4089ba7454b662d03b9c P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.086714 24222 raft_consensus.cc:493] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91f33bdbdd63438294071eb3776db679)
I20260709 13:22:16.087337 14429 tablet_replica.cc:333] T 78234c90229d4615ab9e2c2a811bd0f8 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.087227 24222 raft_consensus.cc:515] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f33bdbdd63438294071eb3776db679" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43155 } } peers { permanent_uuid: "27745ef236cd494684f62a1269411a68" member_type: VOTER last_known_addr { host: "127.14.23.68" port: 44509 } } peers { permanent_uuid: "49cbf11e18ce4268a7fcca76872afb23" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 45323 } }
I20260709 13:22:16.088248 14429 raft_consensus.cc:2243] T 78234c90229d4615ab9e2c2a811bd0f8 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.088812 14429 raft_consensus.cc:2272] T 78234c90229d4615ab9e2c2a811bd0f8 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.089408 24222 leader_election.cc:290] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155), 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323)
I20260709 13:22:16.091625 14429 tablet_replica.cc:333] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.092515 14429 raft_consensus.cc:2243] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.093127 14429 raft_consensus.cc:2272] T 49b752c38e1b47a489a3d720801661d1 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:16.093123 23008 leader_election.cc:336] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91f33bdbdd63438294071eb3776db679 (127.14.23.65:43155): Network error: Client connection negotiation failed: client connection to 127.14.23.65:43155: connect: Connection refused (error 111)
W20260709 13:22:16.093990 23008 leader_election.cc:336] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 49cbf11e18ce4268a7fcca76872afb23 (127.14.23.66:45323): Network error: Client connection negotiation failed: client connection to 127.14.23.66:45323: connect: Connection refused (error 111)
I20260709 13:22:16.094291 23008 leader_election.cc:304] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27745ef236cd494684f62a1269411a68; no voters: 49cbf11e18ce4268a7fcca76872afb23, 91f33bdbdd63438294071eb3776db679
I20260709 13:22:16.094830 24222 raft_consensus.cc:2749] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:22:16.095870 14429 tablet_replica.cc:333] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.096455 14429 raft_consensus.cc:2243] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:16.097172 14429 raft_consensus.cc:2272] T 7802e4d0fd394596b42b661c9eb4c7ce P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.098984 14429 tablet_replica.cc:333] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.099478 14429 raft_consensus.cc:2243] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:16.100203 14429 raft_consensus.cc:2272] T d217f95123f542e886aa049a608a88d4 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.102105 14429 tablet_replica.cc:333] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.102596 14429 raft_consensus.cc:2243] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.102991 14429 raft_consensus.cc:2272] T d1934129c8764c018495eff97d43a3b5 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.104846 14429 tablet_replica.cc:333] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.105358 14429 raft_consensus.cc:2243] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.105737 14429 raft_consensus.cc:2272] T 171526d2fe264ee8b1168192bec31c7a P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.107877 14429 tablet_replica.cc:333] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.108584 14429 raft_consensus.cc:2243] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:16.109452 14429 raft_consensus.cc:2272] T 44841295bfd54159a48e285582824bad P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.111943 14429 tablet_replica.cc:333] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.112743 14429 raft_consensus.cc:2243] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.113241 14429 raft_consensus.cc:2272] T b154956cc17f4ba68f906daffbcedff6 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.115254 14429 tablet_replica.cc:333] T be220d19020b428a8b1cc356f32ff5ad P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.115779 14429 raft_consensus.cc:2243] T be220d19020b428a8b1cc356f32ff5ad P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.116216 14429 raft_consensus.cc:2272] T be220d19020b428a8b1cc356f32ff5ad P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.118299 14429 tablet_replica.cc:333] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.118831 14429 raft_consensus.cc:2243] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.119232 14429 raft_consensus.cc:2272] T cd78248be3954373ad4fd60b8ae7593d P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.121101 14429 tablet_replica.cc:333] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.121634 14429 raft_consensus.cc:2243] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:16.122448 14429 raft_consensus.cc:2272] T 10066ccf8d174f4a9dd69253613857b8 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.124396 14429 tablet_replica.cc:333] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.124931 14429 raft_consensus.cc:2243] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.125370 14429 raft_consensus.cc:2272] T c8be856e3d604f8d8a1c56a88ea18643 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.127349 14429 tablet_replica.cc:333] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.127867 14429 raft_consensus.cc:2243] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:22:16.128659 14429 raft_consensus.cc:2272] T 1e4148bbaf3347bb942721170efe1ecc P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.130719 14429 tablet_replica.cc:333] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.131317 14429 raft_consensus.cc:2243] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.131738 14429 raft_consensus.cc:2272] T 1a4c294dbaeb4040b157ad72102a9e56 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.133996 14429 tablet_replica.cc:333] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.134754 14429 raft_consensus.cc:2243] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.135439 14429 raft_consensus.cc:2272] T 968e16cef40146509d654274b9fd2687 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:16.137348 23008 consensus_peers.cc:597] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 -> Peer 8ef3887fb3874f8b81079e542d252944 (127.14.23.67:43011): Couldn't send request to peer 8ef3887fb3874f8b81079e542d252944. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.67:43011: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:16.137939 14429 tablet_replica.cc:333] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.138574 14429 raft_consensus.cc:2243] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.139042 14429 raft_consensus.cc:2272] T 8ad9b8807851437d9a22c0a33f567c15 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.141042 14429 tablet_replica.cc:333] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.141594 14429 raft_consensus.cc:2243] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:16.142398 14429 raft_consensus.cc:2272] T 50b783c62bd54d4c8f4f1795b7409958 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.144354 14429 tablet_replica.cc:333] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.144913 14429 raft_consensus.cc:2243] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.145332 14429 raft_consensus.cc:2272] T 39611d2ff3524bc0840e4fd222d96a84 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.147315 14429 tablet_replica.cc:333] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.147850 14429 raft_consensus.cc:2243] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.148272 14429 raft_consensus.cc:2272] T 6972fb562de24785b0feffe6bcd061c5 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.150264 14429 tablet_replica.cc:333] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.150805 14429 raft_consensus.cc:2243] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:16.151556 14429 raft_consensus.cc:2272] T cabd9adbf414496bbaab3946b413d313 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.153513 14429 tablet_replica.cc:333] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.154100 14429 raft_consensus.cc:2243] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:16.154906 14429 raft_consensus.cc:2272] T 0abfe80c1fa642ec8a0d075f75264816 P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.156840 14429 tablet_replica.cc:333] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.157351 14429 raft_consensus.cc:2243] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.157761 14429 raft_consensus.cc:2272] T 8e230e64c6984b72a2ba543f2a017e25 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.159622 14429 tablet_replica.cc:333] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.160130 14429 raft_consensus.cc:2243] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.160545 14429 raft_consensus.cc:2272] T 94800e302e4641bb857f2d160f1f60ea P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.162520 14429 tablet_replica.cc:333] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.163035 14429 raft_consensus.cc:2243] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.163476 14429 raft_consensus.cc:2272] T abacd5d0d1e54db38a397e3c35c0cb81 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.165405 14429 tablet_replica.cc:333] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.165903 14429 raft_consensus.cc:2243] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.166358 14429 raft_consensus.cc:2272] T 89ecf8a5022f43c787aaff1cf7679264 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.168164 14429 tablet_replica.cc:333] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.168684 14429 raft_consensus.cc:2243] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:16.169452 14429 raft_consensus.cc:2272] T 745e7e3bed5b4d59949aeddcede543f3 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.171344 14429 tablet_replica.cc:333] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.171794 14429 raft_consensus.cc:2243] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:16.172513 14429 raft_consensus.cc:2272] T 69e638dc505d4b04b802e91b8863d833 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.174384 14429 tablet_replica.cc:333] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.174871 14429 raft_consensus.cc:2243] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.175253 14429 raft_consensus.cc:2272] T 03a064be2ec640a4a8283957d0b9b98c P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.177088 14429 tablet_replica.cc:333] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.177570 14429 raft_consensus.cc:2243] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:22:16.178328 14429 raft_consensus.cc:2272] T 56df9023b48a4dc5ae4b437c358860ed P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.180112 14429 tablet_replica.cc:333] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.180572 14429 raft_consensus.cc:2243] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.180972 14429 raft_consensus.cc:2272] T 0957b803701e45648f538789ec62e507 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.182868 14429 tablet_replica.cc:333] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.183398 14429 raft_consensus.cc:2243] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:16.184157 14429 raft_consensus.cc:2272] T 086a7123d3ad43d79101a7c5de5158fe P 27745ef236cd494684f62a1269411a68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.186203 14429 tablet_replica.cc:333] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.186692 14429 raft_consensus.cc:2243] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.187094 14429 raft_consensus.cc:2272] T 8f631a34c02c4de8ae5bd70760fba972 P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.188971 14429 tablet_replica.cc:333] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.189469 14429 raft_consensus.cc:2243] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.189890 14429 raft_consensus.cc:2272] T c005b0f2343140f68e3069174596c1ad P 27745ef236cd494684f62a1269411a68 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.191738 14429 tablet_replica.cc:333] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.192230 14429 raft_consensus.cc:2243] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:16.193007 14429 raft_consensus.cc:2272] T 02259962b5504debb700dd6b7c12915b P 27745ef236cd494684f62a1269411a68 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.194959 14429 tablet_replica.cc:333] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.195423 14429 raft_consensus.cc:2243] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:22:16.196293 14429 raft_consensus.cc:2272] T 536d31f168a54e19aa9aa819e6818d64 P 27745ef236cd494684f62a1269411a68 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.198211 14429 tablet_replica.cc:333] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.198655 14429 raft_consensus.cc:2243] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.199090 14429 raft_consensus.cc:2272] T 6e09daa4f2dd474d96ae9c4dfa1a8758 P 27745ef236cd494684f62a1269411a68 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.200911 14429 tablet_replica.cc:333] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68: stopping tablet replica
I20260709 13:22:16.201345 14429 raft_consensus.cc:2243] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:16.201766 14429 raft_consensus.cc:2272] T 084ee2ebbf564ed583f84c1227d29bd4 P 27745ef236cd494684f62a1269411a68 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.266381 14429 tablet_server.cc:196] TabletServer@127.14.23.68:0 shutdown complete.
I20260709 13:22:16.327883 14429 master.cc:562] Master@127.14.23.126:41523 shutting down...
I20260709 13:22:16.353092 14429 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:16.353626 14429 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:16.353906 14429 tablet_replica.cc:333] T 00000000000000000000000000000000 P 597f4a6dec454e9386f272dfcba9216c: stopping tablet replica
I20260709 13:22:16.413390 14429 master.cc:584] Master@127.14.23.126:41523 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (37969 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:22:16.496513 14429 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.23.126:39899
I20260709 13:22:16.497611 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:16.503609 24226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:16.503597 24225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:16.504386 24228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:16.505805 14429 server_base.cc:1061] running on GCE node
I20260709 13:22:16.506927 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:16.507112 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:16.507272 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603336507252 us; error 0 us; skew 500 ppm
I20260709 13:22:16.507817 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:16.510425 14429 webserver.cc:533] Webserver started at http://127.14.23.126:32857/ using document root <none> and password file <none>
I20260709 13:22:16.510891 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:16.511044 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:16.511276 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:16.512382 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/master-0-root/instance:
uuid: "ff69dc99e4c14bbbb85e0b2b71152203"
format_stamp: "Formatted at 2026-07-09 13:22:16 on dist-test-slave-twc1"
I20260709 13:22:16.516810 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:22:16.520315 24233 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:16.521037 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.001s
I20260709 13:22:16.521353 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/master-0-root
uuid: "ff69dc99e4c14bbbb85e0b2b71152203"
format_stamp: "Formatted at 2026-07-09 13:22:16 on dist-test-slave-twc1"
I20260709 13:22:16.521626 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:16.532871 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:16.534241 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:16.572067 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.126:39899
I20260709 13:22:16.572175 24284 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.126:39899 every 8 connection(s)
I20260709 13:22:16.575977 24285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.586827 24285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203: Bootstrap starting.
I20260709 13:22:16.591671 24285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.596046 24285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203: No bootstrap required, opened a new log
I20260709 13:22:16.598323 24285 raft_consensus.cc:359] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff69dc99e4c14bbbb85e0b2b71152203" member_type: VOTER }
I20260709 13:22:16.598739 24285 raft_consensus.cc:385] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.598989 24285 raft_consensus.cc:740] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff69dc99e4c14bbbb85e0b2b71152203, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.599598 24285 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff69dc99e4c14bbbb85e0b2b71152203" member_type: VOTER }
I20260709 13:22:16.600049 24285 raft_consensus.cc:399] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:22:16.600255 24285 raft_consensus.cc:493] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:22:16.600517 24285 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.605295 24285 raft_consensus.cc:515] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff69dc99e4c14bbbb85e0b2b71152203" member_type: VOTER }
I20260709 13:22:16.605816 24285 leader_election.cc:304] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff69dc99e4c14bbbb85e0b2b71152203; no voters: 
I20260709 13:22:16.606943 24285 leader_election.cc:290] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:22:16.607342 24288 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:16.608652 24288 raft_consensus.cc:697] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203 [term 1 LEADER]: Becoming Leader. State: Replica: ff69dc99e4c14bbbb85e0b2b71152203, State: Running, Role: LEADER
I20260709 13:22:16.609364 24288 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff69dc99e4c14bbbb85e0b2b71152203" member_type: VOTER }
I20260709 13:22:16.609756 24285 sys_catalog.cc:565] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:22:16.613735 24289 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff69dc99e4c14bbbb85e0b2b71152203" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff69dc99e4c14bbbb85e0b2b71152203" member_type: VOTER } }
I20260709 13:22:16.613852 24290 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff69dc99e4c14bbbb85e0b2b71152203. Latest consensus state: current_term: 1 leader_uuid: "ff69dc99e4c14bbbb85e0b2b71152203" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff69dc99e4c14bbbb85e0b2b71152203" member_type: VOTER } }
I20260709 13:22:16.614516 24289 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203 [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:16.614771 24290 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203 [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:16.622258 24295 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:22:16.627807 24295 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:22:16.629132 14429 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:22:16.636798 24295 catalog_manager.cc:1383] Generated new cluster ID: bc068ea820ce49c6a3947a0af5b1445f
I20260709 13:22:16.637117 24295 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:22:16.656352 24295 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:22:16.657673 24295 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:22:16.671034 24295 catalog_manager.cc:6070] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203: Generated new TSK 0
I20260709 13:22:16.671669 24295 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:22:16.696070 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:16.701992 24306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:16.702768 24307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:16.704550 14429 server_base.cc:1061] running on GCE node
W20260709 13:22:16.704502 24309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:16.705983 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:16.706185 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:16.706306 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603336706295 us; error 0 us; skew 500 ppm
I20260709 13:22:16.706794 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:16.709009 14429 webserver.cc:533] Webserver started at http://127.14.23.65:38483/ using document root <none> and password file <none>
I20260709 13:22:16.709465 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:16.709623 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:16.709837 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:16.710906 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/ts-0-root/instance:
uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14"
format_stamp: "Formatted at 2026-07-09 13:22:16 on dist-test-slave-twc1"
I20260709 13:22:16.715350 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:22:16.718792 24314 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:16.719578 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 13:22:16.719893 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/ts-0-root
uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14"
format_stamp: "Formatted at 2026-07-09 13:22:16 on dist-test-slave-twc1"
I20260709 13:22:16.720167 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:16.744063 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:16.745348 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:16.746884 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:16.749241 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:16.749441 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:16.749675 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:16.749845 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:16.789654 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.65:40105
I20260709 13:22:16.789754 24376 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.65:40105 every 8 connection(s)
I20260709 13:22:16.794485 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:16.801608 24381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:16.802505 24382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:16.805131 14429 server_base.cc:1061] running on GCE node
W20260709 13:22:16.806509 24384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:16.807657 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:16.807935 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:16.807973 24377 heartbeater.cc:344] Connected to a master server at 127.14.23.126:39899
I20260709 13:22:16.808226 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603336808207 us; error 0 us; skew 500 ppm
I20260709 13:22:16.808463 24377 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:16.809006 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:16.809374 24377 heartbeater.cc:507] Master 127.14.23.126:39899 requested a full tablet report, sending...
I20260709 13:22:16.811924 14429 webserver.cc:533] Webserver started at http://127.14.23.66:34081/ using document root <none> and password file <none>
I20260709 13:22:16.812135 24250 ts_manager.cc:194] Registered new tserver with Master: 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105)
I20260709 13:22:16.812494 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:16.812762 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:16.813015 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:16.814141 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/ts-1-root/instance:
uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
format_stamp: "Formatted at 2026-07-09 13:22:16 on dist-test-slave-twc1"
I20260709 13:22:16.814239 24250 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37812
I20260709 13:22:16.819434 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:22:16.823381 24389 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:16.824160 14429 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:22:16.824460 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/ts-1-root
uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
format_stamp: "Formatted at 2026-07-09 13:22:16 on dist-test-slave-twc1"
I20260709 13:22:16.824741 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:16.856899 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:16.858199 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:16.859701 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:16.862074 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:16.862283 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:16.862512 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:16.862670 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:16.902179 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.66:41577
I20260709 13:22:16.902297 24451 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.66:41577 every 8 connection(s)
I20260709 13:22:16.906840 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:16.913833 24455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:16.914760 24456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:16.918596 24458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:16.919981 24452 heartbeater.cc:344] Connected to a master server at 127.14.23.126:39899
I20260709 13:22:16.920678 24452 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:16.922031 24452 heartbeater.cc:507] Master 127.14.23.126:39899 requested a full tablet report, sending...
I20260709 13:22:16.923846 14429 server_base.cc:1061] running on GCE node
I20260709 13:22:16.925031 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:16.925257 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:16.925429 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603336925413 us; error 0 us; skew 500 ppm
I20260709 13:22:16.926090 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:16.926723 24250 ts_manager.cc:194] Registered new tserver with Master: 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577)
I20260709 13:22:16.928320 24250 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37814
I20260709 13:22:16.928813 14429 webserver.cc:533] Webserver started at http://127.14.23.67:45933/ using document root <none> and password file <none>
I20260709 13:22:16.929438 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:16.929675 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:16.929960 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:16.931143 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/ts-2-root/instance:
uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
format_stamp: "Formatted at 2026-07-09 13:22:16 on dist-test-slave-twc1"
I20260709 13:22:16.935653 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.003s
I20260709 13:22:16.939119 24463 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:16.939908 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:22:16.940227 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/ts-2-root
uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
format_stamp: "Formatted at 2026-07-09 13:22:16 on dist-test-slave-twc1"
I20260709 13:22:16.940500 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:16.958362 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:16.959596 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:16.961082 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:16.963454 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:16.963665 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:16.963887 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:16.964035 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:17.004314 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.67:44867
I20260709 13:22:17.004403 24525 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.67:44867 every 8 connection(s)
I20260709 13:22:17.017859 24526 heartbeater.cc:344] Connected to a master server at 127.14.23.126:39899
I20260709 13:22:17.018337 24526 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:17.019073 24526 heartbeater.cc:507] Master 127.14.23.126:39899 requested a full tablet report, sending...
I20260709 13:22:17.022082 24250 ts_manager.cc:194] Registered new tserver with Master: 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:17.022289 14429 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015166639s
I20260709 13:22:17.024740 24250 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37824
I20260709 13:22:17.049589 24250 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37834:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:22:17.052484 24250 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:22:17.139858 24417 tablet_service.cc:1511] Processing CreateTablet for tablet 4e4065825e694d4fa259bbdb5538289f (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:22:17.141335 24417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e4065825e694d4fa259bbdb5538289f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.142917 24416 tablet_service.cc:1511] Processing CreateTablet for tablet cefb0660f2f84bdfb7e9b47cb002b525 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:22:17.144439 24416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cefb0660f2f84bdfb7e9b47cb002b525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.144234 24489 tablet_service.cc:1511] Processing CreateTablet for tablet 248862fab3014daba862723841b4730a (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:22:17.145119 24491 tablet_service.cc:1511] Processing CreateTablet for tablet 4e4065825e694d4fa259bbdb5538289f (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:22:17.145701 24489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 248862fab3014daba862723841b4730a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.154299 24484 tablet_service.cc:1511] Processing CreateTablet for tablet ee0eadd86f434502b7ccabf5024d1253 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:22:17.147061 24488 tablet_service.cc:1511] Processing CreateTablet for tablet a5eeb2f0a1e54fdc8c94f815d9819277 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:22:17.148833 24487 tablet_service.cc:1511] Processing CreateTablet for tablet 688b155b91274e1c9102292eb67cc0c6 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:22:17.155468 24415 tablet_service.cc:1511] Processing CreateTablet for tablet 248862fab3014daba862723841b4730a (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:22:17.155390 24491 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e4065825e694d4fa259bbdb5538289f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.156956 24487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 688b155b91274e1c9102292eb67cc0c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.157115 24415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 248862fab3014daba862723841b4730a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.152426 24485 tablet_service.cc:1511] Processing CreateTablet for tablet 332fefa9cf2f4af28b3888c8620703d9 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:22:17.159741 24485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 332fefa9cf2f4af28b3888c8620703d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.150655 24486 tablet_service.cc:1511] Processing CreateTablet for tablet 968525e758744fb0b8b1a80822467255 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:22:17.162459 24486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 968525e758744fb0b8b1a80822467255. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.164083 24414 tablet_service.cc:1511] Processing CreateTablet for tablet a5eeb2f0a1e54fdc8c94f815d9819277 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:22:17.142829 24490 tablet_service.cc:1511] Processing CreateTablet for tablet cefb0660f2f84bdfb7e9b47cb002b525 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:22:17.165753 24414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5eeb2f0a1e54fdc8c94f815d9819277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.166783 24490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cefb0660f2f84bdfb7e9b47cb002b525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.167011 24541 tablet_bootstrap.cc:492] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6: Bootstrap starting.
I20260709 13:22:17.168560 24488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5eeb2f0a1e54fdc8c94f815d9819277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.171017 24413 tablet_service.cc:1511] Processing CreateTablet for tablet 688b155b91274e1c9102292eb67cc0c6 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:22:17.172194 24484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee0eadd86f434502b7ccabf5024d1253. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.172472 24413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 688b155b91274e1c9102292eb67cc0c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.173396 24541 tablet_bootstrap.cc:654] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.179724 24417 tablet_service.cc:1511] Processing CreateTablet for tablet 968525e758744fb0b8b1a80822467255 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:22:17.180220 24412 tablet_service.cc:1511] Processing CreateTablet for tablet 332fefa9cf2f4af28b3888c8620703d9 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:22:17.181218 24417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 968525e758744fb0b8b1a80822467255. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.181892 24412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 332fefa9cf2f4af28b3888c8620703d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.188575 24342 tablet_service.cc:1511] Processing CreateTablet for tablet 4e4065825e694d4fa259bbdb5538289f (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:22:17.191146 24341 tablet_service.cc:1511] Processing CreateTablet for tablet cefb0660f2f84bdfb7e9b47cb002b525 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:22:17.193768 24340 tablet_service.cc:1511] Processing CreateTablet for tablet 248862fab3014daba862723841b4730a (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:22:17.196345 24339 tablet_service.cc:1511] Processing CreateTablet for tablet a5eeb2f0a1e54fdc8c94f815d9819277 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:22:17.198983 24338 tablet_service.cc:1511] Processing CreateTablet for tablet 688b155b91274e1c9102292eb67cc0c6 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:22:17.203238 24336 tablet_service.cc:1511] Processing CreateTablet for tablet 332fefa9cf2f4af28b3888c8620703d9 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:22:17.205331 24334 tablet_service.cc:1511] Processing CreateTablet for tablet 7c88fba243c24e7db4b3e7530fc9e881 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:22:17.208019 24335 tablet_service.cc:1511] Processing CreateTablet for tablet ee0eadd86f434502b7ccabf5024d1253 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:22:17.201596 24337 tablet_service.cc:1511] Processing CreateTablet for tablet 968525e758744fb0b8b1a80822467255 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:22:17.210968 24342 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e4065825e694d4fa259bbdb5538289f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.211645 24337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 968525e758744fb0b8b1a80822467255. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.214342 24334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c88fba243c24e7db4b3e7530fc9e881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.218609 24338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 688b155b91274e1c9102292eb67cc0c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.221443 24335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee0eadd86f434502b7ccabf5024d1253. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.222404 24339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5eeb2f0a1e54fdc8c94f815d9819277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.223543 24336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 332fefa9cf2f4af28b3888c8620703d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.224212 24341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cefb0660f2f84bdfb7e9b47cb002b525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.225203 24340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 248862fab3014daba862723841b4730a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.208063 24541 tablet_bootstrap.cc:492] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6: No bootstrap required, opened a new log
I20260709 13:22:17.241057 24541 ts_tablet_manager.cc:1403] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6: Time spent bootstrapping tablet: real 0.074s	user 0.011s	sys 0.007s
I20260709 13:22:17.271021 24541 raft_consensus.cc:359] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } }
I20260709 13:22:17.271835 24541 raft_consensus.cc:385] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.272140 24541 raft_consensus.cc:740] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.278241 24541 consensus_queue.cc:260] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } }
I20260709 13:22:17.279098 24483 tablet_service.cc:1511] Processing CreateTablet for tablet 7c88fba243c24e7db4b3e7530fc9e881 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:22:17.280815 24483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c88fba243c24e7db4b3e7530fc9e881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.286221 24411 tablet_service.cc:1511] Processing CreateTablet for tablet 7c88fba243c24e7db4b3e7530fc9e881 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:22:17.288010 24411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c88fba243c24e7db4b3e7530fc9e881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.290928 24417 tablet_service.cc:1511] Processing CreateTablet for tablet ee0eadd86f434502b7ccabf5024d1253 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:22:17.292359 24417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee0eadd86f434502b7ccabf5024d1253. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.305851 24333 tablet_service.cc:1511] Processing CreateTablet for tablet fe46fad433bd4a1d8070be4a4f4fdcf6 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:22:17.307684 24333 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe46fad433bd4a1d8070be4a4f4fdcf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.315129 24541 ts_tablet_manager.cc:1434] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6: Time spent starting tablet: real 0.074s	user 0.006s	sys 0.001s
I20260709 13:22:17.316000 24452 heartbeater.cc:499] Master 127.14.23.126:39899 was elected leader, sending a full tablet report...
I20260709 13:22:17.317608 24541 tablet_bootstrap.cc:492] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Bootstrap starting.
I20260709 13:22:17.344257 24541 tablet_bootstrap.cc:654] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.389318 24415 tablet_service.cc:1511] Processing CreateTablet for tablet fe46fad433bd4a1d8070be4a4f4fdcf6 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:22:17.390759 24415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe46fad433bd4a1d8070be4a4f4fdcf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.399194 24542 tablet_bootstrap.cc:492] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168: Bootstrap starting.
W20260709 13:22:17.406239 24453 tablet.cc:2406] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:22:17.410790 24545 tablet_bootstrap.cc:492] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap starting.
I20260709 13:22:17.414314 24541 tablet_bootstrap.cc:492] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6: No bootstrap required, opened a new log
I20260709 13:22:17.415004 24541 ts_tablet_manager.cc:1403] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Time spent bootstrapping tablet: real 0.098s	user 0.020s	sys 0.008s
I20260709 13:22:17.418242 24542 tablet_bootstrap.cc:654] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.418462 24488 tablet_service.cc:1511] Processing CreateTablet for tablet fe46fad433bd4a1d8070be4a4f4fdcf6 (DEFAULT_TABLE table=test-workload [id=21f9e0aa5e77403688c2d8448e5ffc05]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:22:17.418915 24541 raft_consensus.cc:359] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.419754 24541 raft_consensus.cc:385] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.420082 24541 raft_consensus.cc:740] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.420136 24488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe46fad433bd4a1d8070be4a4f4fdcf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:17.420882 24541 consensus_queue.cc:260] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.423233 24541 ts_tablet_manager.cc:1434] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.005s
I20260709 13:22:17.424376 24541 tablet_bootstrap.cc:492] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6: Bootstrap starting.
I20260709 13:22:17.430696 24541 tablet_bootstrap.cc:654] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.431094 24545 tablet_bootstrap.cc:654] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.444461 24545 tablet_bootstrap.cc:492] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: No bootstrap required, opened a new log
I20260709 13:22:17.444895 24545 ts_tablet_manager.cc:1403] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent bootstrapping tablet: real 0.035s	user 0.007s	sys 0.004s
I20260709 13:22:17.445380 24542 tablet_bootstrap.cc:492] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168: No bootstrap required, opened a new log
I20260709 13:22:17.445827 24542 ts_tablet_manager.cc:1403] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168: Time spent bootstrapping tablet: real 0.047s	user 0.007s	sys 0.019s
I20260709 13:22:17.447328 24545 raft_consensus.cc:359] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.448122 24545 raft_consensus.cc:385] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.448213 24541 tablet_bootstrap.cc:492] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6: No bootstrap required, opened a new log
I20260709 13:22:17.448472 24545 raft_consensus.cc:740] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.448238 24542 raft_consensus.cc:359] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } }
I20260709 13:22:17.448973 24541 ts_tablet_manager.cc:1403] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.009s
I20260709 13:22:17.449199 24542 raft_consensus.cc:385] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.449532 24542 raft_consensus.cc:740] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.449425 24545 consensus_queue.cc:260] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.450352 24542 consensus_queue.cc:260] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } }
I20260709 13:22:17.451876 24377 heartbeater.cc:499] Master 127.14.23.126:39899 was elected leader, sending a full tablet report...
I20260709 13:22:17.452304 24545 ts_tablet_manager.cc:1434] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:22:17.452071 24541 raft_consensus.cc:359] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.452847 24541 raft_consensus.cc:385] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.453140 24541 raft_consensus.cc:740] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.453372 24545 tablet_bootstrap.cc:492] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap starting.
I20260709 13:22:17.454022 24541 consensus_queue.cc:260] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.456588 24541 ts_tablet_manager.cc:1434] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:22:17.457669 24541 tablet_bootstrap.cc:492] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Bootstrap starting.
I20260709 13:22:17.458546 24526 heartbeater.cc:499] Master 127.14.23.126:39899 was elected leader, sending a full tablet report...
I20260709 13:22:17.459821 24545 tablet_bootstrap.cc:654] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.464198 24541 tablet_bootstrap.cc:654] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.469642 24545 tablet_bootstrap.cc:492] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14: No bootstrap required, opened a new log
I20260709 13:22:17.469712 24541 tablet_bootstrap.cc:492] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6: No bootstrap required, opened a new log
I20260709 13:22:17.470299 24545 ts_tablet_manager.cc:1403] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.008s
I20260709 13:22:17.470386 24541 ts_tablet_manager.cc:1403] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Time spent bootstrapping tablet: real 0.013s	user 0.012s	sys 0.000s
I20260709 13:22:17.473178 24545 raft_consensus.cc:359] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } }
I20260709 13:22:17.473984 24545 raft_consensus.cc:385] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.473608 24541 raft_consensus.cc:359] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.474324 24542 ts_tablet_manager.cc:1434] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.005s
I20260709 13:22:17.474463 24541 raft_consensus.cc:385] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.474880 24541 raft_consensus.cc:740] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.475592 24542 tablet_bootstrap.cc:492] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168: Bootstrap starting.
I20260709 13:22:17.475672 24541 consensus_queue.cc:260] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.478017 24541 ts_tablet_manager.cc:1434] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:22:17.474327 24545 raft_consensus.cc:740] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.479302 24541 tablet_bootstrap.cc:492] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Bootstrap starting.
I20260709 13:22:17.478977 24545 consensus_queue.cc:260] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } }
I20260709 13:22:17.481134 24545 ts_tablet_manager.cc:1434] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 13:22:17.482133 24545 tablet_bootstrap.cc:492] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap starting.
I20260709 13:22:17.485689 24541 tablet_bootstrap.cc:654] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.490712 24542 tablet_bootstrap.cc:654] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.491550 24541 tablet_bootstrap.cc:492] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6: No bootstrap required, opened a new log
I20260709 13:22:17.492147 24541 ts_tablet_manager.cc:1403] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Time spent bootstrapping tablet: real 0.013s	user 0.007s	sys 0.003s
I20260709 13:22:17.495246 24541 raft_consensus.cc:359] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.495977 24541 raft_consensus.cc:385] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.496272 24541 raft_consensus.cc:740] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.496371 24545 tablet_bootstrap.cc:654] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.497325 24541 consensus_queue.cc:260] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.499960 24541 ts_tablet_manager.cc:1434] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 13:22:17.501101 24541 tablet_bootstrap.cc:492] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Bootstrap starting.
I20260709 13:22:17.508186 24541 tablet_bootstrap.cc:654] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.520707 24541 tablet_bootstrap.cc:492] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6: No bootstrap required, opened a new log
I20260709 13:22:17.521471 24541 ts_tablet_manager.cc:1403] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.005s
I20260709 13:22:17.523586 24542 tablet_bootstrap.cc:492] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168: No bootstrap required, opened a new log
I20260709 13:22:17.524082 24545 tablet_bootstrap.cc:492] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: No bootstrap required, opened a new log
I20260709 13:22:17.524178 24542 ts_tablet_manager.cc:1403] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168: Time spent bootstrapping tablet: real 0.049s	user 0.012s	sys 0.022s
I20260709 13:22:17.524762 24545 ts_tablet_manager.cc:1403] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent bootstrapping tablet: real 0.043s	user 0.014s	sys 0.019s
I20260709 13:22:17.524696 24541 raft_consensus.cc:359] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.525627 24541 raft_consensus.cc:385] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.525954 24541 raft_consensus.cc:740] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.526938 24542 raft_consensus.cc:359] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.526741 24541 consensus_queue.cc:260] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.527534 24542 raft_consensus.cc:385] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.527837 24542 raft_consensus.cc:740] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.527896 24545 raft_consensus.cc:359] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.528652 24545 raft_consensus.cc:385] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.528977 24545 raft_consensus.cc:740] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.528785 24542 consensus_queue.cc:260] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.529708 24541 ts_tablet_manager.cc:1434] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:22:17.529768 24545 consensus_queue.cc:260] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.530977 24541 tablet_bootstrap.cc:492] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Bootstrap starting.
I20260709 13:22:17.532140 24545 ts_tablet_manager.cc:1434] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:22:17.533210 24545 tablet_bootstrap.cc:492] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap starting.
I20260709 13:22:17.533969 24542 ts_tablet_manager.cc:1434] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:22:17.534988 24542 tablet_bootstrap.cc:492] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168: Bootstrap starting.
I20260709 13:22:17.538028 24541 tablet_bootstrap.cc:654] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.539456 24545 tablet_bootstrap.cc:654] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.540947 24542 tablet_bootstrap.cc:654] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.554816 24541 tablet_bootstrap.cc:492] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6: No bootstrap required, opened a new log
I20260709 13:22:17.555471 24541 ts_tablet_manager.cc:1403] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.005s
I20260709 13:22:17.558342 24541 raft_consensus.cc:359] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.559121 24541 raft_consensus.cc:385] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.559481 24541 raft_consensus.cc:740] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.560176 24541 consensus_queue.cc:260] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.562090 24541 ts_tablet_manager.cc:1434] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.006s
I20260709 13:22:17.563086 24541 tablet_bootstrap.cc:492] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Bootstrap starting.
I20260709 13:22:17.568681 24545 tablet_bootstrap.cc:492] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: No bootstrap required, opened a new log
I20260709 13:22:17.569284 24545 ts_tablet_manager.cc:1403] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent bootstrapping tablet: real 0.036s	user 0.012s	sys 0.020s
I20260709 13:22:17.569622 24542 tablet_bootstrap.cc:492] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168: No bootstrap required, opened a new log
I20260709 13:22:17.570214 24542 ts_tablet_manager.cc:1403] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168: Time spent bootstrapping tablet: real 0.035s	user 0.016s	sys 0.009s
I20260709 13:22:17.571303 24541 tablet_bootstrap.cc:654] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.572326 24545 raft_consensus.cc:359] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.573122 24545 raft_consensus.cc:385] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.573505 24545 raft_consensus.cc:740] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.573153 24542 raft_consensus.cc:359] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.573998 24542 raft_consensus.cc:385] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.574306 24542 raft_consensus.cc:740] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.574434 24545 consensus_queue.cc:260] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.575124 24542 consensus_queue.cc:260] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.577235 24542 ts_tablet_manager.cc:1434] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:22:17.577237 24545 ts_tablet_manager.cc:1434] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:22:17.578358 24545 tablet_bootstrap.cc:492] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap starting.
I20260709 13:22:17.578358 24542 tablet_bootstrap.cc:492] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168: Bootstrap starting.
I20260709 13:22:17.585103 24542 tablet_bootstrap.cc:654] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.585103 24545 tablet_bootstrap.cc:654] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.588526 24544 raft_consensus.cc:493] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.589207 24544 raft_consensus.cc:515] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.597368 24544 leader_election.cc:290] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:17.598635 24542 tablet_bootstrap.cc:492] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168: No bootstrap required, opened a new log
I20260709 13:22:17.598773 24545 tablet_bootstrap.cc:492] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: No bootstrap required, opened a new log
I20260709 13:22:17.599359 24542 ts_tablet_manager.cc:1403] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.009s
I20260709 13:22:17.599411 24545 ts_tablet_manager.cc:1403] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.003s
I20260709 13:22:17.602006 24550 raft_consensus.cc:493] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.602496 24550 raft_consensus.cc:515] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.602846 24542 raft_consensus.cc:359] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.603596 24542 raft_consensus.cc:385] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.603904 24542 raft_consensus.cc:740] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.604673 24542 consensus_queue.cc:260] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.607002 24542 ts_tablet_manager.cc:1434] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:22:17.608072 24542 tablet_bootstrap.cc:492] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168: Bootstrap starting.
I20260709 13:22:17.614792 24542 tablet_bootstrap.cc:654] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.615547 24550 leader_election.cc:290] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:17.619537 24544 raft_consensus.cc:493] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.620317 24544 raft_consensus.cc:515] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.622816 24544 leader_election.cc:290] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
W20260709 13:22:17.624626 24298 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:17.624846 24551 raft_consensus.cc:493] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.625604 24551 raft_consensus.cc:515] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.634830 24545 raft_consensus.cc:359] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.635830 24545 raft_consensus.cc:385] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.636137 24545 raft_consensus.cc:740] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.636847 24551 leader_election.cc:290] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577)
I20260709 13:22:17.668478 24551 raft_consensus.cc:493] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.669148 24551 raft_consensus.cc:515] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.671636 24551 leader_election.cc:290] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577)
I20260709 13:22:17.676985 24501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c88fba243c24e7db4b3e7530fc9e881" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" is_pre_election: true
W20260709 13:22:17.679210 24317 leader_election.cc:343] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.686328 24427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c88fba243c24e7db4b3e7530fc9e881" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" is_pre_election: true
I20260709 13:22:17.687281 24427 raft_consensus.cc:2468] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in term 0.
I20260709 13:22:17.689051 24316 leader_election.cc:304] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 3c997d7a5c7f499abf3cb90c38f0ebe6; no voters: 97364f1f8ccd429e9c9d22c09ecfd168
I20260709 13:22:17.690249 24550 raft_consensus.cc:2804] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.690675 24550 raft_consensus.cc:493] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.691007 24550 raft_consensus.cc:3060] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.700326 24550 raft_consensus.cc:515] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.703660 24427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c88fba243c24e7db4b3e7530fc9e881" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
I20260709 13:22:17.704345 24427 raft_consensus.cc:3060] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.710283 24551 raft_consensus.cc:493] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.710983 24551 raft_consensus.cc:515] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } }
I20260709 13:22:17.713305 24551 leader_election.cc:290] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577), 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105)
I20260709 13:22:17.717772 24544 raft_consensus.cc:493] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.718590 24544 raft_consensus.cc:515] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.720484 24542 tablet_bootstrap.cc:492] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168: No bootstrap required, opened a new log
I20260709 13:22:17.721083 24542 ts_tablet_manager.cc:1403] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168: Time spent bootstrapping tablet: real 0.113s	user 0.050s	sys 0.058s
I20260709 13:22:17.720932 24501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c88fba243c24e7db4b3e7530fc9e881" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
W20260709 13:22:17.722596 24317 leader_election.cc:343] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.724280 24542 raft_consensus.cc:359] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.725194 24542 raft_consensus.cc:385] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.725497 24542 raft_consensus.cc:740] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.721089 24544 leader_election.cc:290] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:17.727025 24542 consensus_queue.cc:260] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.729734 24542 ts_tablet_manager.cc:1434] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:22:17.730772 24542 tablet_bootstrap.cc:492] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168: Bootstrap starting.
I20260709 13:22:17.736985 24542 tablet_bootstrap.cc:654] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.747684 24550 leader_election.cc:290] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 election: Requested vote from peers 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:17.748338 24550 raft_consensus.cc:493] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.748569 24542 tablet_bootstrap.cc:492] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168: No bootstrap required, opened a new log
I20260709 13:22:17.749109 24542 ts_tablet_manager.cc:1403] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.004s
I20260709 13:22:17.717832 24427 raft_consensus.cc:2468] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in term 1.
I20260709 13:22:17.752245 24542 raft_consensus.cc:359] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.753059 24542 raft_consensus.cc:385] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.753372 24542 raft_consensus.cc:740] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.748831 24550 raft_consensus.cc:515] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.636951 24545 consensus_queue.cc:260] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.755668 24550 leader_election.cc:290] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:17.756057 24544 raft_consensus.cc:493] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.756210 24550 raft_consensus.cc:493] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.756603 24544 raft_consensus.cc:515] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } }
I20260709 13:22:17.757593 24542 consensus_queue.cc:260] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.758932 24544 leader_election.cc:290] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867), 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105)
I20260709 13:22:17.759675 24542 ts_tablet_manager.cc:1434] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:22:17.760692 24542 tablet_bootstrap.cc:492] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168: Bootstrap starting.
I20260709 13:22:17.766207 24501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c88fba243c24e7db4b3e7530fc9e881" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" is_pre_election: true
I20260709 13:22:17.766990 24542 tablet_bootstrap.cc:654] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.766914 24501 raft_consensus.cc:2468] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 0.
I20260709 13:22:17.767668 24426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "688b155b91274e1c9102292eb67cc0c6" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" is_pre_election: true
I20260709 13:22:17.768483 24426 raft_consensus.cc:2468] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 0.
I20260709 13:22:17.756657 24550 raft_consensus.cc:515] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } }
I20260709 13:22:17.770100 24426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4065825e694d4fa259bbdb5538289f" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" is_pre_election: true
I20260709 13:22:17.770781 24426 raft_consensus.cc:2468] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 0.
I20260709 13:22:17.771057 24500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5eeb2f0a1e54fdc8c94f815d9819277" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" is_pre_election: true
I20260709 13:22:17.771337 24550 leader_election.cc:290] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867), 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577)
I20260709 13:22:17.770100 24501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "688b155b91274e1c9102292eb67cc0c6" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" is_pre_election: true
I20260709 13:22:17.772282 24501 raft_consensus.cc:2468] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 0.
I20260709 13:22:17.773111 24500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4065825e694d4fa259bbdb5538289f" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" is_pre_election: true
I20260709 13:22:17.773746 24500 raft_consensus.cc:2468] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 0.
I20260709 13:22:17.774262 24465 leader_election.cc:304] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 
I20260709 13:22:17.780105 24427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968525e758744fb0b8b1a80822467255" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" is_pre_election: true
I20260709 13:22:17.780982 24427 raft_consensus.cc:2468] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 0.
I20260709 13:22:17.784256 24545 ts_tablet_manager.cc:1434] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent starting tablet: real 0.184s	user 0.007s	sys 0.001s
I20260709 13:22:17.784754 24500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "688b155b91274e1c9102292eb67cc0c6" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" is_pre_election: true
I20260709 13:22:17.785656 24500 raft_consensus.cc:2468] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in term 0.
I20260709 13:22:17.787449 24392 leader_election.cc:304] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 
W20260709 13:22:17.788771 24392 leader_election.cc:343] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.789635 24392 leader_election.cc:304] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 
I20260709 13:22:17.793234 24544 raft_consensus.cc:2764] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:22:17.793623 24352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968525e758744fb0b8b1a80822467255" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" is_pre_election: true
I20260709 13:22:17.794063 24544 raft_consensus.cc:2804] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.794552 24352 raft_consensus.cc:2468] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 0.
I20260709 13:22:17.794548 24544 raft_consensus.cc:493] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.798563 24544 raft_consensus.cc:3060] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.799332 24352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "688b155b91274e1c9102292eb67cc0c6" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" is_pre_election: true
I20260709 13:22:17.800271 24352 raft_consensus.cc:2468] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 0.
I20260709 13:22:17.806656 24464 leader_election.cc:304] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 
I20260709 13:22:17.809247 24316 leader_election.cc:304] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 3c997d7a5c7f499abf3cb90c38f0ebe6; no voters: 97364f1f8ccd429e9c9d22c09ecfd168
I20260709 13:22:17.811681 24351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c88fba243c24e7db4b3e7530fc9e881" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" is_pre_election: true
I20260709 13:22:17.812748 24351 raft_consensus.cc:2393] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in current term 1: Already voted for candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in this term.
I20260709 13:22:17.811528 24427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "688b155b91274e1c9102292eb67cc0c6" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" is_pre_election: true
I20260709 13:22:17.814606 24544 raft_consensus.cc:515] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.801934 24551 raft_consensus.cc:2804] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.815805 24551 raft_consensus.cc:493] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.815995 24351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "688b155b91274e1c9102292eb67cc0c6" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" is_pre_election: true
I20260709 13:22:17.816690 24427 raft_consensus.cc:2393] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in current term 1: Already voted for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in this term.
I20260709 13:22:17.815214 24426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4065825e694d4fa259bbdb5538289f" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" is_pre_election: true
I20260709 13:22:17.817632 24426 raft_consensus.cc:2468] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in term 0.
I20260709 13:22:17.820011 24350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5eeb2f0a1e54fdc8c94f815d9819277" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" is_pre_election: true
I20260709 13:22:17.821276 24500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4065825e694d4fa259bbdb5538289f" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" is_pre_election: true
I20260709 13:22:17.822073 24500 raft_consensus.cc:2468] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in term 0.
I20260709 13:22:17.822093 24350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4065825e694d4fa259bbdb5538289f" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" is_pre_election: true
I20260709 13:22:17.824064 24545 tablet_bootstrap.cc:492] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap starting.
I20260709 13:22:17.824083 24350 raft_consensus.cc:2468] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 0.
W20260709 13:22:17.825635 24390 leader_election.cc:343] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.826249 24390 leader_election.cc:304] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6; no voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 97364f1f8ccd429e9c9d22c09ecfd168
I20260709 13:22:17.823166 24316 leader_election.cc:304] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 3c997d7a5c7f499abf3cb90c38f0ebe6; no voters: 
I20260709 13:22:17.816799 24351 raft_consensus.cc:2468] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 0.
I20260709 13:22:17.829573 24550 raft_consensus.cc:2804] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.829990 24550 raft_consensus.cc:493] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.830345 24550 raft_consensus.cc:3060] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.830147 24574 raft_consensus.cc:2804] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:17.832284 24574 raft_consensus.cc:697] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 LEADER]: Becoming Leader. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Running, Role: LEADER
I20260709 13:22:17.834101 24352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4065825e694d4fa259bbdb5538289f" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" is_pre_election: true
I20260709 13:22:17.828189 24551 raft_consensus.cc:3060] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.838814 24550 raft_consensus.cc:515] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } }
I20260709 13:22:17.825667 24317 leader_election.cc:304] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 
I20260709 13:22:17.840042 24352 raft_consensus.cc:2393] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97364f1f8ccd429e9c9d22c09ecfd168 in current term 1: Already voted for candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in this term.
I20260709 13:22:17.838593 24545 tablet_bootstrap.cc:654] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.835515 24544 leader_election.cc:290] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 election: Requested vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:17.842078 24550 leader_election.cc:290] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 election: Requested vote from peers 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867), 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577)
I20260709 13:22:17.842320 24544 raft_consensus.cc:493] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.842856 24550 raft_consensus.cc:2804] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.843241 24550 raft_consensus.cc:493] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.844226 24580 raft_consensus.cc:2749] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:17.844256 24550 raft_consensus.cc:3060] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.844619 24427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4065825e694d4fa259bbdb5538289f" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
I20260709 13:22:17.845371 24427 raft_consensus.cc:3060] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.845782 24390 leader_election.cc:304] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 3c997d7a5c7f499abf3cb90c38f0ebe6; no voters: 
I20260709 13:22:17.846196 24352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "688b155b91274e1c9102292eb67cc0c6" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14"
I20260709 13:22:17.846796 24579 raft_consensus.cc:2804] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.847170 24579 raft_consensus.cc:493] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.847436 24579 raft_consensus.cc:3060] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:17.844117 24465 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.66:41577, user_credentials={real_user=slave}} blocked reactor thread for 70173us
I20260709 13:22:17.843370 24544 raft_consensus.cc:515] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.851997 24542 tablet_bootstrap.cc:492] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168: No bootstrap required, opened a new log
I20260709 13:22:17.852578 24542 ts_tablet_manager.cc:1403] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168: Time spent bootstrapping tablet: real 0.092s	user 0.028s	sys 0.036s
I20260709 13:22:17.852836 24541 tablet_bootstrap.cc:492] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6: No bootstrap required, opened a new log
I20260709 13:22:17.853123 24351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248862fab3014daba862723841b4730a" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" is_pre_election: true
I20260709 13:22:17.853343 24541 ts_tablet_manager.cc:1403] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Time spent bootstrapping tablet: real 0.290s	user 0.079s	sys 0.164s
I20260709 13:22:17.855062 24579 raft_consensus.cc:515] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.855628 24542 raft_consensus.cc:359] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.856408 24542 raft_consensus.cc:385] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.856757 24574 consensus_queue.cc:237] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.856612 24541 raft_consensus.cc:359] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.862318 24541 raft_consensus.cc:385] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.862607 24541 raft_consensus.cc:740] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.863296 24541 consensus_queue.cc:260] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.864326 24427 raft_consensus.cc:2468] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in term 1.
I20260709 13:22:17.865326 24541 ts_tablet_manager.cc:1434] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.002s
W20260709 13:22:17.858124 24390 leader_election.cc:343] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.850634 24465 leader_election.cc:304] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14
I20260709 13:22:17.858146 24351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968525e758744fb0b8b1a80822467255" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14"
I20260709 13:22:17.867170 24351 raft_consensus.cc:3060] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.867534 24541 tablet_bootstrap.cc:492] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Bootstrap starting.
I20260709 13:22:17.868266 24545 tablet_bootstrap.cc:492] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14: No bootstrap required, opened a new log
I20260709 13:22:17.868786 24545 ts_tablet_manager.cc:1403] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent bootstrapping tablet: real 0.045s	user 0.011s	sys 0.000s
I20260709 13:22:17.858786 24550 raft_consensus.cc:515] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.856707 24542 raft_consensus.cc:740] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.871011 24542 consensus_queue.cc:260] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
W20260709 13:22:17.872136 24392 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.67:44867, user_credentials={real_user=slave}} blocked reactor thread for 82575.2us
I20260709 13:22:17.859411 24579 leader_election.cc:290] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 1 election: Requested vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577)
I20260709 13:22:17.872989 24352 raft_consensus.cc:2393] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in current term 1: Already voted for candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in this term.
I20260709 13:22:17.873982 24550 leader_election.cc:290] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 election: Requested vote from peers 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:17.874511 24579 raft_consensus.cc:2804] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.874905 24579 raft_consensus.cc:493] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.866142 24316 leader_election.cc:304] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 3c997d7a5c7f499abf3cb90c38f0ebe6; no voters: 
I20260709 13:22:17.875231 24579 raft_consensus.cc:3060] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.875541 24501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "688b155b91274e1c9102292eb67cc0c6" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
I20260709 13:22:17.878328 24541 tablet_bootstrap.cc:654] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.870078 24580 raft_consensus.cc:2764] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:22:17.866811 24500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4065825e694d4fa259bbdb5538289f" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
I20260709 13:22:17.880312 24499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "688b155b91274e1c9102292eb67cc0c6" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
I20260709 13:22:17.882959 24351 raft_consensus.cc:2468] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 1.
I20260709 13:22:17.883831 24392 leader_election.cc:304] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6; no voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 97364f1f8ccd429e9c9d22c09ecfd168
I20260709 13:22:17.884276 24464 leader_election.cc:304] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 
I20260709 13:22:17.884354 24544 leader_election.cc:290] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:17.887274 24544 raft_consensus.cc:2749] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:22:17.888121 24583 raft_consensus.cc:2804] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:17.888386 24542 ts_tablet_manager.cc:1434] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.008s
I20260709 13:22:17.888831 24499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248862fab3014daba862723841b4730a" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" is_pre_election: true
I20260709 13:22:17.889261 24583 raft_consensus.cc:697] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 LEADER]: Becoming Leader. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Running, Role: LEADER
I20260709 13:22:17.889845 24426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968525e758744fb0b8b1a80822467255" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
I20260709 13:22:17.890540 24426 raft_consensus.cc:3060] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.890183 24583 consensus_queue.cc:237] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.892992 24579 raft_consensus.cc:515] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } }
I20260709 13:22:17.897356 24500 raft_consensus.cc:2393] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in current term 1: Already voted for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in this term.
W20260709 13:22:17.897519 24392 leader_election.cc:343] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.898488 24426 raft_consensus.cc:2468] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 1.
I20260709 13:22:17.885238 24550 raft_consensus.cc:2804] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:17.899169 24550 raft_consensus.cc:697] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 LEADER]: Becoming Leader. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Running, Role: LEADER
I20260709 13:22:17.899978 24550 consensus_queue.cc:237] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } }
I20260709 13:22:17.884596 24545 raft_consensus.cc:359] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.886165 24580 raft_consensus.cc:493] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.886013 24427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "688b155b91274e1c9102292eb67cc0c6" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
I20260709 13:22:17.902380 24427 raft_consensus.cc:2393] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in current term 1: Already voted for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in this term.
I20260709 13:22:17.902060 24580 raft_consensus.cc:515] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.904469 24580 leader_election.cc:290] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:17.906229 24542 tablet_bootstrap.cc:492] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168: Bootstrap starting.
I20260709 13:22:17.898119 24392 leader_election.cc:304] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6; no voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 97364f1f8ccd429e9c9d22c09ecfd168
I20260709 13:22:17.861536 24551 raft_consensus.cc:515] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.901338 24545 raft_consensus.cc:385] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.909706 24545 raft_consensus.cc:740] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.910046 24501 raft_consensus.cc:2393] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in current term 1: Already voted for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in this term.
I20260709 13:22:17.906263 24351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cefb0660f2f84bdfb7e9b47cb002b525" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" is_pre_election: true
I20260709 13:22:17.911736 24545 consensus_queue.cc:260] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.913774 24501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cefb0660f2f84bdfb7e9b47cb002b525" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" is_pre_election: true
I20260709 13:22:17.913326 24247 catalog_manager.cc:5697] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:17.914695 24501 raft_consensus.cc:2468] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 0.
I20260709 13:22:17.914908 24579 leader_election.cc:290] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 1 election: Requested vote from peers 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577), 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105)
I20260709 13:22:17.913223 24317 leader_election.cc:304] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14; no voters: 3c997d7a5c7f499abf3cb90c38f0ebe6, 97364f1f8ccd429e9c9d22c09ecfd168
I20260709 13:22:17.916591 24352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4065825e694d4fa259bbdb5538289f" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14"
W20260709 13:22:17.917003 24390 leader_election.cc:343] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.910490 24580 raft_consensus.cc:2749] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:17.917235 24545 ts_tablet_manager.cc:1434] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
I20260709 13:22:17.918275 24392 leader_election.cc:304] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14
I20260709 13:22:17.919320 24545 tablet_bootstrap.cc:492] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap starting.
I20260709 13:22:17.919746 24580 raft_consensus.cc:2804] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.920249 24580 raft_consensus.cc:493] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.920396 24352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "688b155b91274e1c9102292eb67cc0c6" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14"
I20260709 13:22:17.913887 24426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4065825e694d4fa259bbdb5538289f" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
I20260709 13:22:17.920768 24580 raft_consensus.cc:3060] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.917083 24574 raft_consensus.cc:2749] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:22:17.921279 24426 raft_consensus.cc:2393] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 97364f1f8ccd429e9c9d22c09ecfd168 in current term 1: Already voted for candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in this term.
I20260709 13:22:17.923429 24352 raft_consensus.cc:2393] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 97364f1f8ccd429e9c9d22c09ecfd168 in current term 1: Already voted for candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in this term.
I20260709 13:22:17.922732 24427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "688b155b91274e1c9102292eb67cc0c6" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
I20260709 13:22:17.925694 24427 raft_consensus.cc:2393] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 97364f1f8ccd429e9c9d22c09ecfd168 in current term 1: Already voted for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in this term.
I20260709 13:22:17.927615 24465 leader_election.cc:304] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 3c997d7a5c7f499abf3cb90c38f0ebe6
I20260709 13:22:17.928566 24584 raft_consensus.cc:493] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.928964 24465 leader_election.cc:304] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 3c997d7a5c7f499abf3cb90c38f0ebe6
I20260709 13:22:17.929116 24584 raft_consensus.cc:515] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.930384 24579 raft_consensus.cc:2749] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:22:17.929994 24551 leader_election.cc:290] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 1 election: Requested vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577)
I20260709 13:22:17.934511 24545 tablet_bootstrap.cc:654] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.933449 24544 raft_consensus.cc:493] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.934913 24579 raft_consensus.cc:2749] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:22:17.936321 24544 raft_consensus.cc:515] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.938017 24541 tablet_bootstrap.cc:492] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6: No bootstrap required, opened a new log
I20260709 13:22:17.938657 24541 ts_tablet_manager.cc:1403] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Time spent bootstrapping tablet: real 0.071s	user 0.018s	sys 0.004s
I20260709 13:22:17.939724 24352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee0eadd86f434502b7ccabf5024d1253" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" is_pre_election: true
I20260709 13:22:17.940441 24544 leader_election.cc:290] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:17.941157 24352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "332fefa9cf2f4af28b3888c8620703d9" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" is_pre_election: true
W20260709 13:22:17.941363 24464 leader_election.cc:343] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.941468 24541 raft_consensus.cc:359] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.941859 24352 raft_consensus.cc:2468] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 0.
I20260709 13:22:17.936798 24584 leader_election.cc:290] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577)
I20260709 13:22:17.942337 24541 raft_consensus.cc:385] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.942102 24501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "332fefa9cf2f4af28b3888c8620703d9" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" is_pre_election: true
I20260709 13:22:17.942976 24541 raft_consensus.cc:740] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.944047 24541 consensus_queue.cc:260] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.944238 24501 raft_consensus.cc:2468] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 0.
I20260709 13:22:17.943737 24390 leader_election.cc:304] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 3c997d7a5c7f499abf3cb90c38f0ebe6; no voters: 
I20260709 13:22:17.946729 24427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee0eadd86f434502b7ccabf5024d1253" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" is_pre_election: true
I20260709 13:22:17.947410 24427 raft_consensus.cc:2468] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 0.
I20260709 13:22:17.948773 24578 raft_consensus.cc:2804] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.949098 24541 ts_tablet_manager.cc:1434] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:22:17.950246 24541 tablet_bootstrap.cc:492] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Bootstrap starting.
I20260709 13:22:17.950116 24465 leader_election.cc:304] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14
I20260709 13:22:17.951319 24583 raft_consensus.cc:2804] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.951668 24583 raft_consensus.cc:493] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.951983 24583 raft_consensus.cc:3060] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.952936 24249 catalog_manager.cc:5697] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:17.955832 24541 tablet_bootstrap.cc:654] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.959168 24542 tablet_bootstrap.cc:654] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.958913 24580 raft_consensus.cc:515] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.961977 24352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cefb0660f2f84bdfb7e9b47cb002b525" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14"
I20260709 13:22:17.949210 24578 raft_consensus.cc:493] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:22:17.963513 24390 leader_election.cc:343] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.963539 24578 raft_consensus.cc:3060] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.964942 24580 leader_election.cc:290] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 election: Requested vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:17.967594 24501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cefb0660f2f84bdfb7e9b47cb002b525" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
I20260709 13:22:17.968271 24501 raft_consensus.cc:3060] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.973227 24578 raft_consensus.cc:515] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.983307 24352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "332fefa9cf2f4af28b3888c8620703d9" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14"
I20260709 13:22:17.984007 24352 raft_consensus.cc:3060] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.984247 24583 raft_consensus.cc:515] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:17.989218 24583 leader_election.cc:290] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 1 election: Requested vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577)
I20260709 13:22:17.990047 24351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee0eadd86f434502b7ccabf5024d1253" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14"
I20260709 13:22:17.990196 24427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee0eadd86f434502b7ccabf5024d1253" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
I20260709 13:22:17.990962 24427 raft_consensus.cc:3060] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:17.993078 24464 leader_election.cc:343] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.995076 24541 tablet_bootstrap.cc:492] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6: No bootstrap required, opened a new log
I20260709 13:22:17.995833 24541 ts_tablet_manager.cc:1403] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Time spent bootstrapping tablet: real 0.046s	user 0.009s	sys 0.009s
I20260709 13:22:17.998976 24541 raft_consensus.cc:359] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.001828 24500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "332fefa9cf2f4af28b3888c8620703d9" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
I20260709 13:22:18.003237 24500 raft_consensus.cc:3060] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.006624 24501 raft_consensus.cc:2468] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 1.
I20260709 13:22:18.008323 24392 leader_election.cc:304] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14
I20260709 13:22:18.009763 24580 raft_consensus.cc:2804] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:18.002519 24578 leader_election.cc:290] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 election: Requested vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:18.010353 24580 raft_consensus.cc:697] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 LEADER]: Becoming Leader. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Running, Role: LEADER
I20260709 13:22:18.011341 24580 consensus_queue.cc:237] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.014314 24500 raft_consensus.cc:2468] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 1.
I20260709 13:22:18.016361 24392 leader_election.cc:304] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 
I20260709 13:22:18.017520 24578 raft_consensus.cc:2804] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:18.018169 24578 raft_consensus.cc:697] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 LEADER]: Becoming Leader. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Running, Role: LEADER
I20260709 13:22:18.019095 24578 consensus_queue.cc:237] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.020195 24427 raft_consensus.cc:2468] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 1.
I20260709 13:22:18.013901 24541 raft_consensus.cc:385] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.021052 24541 raft_consensus.cc:740] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Initialized, Role: FOLLOWER
I20260709 13:22:18.023104 24541 consensus_queue.cc:260] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.027354 24541 ts_tablet_manager.cc:1434] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 13:22:18.028861 24352 raft_consensus.cc:2468] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 1.
I20260709 13:22:18.029112 24545 tablet_bootstrap.cc:492] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: No bootstrap required, opened a new log
I20260709 13:22:18.029758 24545 ts_tablet_manager.cc:1403] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent bootstrapping tablet: real 0.111s	user 0.040s	sys 0.019s
I20260709 13:22:18.032248 24542 tablet_bootstrap.cc:492] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168: No bootstrap required, opened a new log
I20260709 13:22:18.032805 24542 ts_tablet_manager.cc:1403] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168: Time spent bootstrapping tablet: real 0.127s	user 0.011s	sys 0.013s
I20260709 13:22:18.035616 24542 raft_consensus.cc:359] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.036340 24542 raft_consensus.cc:385] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.035456 24248 catalog_manager.cc:5697] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:18.041006 24542 raft_consensus.cc:740] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Initialized, Role: FOLLOWER
I20260709 13:22:18.036986 24465 leader_election.cc:304] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14
I20260709 13:22:18.041889 24542 consensus_queue.cc:260] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.043876 24584 raft_consensus.cc:2804] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:18.044463 24584 raft_consensus.cc:697] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 LEADER]: Becoming Leader. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Running, Role: LEADER
I20260709 13:22:18.045380 24584 consensus_queue.cc:237] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.049062 24545 raft_consensus.cc:359] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.049942 24545 raft_consensus.cc:385] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.050251 24545 raft_consensus.cc:740] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Initialized, Role: FOLLOWER
I20260709 13:22:18.051007 24542 ts_tablet_manager.cc:1434] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.004s
I20260709 13:22:18.051155 24545 consensus_queue.cc:260] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.052084 24542 tablet_bootstrap.cc:492] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168: Bootstrap starting.
I20260709 13:22:18.052428 24249 catalog_manager.cc:5697] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:18.056813 24545 ts_tablet_manager.cc:1434] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:22:18.057952 24545 tablet_bootstrap.cc:492] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap starting.
I20260709 13:22:18.066766 24545 tablet_bootstrap.cc:654] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:18.065162 24248 catalog_manager.cc:5697] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:18.061851 24542 tablet_bootstrap.cc:654] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:18.070919 24247 catalog_manager.cc:5697] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:18.082705 24542 tablet_bootstrap.cc:492] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168: No bootstrap required, opened a new log
I20260709 13:22:18.083252 24542 ts_tablet_manager.cc:1403] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.006s
I20260709 13:22:18.085896 24542 raft_consensus.cc:359] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.086596 24542 raft_consensus.cc:385] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.086858 24542 raft_consensus.cc:740] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Initialized, Role: FOLLOWER
I20260709 13:22:18.087790 24545 tablet_bootstrap.cc:492] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: No bootstrap required, opened a new log
I20260709 13:22:18.087487 24542 consensus_queue.cc:260] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.088327 24545 ts_tablet_manager.cc:1403] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.000s
I20260709 13:22:18.089533 24542 ts_tablet_manager.cc:1434] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 13:22:18.090559 24542 tablet_bootstrap.cc:492] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168: Bootstrap starting.
I20260709 13:22:18.090950 24545 raft_consensus.cc:359] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.091653 24545 raft_consensus.cc:385] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.091910 24545 raft_consensus.cc:740] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Initialized, Role: FOLLOWER
I20260709 13:22:18.092660 24545 consensus_queue.cc:260] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.095009 24545 ts_tablet_manager.cc:1434] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:22:18.095956 24545 tablet_bootstrap.cc:492] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap starting.
I20260709 13:22:18.096575 24542 tablet_bootstrap.cc:654] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:18.101503 24545 tablet_bootstrap.cc:654] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:18.108021 24542 tablet_bootstrap.cc:492] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168: No bootstrap required, opened a new log
I20260709 13:22:18.108533 24542 ts_tablet_manager.cc:1403] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168: Time spent bootstrapping tablet: real 0.018s	user 0.006s	sys 0.009s
I20260709 13:22:18.111135 24545 tablet_bootstrap.cc:492] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: No bootstrap required, opened a new log
I20260709 13:22:18.110960 24542 raft_consensus.cc:359] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.111608 24542 raft_consensus.cc:385] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.111670 24545 ts_tablet_manager.cc:1403] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.003s
I20260709 13:22:18.111908 24542 raft_consensus.cc:740] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Initialized, Role: FOLLOWER
I20260709 13:22:18.112624 24542 consensus_queue.cc:260] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.114562 24542 ts_tablet_manager.cc:1434] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.002s
I20260709 13:22:18.114331 24545 raft_consensus.cc:359] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.115029 24545 raft_consensus.cc:385] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.115293 24545 raft_consensus.cc:740] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Initialized, Role: FOLLOWER
I20260709 13:22:18.115909 24545 consensus_queue.cc:260] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.117120 24550 raft_consensus.cc:493] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.117657 24545 ts_tablet_manager.cc:1434] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:22:18.117599 24550 raft_consensus.cc:515] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.118734 24545 tablet_bootstrap.cc:492] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap starting.
I20260709 13:22:18.119836 24550 leader_election.cc:290] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:18.120564 24427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5eeb2f0a1e54fdc8c94f815d9819277" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" is_pre_election: true
I20260709 13:22:18.121245 24427 raft_consensus.cc:2468] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in term 0.
I20260709 13:22:18.121852 24501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5eeb2f0a1e54fdc8c94f815d9819277" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" is_pre_election: true
I20260709 13:22:18.122570 24501 raft_consensus.cc:2468] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in term 0.
I20260709 13:22:18.122584 24316 leader_election.cc:304] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 3c997d7a5c7f499abf3cb90c38f0ebe6; no voters: 
I20260709 13:22:18.124691 24545 tablet_bootstrap.cc:654] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:18.126968 24550 raft_consensus.cc:2804] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:18.127431 24550 raft_consensus.cc:493] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:18.127775 24550 raft_consensus.cc:3060] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.130908 24545 tablet_bootstrap.cc:492] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: No bootstrap required, opened a new log
I20260709 13:22:18.131345 24545 ts_tablet_manager.cc:1403] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 13:22:18.133364 24545 raft_consensus.cc:359] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.133821 24545 raft_consensus.cc:385] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.134028 24545 raft_consensus.cc:740] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Initialized, Role: FOLLOWER
I20260709 13:22:18.134529 24545 consensus_queue.cc:260] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.135103 24550 raft_consensus.cc:515] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.135957 24545 ts_tablet_manager.cc:1434] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:22:18.137840 24427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5eeb2f0a1e54fdc8c94f815d9819277" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
I20260709 13:22:18.138299 24501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5eeb2f0a1e54fdc8c94f815d9819277" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
I20260709 13:22:18.138613 24427 raft_consensus.cc:3060] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.139088 24501 raft_consensus.cc:3060] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.138093 24550 leader_election.cc:290] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 election: Requested vote from peers 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:18.144796 24501 raft_consensus.cc:2468] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in term 1.
I20260709 13:22:18.144915 24427 raft_consensus.cc:2468] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in term 1.
I20260709 13:22:18.145886 24317 leader_election.cc:304] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 
I20260709 13:22:18.146724 24550 raft_consensus.cc:2804] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:18.147123 24550 raft_consensus.cc:697] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 LEADER]: Becoming Leader. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Running, Role: LEADER
I20260709 13:22:18.147908 24550 consensus_queue.cc:237] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.156073 24247 catalog_manager.cc:5697] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:18.286728 24501 raft_consensus.cc:3060] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.293267 24550 consensus_queue.cc:1048] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.294281 24574 raft_consensus.cc:493] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.294860 24574 raft_consensus.cc:515] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.296801 24574 leader_election.cc:290] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:18.297626 24427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cefb0660f2f84bdfb7e9b47cb002b525" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" is_pre_election: true
I20260709 13:22:18.298089 24500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cefb0660f2f84bdfb7e9b47cb002b525" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" is_pre_election: true
I20260709 13:22:18.298960 24500 raft_consensus.cc:2393] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in current term 1: Already voted for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in this term.
I20260709 13:22:18.299976 24574 raft_consensus.cc:493] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.300082 24317 leader_election.cc:304] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14; no voters: 3c997d7a5c7f499abf3cb90c38f0ebe6, 97364f1f8ccd429e9c9d22c09ecfd168
I20260709 13:22:18.301021 24550 raft_consensus.cc:3060] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.301115 24574 raft_consensus.cc:515] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.304246 24427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe46fad433bd4a1d8070be4a4f4fdcf6" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" is_pre_election: true
I20260709 13:22:18.304937 24427 raft_consensus.cc:2468] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in term 0.
I20260709 13:22:18.306171 24574 leader_election.cc:290] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:18.306442 24316 leader_election.cc:304] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 3c997d7a5c7f499abf3cb90c38f0ebe6; no voters: 
I20260709 13:22:18.307435 24576 raft_consensus.cc:2804] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:18.307837 24576 raft_consensus.cc:493] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:18.308163 24576 raft_consensus.cc:3060] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.310863 24550 raft_consensus.cc:2749] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:18.315770 24501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe46fad433bd4a1d8070be4a4f4fdcf6" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" is_pre_election: true
I20260709 13:22:18.315907 24576 raft_consensus.cc:515] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.316635 24501 raft_consensus.cc:2468] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in term 0.
I20260709 13:22:18.318671 24576 leader_election.cc:290] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 election: Requested vote from peers 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:18.318849 24427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe46fad433bd4a1d8070be4a4f4fdcf6" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
I20260709 13:22:18.320124 24427 raft_consensus.cc:3060] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.320216 24501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe46fad433bd4a1d8070be4a4f4fdcf6" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
I20260709 13:22:18.320911 24501 raft_consensus.cc:3060] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.323468 24550 consensus_queue.cc:1048] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:18.328691 24501 raft_consensus.cc:2468] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in term 1.
I20260709 13:22:18.328998 24427 raft_consensus.cc:2468] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in term 1.
I20260709 13:22:18.330166 24317 leader_election.cc:304] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 
I20260709 13:22:18.331123 24550 raft_consensus.cc:2804] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:18.331620 24550 raft_consensus.cc:697] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 LEADER]: Becoming Leader. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Running, Role: LEADER
I20260709 13:22:18.332466 24550 consensus_queue.cc:237] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.337463 24574 consensus_queue.cc:1048] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.344648 24248 catalog_manager.cc:5697] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:18.349363 24584 consensus_queue.cc:1048] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:18.356640 24550 consensus_queue.cc:1048] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.363863 24583 consensus_queue.cc:1048] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.390095 24550 raft_consensus.cc:493] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.390556 24550 raft_consensus.cc:515] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.392163 24550 leader_election.cc:290] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:18.392940 24427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee0eadd86f434502b7ccabf5024d1253" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" is_pre_election: true
I20260709 13:22:18.393229 24501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee0eadd86f434502b7ccabf5024d1253" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" is_pre_election: true
I20260709 13:22:18.393821 24427 raft_consensus.cc:2393] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in current term 1: Already voted for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in this term.
I20260709 13:22:18.394726 24316 leader_election.cc:304] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14; no voters: 3c997d7a5c7f499abf3cb90c38f0ebe6, 97364f1f8ccd429e9c9d22c09ecfd168
I20260709 13:22:18.395406 24550 raft_consensus.cc:3060] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.400449 24550 raft_consensus.cc:2749] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:18.408213 24550 raft_consensus.cc:493] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.408651 24550 raft_consensus.cc:515] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.410830 24550 leader_election.cc:290] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:18.411147 24427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248862fab3014daba862723841b4730a" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" is_pre_election: true
I20260709 13:22:18.411487 24501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248862fab3014daba862723841b4730a" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" is_pre_election: true
I20260709 13:22:18.411916 24427 raft_consensus.cc:2468] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in term 0.
I20260709 13:22:18.412130 24501 raft_consensus.cc:2468] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in term 0.
I20260709 13:22:18.413143 24316 leader_election.cc:304] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 3c997d7a5c7f499abf3cb90c38f0ebe6; no voters: 
I20260709 13:22:18.414122 24550 raft_consensus.cc:2804] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:18.414510 24550 raft_consensus.cc:493] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:18.414762 24550 raft_consensus.cc:3060] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.415678 24578 consensus_queue.cc:1048] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.420776 24550 raft_consensus.cc:515] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.423305 24550 leader_election.cc:290] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 election: Requested vote from peers 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:18.424010 24427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248862fab3014daba862723841b4730a" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
I20260709 13:22:18.424506 24500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248862fab3014daba862723841b4730a" candidate_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
I20260709 13:22:18.424757 24427 raft_consensus.cc:3060] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.425189 24500 raft_consensus.cc:3060] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.432987 24580 consensus_queue.cc:1048] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:18.433259 24427 raft_consensus.cc:2468] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in term 1.
I20260709 13:22:18.434619 24316 leader_election.cc:304] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 3c997d7a5c7f499abf3cb90c38f0ebe6; no voters: 
I20260709 13:22:18.435631 24500 raft_consensus.cc:2468] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bef6e7eea474e7e8f3ec0e55f5b9d14 in term 1.
I20260709 13:22:18.436031 24550 raft_consensus.cc:2804] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:18.436583 24550 raft_consensus.cc:697] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 LEADER]: Becoming Leader. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Running, Role: LEADER
I20260709 13:22:18.437402 24550 consensus_queue.cc:237] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:18.447973 24249 catalog_manager.cc:5697] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:18.484825 24580 consensus_queue.cc:1048] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.496237 24544 consensus_queue.cc:1048] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.515015 24590 consensus_queue.cc:1048] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.527894 24590 consensus_queue.cc:1048] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:22:18.629338 24298 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:18.640977 24550 consensus_queue.cc:1048] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.651232 24574 consensus_queue.cc:1048] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.720909 24574 consensus_queue.cc:1048] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.731793 24576 consensus_queue.cc:1048] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.829257 24576 consensus_queue.cc:1048] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.840849 24574 consensus_queue.cc:1048] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:22:19.633653 24298 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:19.669643 24590 raft_consensus.cc:493] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:19.670243 24590 raft_consensus.cc:515] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:19.671939 24590 leader_election.cc:290] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577)
I20260709 13:22:19.672842 24352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "688b155b91274e1c9102292eb67cc0c6" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" is_pre_election: true
I20260709 13:22:19.673135 24427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "688b155b91274e1c9102292eb67cc0c6" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" is_pre_election: true
I20260709 13:22:19.673508 24352 raft_consensus.cc:2468] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 1.
I20260709 13:22:19.673825 24427 raft_consensus.cc:2468] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 1.
I20260709 13:22:19.674819 24464 leader_election.cc:304] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 
I20260709 13:22:19.675493 24590 raft_consensus.cc:2804] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:19.675778 24590 raft_consensus.cc:493] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:19.676098 24590 raft_consensus.cc:3060] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:19.681113 24590 raft_consensus.cc:515] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:19.682854 24590 leader_election.cc:290] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 election: Requested vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577)
I20260709 13:22:19.683600 24352 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "688b155b91274e1c9102292eb67cc0c6" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14"
I20260709 13:22:19.683856 24427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "688b155b91274e1c9102292eb67cc0c6" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
I20260709 13:22:19.684137 24352 raft_consensus.cc:3060] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:19.684410 24427 raft_consensus.cc:3060] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:19.689484 24427 raft_consensus.cc:2468] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 2.
I20260709 13:22:19.690526 24465 leader_election.cc:304] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 
I20260709 13:22:19.690652 24352 raft_consensus.cc:2468] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 2.
I20260709 13:22:19.691314 24590 raft_consensus.cc:2804] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:19.692243 24590 raft_consensus.cc:697] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 LEADER]: Becoming Leader. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Running, Role: LEADER
I20260709 13:22:19.693027 24590 consensus_queue.cc:237] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:19.702272 24248 catalog_manager.cc:5697] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 reported cstate change: term changed from 0 to 2, leader changed from <none> to 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:19.858012 14429 ts_manager.cc:295] Set tserver state for 1bef6e7eea474e7e8f3ec0e55f5b9d14 to MAINTENANCE_MODE
I20260709 13:22:19.858700 14429 tablet_server.cc:179] TabletServer@127.14.23.65:0 shutting down...
I20260709 13:22:19.884397 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:19.885111 14429 tablet_replica.cc:333] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: stopping tablet replica
I20260709 13:22:19.885761 14429 raft_consensus.cc:2243] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:19.886250 14429 raft_consensus.cc:2272] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:19.888160 14429 tablet_replica.cc:333] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: stopping tablet replica
I20260709 13:22:19.888689 14429 raft_consensus.cc:2243] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:19.889441 14429 raft_consensus.cc:2272] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:19.891335 14429 tablet_replica.cc:333] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: stopping tablet replica
I20260709 13:22:19.891829 14429 raft_consensus.cc:2243] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:19.892206 14429 raft_consensus.cc:2272] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:19.893994 14429 tablet_replica.cc:333] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14: stopping tablet replica
I20260709 13:22:19.894488 14429 raft_consensus.cc:2243] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:19.895205 14429 raft_consensus.cc:2272] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:19.897023 14429 tablet_replica.cc:333] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: stopping tablet replica
I20260709 13:22:19.897517 14429 raft_consensus.cc:2243] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:19.897907 14429 raft_consensus.cc:2272] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:19.899580 14429 tablet_replica.cc:333] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: stopping tablet replica
I20260709 13:22:19.900069 14429 raft_consensus.cc:2243] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:19.900442 14429 raft_consensus.cc:2272] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:19.902194 14429 tablet_replica.cc:333] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: stopping tablet replica
I20260709 13:22:19.902675 14429 raft_consensus.cc:2243] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:19.903036 14429 raft_consensus.cc:2272] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:19.904718 14429 tablet_replica.cc:333] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: stopping tablet replica
I20260709 13:22:19.905196 14429 raft_consensus.cc:2243] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:19.905942 14429 raft_consensus.cc:2272] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:19.907634 14429 tablet_replica.cc:333] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14: stopping tablet replica
I20260709 13:22:19.908094 14429 raft_consensus.cc:2243] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:19.908823 14429 raft_consensus.cc:2272] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:19.910714 14429 tablet_replica.cc:333] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: stopping tablet replica
I20260709 13:22:19.911178 14429 raft_consensus.cc:2243] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:19.911943 14429 raft_consensus.cc:2272] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:19.941093 14429 tablet_server.cc:196] TabletServer@127.14.23.65:0 shutdown complete.
W20260709 13:22:19.949174 24464 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111) [suppressed 121 similar messages]
W20260709 13:22:19.953292 24464 consensus_peers.cc:597] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:19.988364 24390 consensus_peers.cc:597] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:20.171931 24464 consensus_peers.cc:597] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:20.278666 24427 raft_consensus.cc:1275] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Refusing update from remote peer 97364f1f8ccd429e9c9d22c09ecfd168: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:22:20.279999 24590 consensus_queue.cc:1048] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:22:20.293773 24464 consensus_peers.cc:597] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:20.423293 24390 consensus_peers.cc:597] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:20.634631 24298 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:20.958731 24645 raft_consensus.cc:493] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1bef6e7eea474e7e8f3ec0e55f5b9d14)
I20260709 13:22:20.959262 24645 raft_consensus.cc:515] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:20.960906 24645 leader_election.cc:290] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:20.962136 24501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248862fab3014daba862723841b4730a" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" is_pre_election: true
W20260709 13:22:20.964604 24390 leader_election.cc:336] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111)
I20260709 13:22:20.964982 24390 leader_election.cc:304] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6; no voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 97364f1f8ccd429e9c9d22c09ecfd168
I20260709 13:22:20.965570 24645 raft_consensus.cc:2749] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:22:21.065507 24590 raft_consensus.cc:493] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1bef6e7eea474e7e8f3ec0e55f5b9d14)
I20260709 13:22:21.066032 24590 raft_consensus.cc:515] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:21.067685 24590 leader_election.cc:290] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577)
I20260709 13:22:21.068846 24426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5eeb2f0a1e54fdc8c94f815d9819277" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" is_pre_election: true
W20260709 13:22:21.071244 24464 leader_election.cc:336] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111)
I20260709 13:22:21.071656 24464 leader_election.cc:304] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 3c997d7a5c7f499abf3cb90c38f0ebe6
I20260709 13:22:21.072229 24590 raft_consensus.cc:2749] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:22:21.146270 24647 raft_consensus.cc:493] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1bef6e7eea474e7e8f3ec0e55f5b9d14)
I20260709 13:22:21.146816 24647 raft_consensus.cc:515] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:21.148480 24647 leader_election.cc:290] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:21.149683 24501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe46fad433bd4a1d8070be4a4f4fdcf6" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" is_pre_election: true
W20260709 13:22:21.152159 24390 leader_election.cc:336] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111)
I20260709 13:22:21.152536 24390 leader_election.cc:304] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6; no voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 97364f1f8ccd429e9c9d22c09ecfd168
I20260709 13:22:21.153137 24647 raft_consensus.cc:2749] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:22:21.182808 24590 raft_consensus.cc:493] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1bef6e7eea474e7e8f3ec0e55f5b9d14)
I20260709 13:22:21.183805 24590 raft_consensus.cc:515] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:21.186339 24590 leader_election.cc:290] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577)
I20260709 13:22:21.187670 24426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe46fad433bd4a1d8070be4a4f4fdcf6" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" is_pre_election: true
I20260709 13:22:21.188459 24426 raft_consensus.cc:2468] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 1.
I20260709 13:22:21.189611 24465 leader_election.cc:304] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 
I20260709 13:22:21.190548 24590 raft_consensus.cc:2804] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:21.190948 24590 raft_consensus.cc:493] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1bef6e7eea474e7e8f3ec0e55f5b9d14)
I20260709 13:22:21.191394 24590 raft_consensus.cc:3060] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:22:21.192870 24464 leader_election.cc:336] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111)
I20260709 13:22:21.201009 24640 raft_consensus.cc:493] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1bef6e7eea474e7e8f3ec0e55f5b9d14)
I20260709 13:22:21.200830 24590 raft_consensus.cc:515] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:21.201735 24640 raft_consensus.cc:515] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:21.203552 24590 leader_election.cc:290] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 election: Requested vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577)
I20260709 13:22:21.204645 24640 leader_election.cc:290] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577)
I20260709 13:22:21.205958 24426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe46fad433bd4a1d8070be4a4f4fdcf6" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
I20260709 13:22:21.206521 24426 raft_consensus.cc:3060] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:22:21.208804 24464 leader_election.cc:336] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111)
W20260709 13:22:21.209388 24464 leader_election.cc:336] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111)
I20260709 13:22:21.210132 24427 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c88fba243c24e7db4b3e7530fc9e881" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" is_pre_election: true
I20260709 13:22:21.211529 24647 raft_consensus.cc:493] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1bef6e7eea474e7e8f3ec0e55f5b9d14)
I20260709 13:22:21.211431 24465 leader_election.cc:304] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 3c997d7a5c7f499abf3cb90c38f0ebe6
I20260709 13:22:21.212016 24647 raft_consensus.cc:515] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:21.212698 24640 raft_consensus.cc:2749] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:22:21.213004 24426 raft_consensus.cc:2468] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 2.
I20260709 13:22:21.213903 24647 leader_election.cc:290] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:21.214077 24465 leader_election.cc:304] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14
I20260709 13:22:21.215566 24501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5eeb2f0a1e54fdc8c94f815d9819277" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" is_pre_election: true
I20260709 13:22:21.216218 24501 raft_consensus.cc:2468] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 1.
I20260709 13:22:21.217229 24392 leader_election.cc:304] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 
I20260709 13:22:21.250082 24647 raft_consensus.cc:2804] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:21.250559 24647 raft_consensus.cc:493] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1bef6e7eea474e7e8f3ec0e55f5b9d14)
I20260709 13:22:21.250962 24647 raft_consensus.cc:3060] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:21.215727 24640 raft_consensus.cc:2804] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:21.359611 24640 raft_consensus.cc:697] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 LEADER]: Becoming Leader. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Running, Role: LEADER
I20260709 13:22:21.360333 24640 consensus_queue.cc:237] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:21.340369 24645 raft_consensus.cc:493] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1bef6e7eea474e7e8f3ec0e55f5b9d14)
W20260709 13:22:21.351915 24390 leader_election.cc:336] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111)
I20260709 13:22:21.353101 24648 raft_consensus.cc:493] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1bef6e7eea474e7e8f3ec0e55f5b9d14)
I20260709 13:22:21.362994 24645 raft_consensus.cc:515] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:21.363260 24648 raft_consensus.cc:515] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:21.364847 24646 raft_consensus.cc:493] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1bef6e7eea474e7e8f3ec0e55f5b9d14)
I20260709 13:22:21.358167 24647 raft_consensus.cc:515] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:21.365487 24646 raft_consensus.cc:515] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } }
I20260709 13:22:21.366513 24501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c88fba243c24e7db4b3e7530fc9e881" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" is_pre_election: true
I20260709 13:22:21.365356 24648 leader_election.cc:290] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577)
I20260709 13:22:21.367102 24647 leader_election.cc:290] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 election: Requested vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:21.367925 24646 leader_election.cc:290] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867), 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105)
I20260709 13:22:21.368343 24648 raft_consensus.cc:493] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1bef6e7eea474e7e8f3ec0e55f5b9d14)
I20260709 13:22:21.368956 24500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5eeb2f0a1e54fdc8c94f815d9819277" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
I20260709 13:22:21.368916 24648 raft_consensus.cc:515] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } }
I20260709 13:22:21.367209 24501 raft_consensus.cc:2468] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 1.
I20260709 13:22:21.369753 24500 raft_consensus.cc:3060] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:21.370978 24392 leader_election.cc:304] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 
I20260709 13:22:21.371116 24648 leader_election.cc:290] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577), 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105)
I20260709 13:22:21.369401 24499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4065825e694d4fa259bbdb5538289f" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" is_pre_election: true
W20260709 13:22:21.371098 24464 leader_election.cc:336] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111)
I20260709 13:22:21.372581 24499 raft_consensus.cc:2468] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 1.
I20260709 13:22:21.374130 24392 leader_election.cc:304] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 
I20260709 13:22:21.374910 24647 raft_consensus.cc:2804] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:21.375289 24647 raft_consensus.cc:493] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1bef6e7eea474e7e8f3ec0e55f5b9d14)
I20260709 13:22:21.375203 24426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248862fab3014daba862723841b4730a" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" is_pre_election: true
I20260709 13:22:21.375885 24647 raft_consensus.cc:3060] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:21.376094 24426 raft_consensus.cc:2468] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 1.
I20260709 13:22:21.377554 24426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4065825e694d4fa259bbdb5538289f" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" is_pre_election: true
I20260709 13:22:21.377668 24646 raft_consensus.cc:2804] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:21.378199 24646 raft_consensus.cc:493] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1bef6e7eea474e7e8f3ec0e55f5b9d14)
I20260709 13:22:21.378651 24646 raft_consensus.cc:3060] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:21.378741 24465 leader_election.cc:304] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14
I20260709 13:22:21.379539 24645 leader_election.cc:290] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:21.380431 24648 raft_consensus.cc:2804] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:21.380849 24648 raft_consensus.cc:493] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1bef6e7eea474e7e8f3ec0e55f5b9d14)
W20260709 13:22:21.382522 24390 leader_election.cc:336] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111)
I20260709 13:22:21.388626 24648 raft_consensus.cc:3060] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:21.388114 24247 catalog_manager.cc:5697] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 reported cstate change: term changed from 1 to 2, leader changed from 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65) to 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: HEALTHY } } }
W20260709 13:22:21.389144 24390 leader_election.cc:336] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111)
W20260709 13:22:21.389699 24390 leader_election.cc:336] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111)
W20260709 13:22:21.377873 24464 leader_election.cc:336] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111)
I20260709 13:22:21.383612 24647 raft_consensus.cc:515] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:21.393296 24647 leader_election.cc:290] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 election: Requested vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:21.387966 24646 raft_consensus.cc:515] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } }
I20260709 13:22:21.379196 24500 raft_consensus.cc:2468] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 2.
I20260709 13:22:21.395578 24499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c88fba243c24e7db4b3e7530fc9e881" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
I20260709 13:22:21.396265 24499 raft_consensus.cc:3060] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:21.396051 24648 raft_consensus.cc:515] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:21.397568 24646 leader_election.cc:290] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 election: Requested vote from peers 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867), 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105)
I20260709 13:22:21.398252 24426 raft_consensus.cc:2393] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97364f1f8ccd429e9c9d22c09ecfd168 in current term 2: Already voted for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in this term.
I20260709 13:22:21.399128 24648 leader_election.cc:290] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 election: Requested vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577)
W20260709 13:22:21.400120 24390 leader_election.cc:336] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111)
I20260709 13:22:21.401575 24392 leader_election.cc:304] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14
W20260709 13:22:21.401858 24464 leader_election.cc:336] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111)
I20260709 13:22:21.402101 24500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4065825e694d4fa259bbdb5538289f" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
I20260709 13:22:21.402734 24500 raft_consensus.cc:3060] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:22:21.403268 24390 leader_election.cc:336] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111)
I20260709 13:22:21.403949 24465 leader_election.cc:304] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 3c997d7a5c7f499abf3cb90c38f0ebe6
I20260709 13:22:21.404148 24426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248862fab3014daba862723841b4730a" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
I20260709 13:22:21.404723 24426 raft_consensus.cc:3060] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:21.409997 24426 raft_consensus.cc:2468] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 2.
I20260709 13:22:21.410321 24500 raft_consensus.cc:2468] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 2.
I20260709 13:22:21.410964 24640 raft_consensus.cc:2749] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:22:21.411247 24465 leader_election.cc:304] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14
I20260709 13:22:21.412129 24648 raft_consensus.cc:2804] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:21.412740 24499 raft_consensus.cc:2468] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 2.
I20260709 13:22:21.412873 24648 raft_consensus.cc:697] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 LEADER]: Becoming Leader. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Running, Role: LEADER
I20260709 13:22:21.413758 24392 leader_election.cc:304] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14
I20260709 13:22:21.410198 24646 raft_consensus.cc:2804] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:21.413991 24648 consensus_queue.cc:237] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:21.414935 24646 raft_consensus.cc:697] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 LEADER]: Becoming Leader. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Running, Role: LEADER
I20260709 13:22:21.415270 24392 leader_election.cc:304] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c997d7a5c7f499abf3cb90c38f0ebe6, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14
I20260709 13:22:21.415786 24646 consensus_queue.cc:237] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:21.416661 24647 raft_consensus.cc:2804] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:21.418079 24645 raft_consensus.cc:2804] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:21.418556 24645 raft_consensus.cc:697] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 LEADER]: Becoming Leader. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Running, Role: LEADER
I20260709 13:22:21.419412 24645 consensus_queue.cc:237] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:21.438542 24647 raft_consensus.cc:697] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 LEADER]: Becoming Leader. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Running, Role: LEADER
I20260709 13:22:21.439396 24647 consensus_queue.cc:237] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } }
I20260709 13:22:21.443450 24248 catalog_manager.cc:5697] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 reported cstate change: term changed from 1 to 2, leader changed from 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65) to 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:21.452919 24247 catalog_manager.cc:5697] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 reported cstate change: term changed from 1 to 2, leader changed from 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65) to 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:21.458129 24248 catalog_manager.cc:5697] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 reported cstate change: term changed from 1 to 2, leader changed from 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65) to 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:21.459434 24248 catalog_manager.cc:5697] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 reported cstate change: term changed from 1 to 2, leader changed from 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65) to 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: UNKNOWN } } }
W20260709 13:22:21.635704 24298 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:21.755894 24426 raft_consensus.cc:1275] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Refusing update from remote peer 97364f1f8ccd429e9c9d22c09ecfd168: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:21.757424 24640 consensus_queue.cc:1048] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:22:21.770910 24464 consensus_peers.cc:597] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:21.848749 24390 consensus_peers.cc:597] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:21.886320 24500 raft_consensus.cc:1275] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Refusing update from remote peer 3c997d7a5c7f499abf3cb90c38f0ebe6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:21.887426 24652 consensus_queue.cc:1048] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:22:21.892501 24390 consensus_peers.cc:597] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:21.900453 24390 consensus_peers.cc:597] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:21.908674 24426 raft_consensus.cc:1275] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Refusing update from remote peer 97364f1f8ccd429e9c9d22c09ecfd168: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:21.910023 24648 consensus_queue.cc:1048] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:22:21.922323 24464 consensus_peers.cc:597] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:21.991818 24500 raft_consensus.cc:1275] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Refusing update from remote peer 3c997d7a5c7f499abf3cb90c38f0ebe6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:21.993144 24652 consensus_queue.cc:1048] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:22.036258 24500 raft_consensus.cc:1275] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Refusing update from remote peer 3c997d7a5c7f499abf3cb90c38f0ebe6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:22.037503 24647 consensus_queue.cc:1048] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:22:22.258795 24281 debug-util.cc:398] Leaking SignalData structure 0x7b080032a080 after lost signal to thread 14432
W20260709 13:22:22.259775 24281 debug-util.cc:398] Leaking SignalData structure 0x7b08001d5da0 after lost signal to thread 24284
W20260709 13:22:22.260934 24281 debug-util.cc:398] Leaking SignalData structure 0x7b08003008a0 after lost signal to thread 24451
W20260709 13:22:22.262127 24281 debug-util.cc:398] Leaking SignalData structure 0x7b0800252700 after lost signal to thread 24525
W20260709 13:22:22.292972 24390 consensus_peers.cc:597] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:22.502251 24464 consensus_peers.cc:597] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:22.514118 24464 consensus_peers.cc:597] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:22.636819 24298 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:22:22.711258 24464 consensus_peers.cc:597] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:22.945591 24390 consensus_peers.cc:597] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:23.638096 24298 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:22:24.267172 24390 consensus_peers.cc:597] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:24.371438 24464 consensus_peers.cc:597] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:24.380673 24464 consensus_peers.cc:597] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:24.507584 24390 consensus_peers.cc:597] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:24.563935 24390 consensus_peers.cc:597] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:24.639191 24298 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:22:24.774797 24390 consensus_peers.cc:597] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:24.874624 24464 consensus_peers.cc:597] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:24.954106 24464 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111) [suppressed 98 similar messages]
I20260709 13:22:25.020773 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:25.032334 24670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:25.037405 24671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:25.041062 24673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:25.043246 14429 server_base.cc:1061] running on GCE node
I20260709 13:22:25.044847 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:25.045097 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:25.045408 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603345045376 us; error 0 us; skew 500 ppm
I20260709 13:22:25.046378 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:25.050344 14429 webserver.cc:533] Webserver started at http://127.14.23.65:38483/ using document root <none> and password file <none>
I20260709 13:22:25.050964 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:25.051151 14429 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:22:25.057137 14429 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.004s	sys 0.002s
W20260709 13:22:25.058980 24464 consensus_peers.cc:597] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:25.063498 24678 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:25.064739 14429 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 13:22:25.065318 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/ts-0-root
uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14"
format_stamp: "Formatted at 2026-07-09 13:22:16 on dist-test-slave-twc1"
I20260709 13:22:25.066064 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603055563165-14429-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:25.088433 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:25.090368 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:25.092792 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:25.098343 24685 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 13:22:25.161645 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:22:25.162004 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.065s	user 0.001s	sys 0.003s
I20260709 13:22:25.162304 14429 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:22:25.168512 24685 tablet_bootstrap.cc:492] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap starting.
I20260709 13:22:25.189373 24685 tablet_bootstrap.cc:492] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:22:25.190928 24464 consensus_peers.cc:597] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:25.191457 24685 tablet_bootstrap.cc:492] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap complete.
I20260709 13:22:25.192278 24685 ts_tablet_manager.cc:1403] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent bootstrapping tablet: real 0.024s	user 0.020s	sys 0.000s
W20260709 13:22:25.192778 24390 consensus_peers.cc:597] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:25.195391 24685 raft_consensus.cc:359] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.196635 24685 raft_consensus.cc:740] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Initialized, Role: FOLLOWER
I20260709 13:22:25.197497 24685 consensus_queue.cc:260] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.200672 24685 ts_tablet_manager.cc:1434] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:22:25.201546 24685 tablet_bootstrap.cc:492] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap starting.
I20260709 13:22:25.219305 24685 tablet_bootstrap.cc:492] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:25.220381 24685 tablet_bootstrap.cc:492] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap complete.
I20260709 13:22:25.221247 24685 ts_tablet_manager.cc:1403] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent bootstrapping tablet: real 0.020s	user 0.007s	sys 0.012s
I20260709 13:22:25.224218 24685 raft_consensus.cc:359] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.225076 24685 raft_consensus.cc:740] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Initialized, Role: FOLLOWER
I20260709 13:22:25.226204 24685 consensus_queue.cc:260] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.228856 24685 ts_tablet_manager.cc:1434] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.005s
I20260709 13:22:25.229803 24685 tablet_bootstrap.cc:492] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap starting.
I20260709 13:22:25.238687 14429 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:22:25.239116 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.077s	user 0.061s	sys 0.007s
I20260709 13:22:25.248648 24685 tablet_bootstrap.cc:492] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:25.249742 24685 tablet_bootstrap.cc:492] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap complete.
I20260709 13:22:25.250620 24685 ts_tablet_manager.cc:1403] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.003s
I20260709 13:22:25.254137 24685 raft_consensus.cc:359] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.255206 24685 raft_consensus.cc:740] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Initialized, Role: FOLLOWER
I20260709 13:22:25.256305 24685 consensus_queue.cc:260] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.258672 24685 ts_tablet_manager.cc:1434] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:22:25.259698 24685 tablet_bootstrap.cc:492] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap starting.
I20260709 13:22:25.278936 24685 tablet_bootstrap.cc:492] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:25.280066 24685 tablet_bootstrap.cc:492] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap complete.
I20260709 13:22:25.280920 24685 ts_tablet_manager.cc:1403] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.007s
I20260709 13:22:25.283916 24685 raft_consensus.cc:359] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.284761 24685 raft_consensus.cc:740] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Initialized, Role: FOLLOWER
I20260709 13:22:25.285722 24685 consensus_queue.cc:260] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.287860 24685 ts_tablet_manager.cc:1434] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:22:25.288765 24685 tablet_bootstrap.cc:492] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap starting.
I20260709 13:22:25.307276 24685 tablet_bootstrap.cc:492] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:25.309188 24685 tablet_bootstrap.cc:492] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap complete.
I20260709 13:22:25.309994 24685 ts_tablet_manager.cc:1403] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.007s
I20260709 13:22:25.312870 24685 raft_consensus.cc:359] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.313608 24685 raft_consensus.cc:740] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Initialized, Role: FOLLOWER
I20260709 13:22:25.314549 24685 consensus_queue.cc:260] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.317015 24685 ts_tablet_manager.cc:1434] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:22:25.318048 24685 tablet_bootstrap.cc:492] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap starting.
I20260709 13:22:25.325847 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.65:40105
I20260709 13:22:25.330817 24747 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.65:40105 every 8 connection(s)
I20260709 13:22:25.337823 24685 tablet_bootstrap.cc:492] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:25.338910 24685 tablet_bootstrap.cc:492] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap complete.
I20260709 13:22:25.339646 24685 ts_tablet_manager.cc:1403] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.009s
I20260709 13:22:25.342579 24685 raft_consensus.cc:359] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.343699 24685 raft_consensus.cc:740] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Initialized, Role: FOLLOWER
I20260709 13:22:25.344647 24685 consensus_queue.cc:260] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.347173 24685 ts_tablet_manager.cc:1434] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 13:22:25.347811 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:25.348284 24685 tablet_bootstrap.cc:492] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap starting.
I20260709 13:22:25.384895 24748 heartbeater.cc:344] Connected to a master server at 127.14.23.126:39899
I20260709 13:22:25.385425 24748 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:25.386199 24685 tablet_bootstrap.cc:492] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:25.386446 24748 heartbeater.cc:507] Master 127.14.23.126:39899 requested a full tablet report, sending...
I20260709 13:22:25.387255 24685 tablet_bootstrap.cc:492] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap complete.
I20260709 13:22:25.388046 24685 ts_tablet_manager.cc:1403] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent bootstrapping tablet: real 0.040s	user 0.026s	sys 0.011s
I20260709 13:22:25.390741 24685 raft_consensus.cc:359] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.391474 24685 raft_consensus.cc:740] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Initialized, Role: FOLLOWER
I20260709 13:22:25.394359 24248 ts_manager.cc:194] Re-registered known tserver with Master: 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105)
I20260709 13:22:25.398243 24722 raft_consensus.cc:3060] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:25.406592 24685 consensus_queue.cc:260] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.409262 24685 ts_tablet_manager.cc:1434] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 13:22:25.410305 24685 tablet_bootstrap.cc:492] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap starting.
I20260709 13:22:25.412931 24248 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58642
I20260709 13:22:25.418232 24748 heartbeater.cc:499] Master 127.14.23.126:39899 was elected leader, sending a full tablet report...
I20260709 13:22:25.426985 24500 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "248862fab3014daba862723841b4730a"
dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
mode: GRACEFUL
new_leader_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
 from {username='slave'} at 127.0.0.1:48920
I20260709 13:22:25.427686 24500 raft_consensus.cc:606] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 LEADER]: Received request to transfer leadership to TS 3c997d7a5c7f499abf3cb90c38f0ebe6
I20260709 13:22:25.430864 24685 tablet_bootstrap.cc:492] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:25.431953 24685 tablet_bootstrap.cc:492] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap complete.
I20260709 13:22:25.432806 24685 ts_tablet_manager.cc:1403] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.003s
I20260709 13:22:25.435463 24685 raft_consensus.cc:359] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.436224 24685 raft_consensus.cc:740] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Initialized, Role: FOLLOWER
I20260709 13:22:25.437001 24685 consensus_queue.cc:260] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.439411 24685 ts_tablet_manager.cc:1434] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:22:25.440332 24685 tablet_bootstrap.cc:492] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap starting.
I20260709 13:22:25.463078 24685 tablet_bootstrap.cc:492] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:25.464128 24685 tablet_bootstrap.cc:492] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap complete.
I20260709 13:22:25.464813 24685 ts_tablet_manager.cc:1403] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent bootstrapping tablet: real 0.025s	user 0.019s	sys 0.004s
I20260709 13:22:25.467401 24685 raft_consensus.cc:359] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.467916 24685 raft_consensus.cc:740] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Initialized, Role: FOLLOWER
I20260709 13:22:25.468698 24685 consensus_queue.cc:260] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.473379 24685 ts_tablet_manager.cc:1434] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 13:22:25.473994 24426 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5eeb2f0a1e54fdc8c94f815d9819277"
dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
mode: GRACEFUL
new_leader_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
 from {username='slave'} at 127.0.0.1:47522
I20260709 13:22:25.474363 24685 tablet_bootstrap.cc:492] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap starting.
I20260709 13:22:25.474849 24426 raft_consensus.cc:606] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 LEADER]: Received request to transfer leadership to TS 97364f1f8ccd429e9c9d22c09ecfd168
I20260709 13:22:25.476652 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:22:25.477355 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:25.478197 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:25.493482 24685 tablet_bootstrap.cc:492] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:25.495934 24685 tablet_bootstrap.cc:492] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Bootstrap complete.
I20260709 13:22:25.496762 24685 ts_tablet_manager.cc:1403] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.000s
I20260709 13:22:25.499564 24685 raft_consensus.cc:359] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } }
I20260709 13:22:25.500322 24685 raft_consensus.cc:740] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bef6e7eea474e7e8f3ec0e55f5b9d14, State: Initialized, Role: FOLLOWER
I20260709 13:22:25.501121 24685 consensus_queue.cc:260] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } }
I20260709 13:22:25.503225 24685 ts_tablet_manager.cc:1434] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 13:22:25.506937 24721 raft_consensus.cc:3060] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:25.548462 24721 raft_consensus.cc:3060] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:25.563145 24765 raft_consensus.cc:993] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168: : Instructing follower 3c997d7a5c7f499abf3cb90c38f0ebe6 to start an election
I20260709 13:22:25.563910 24668 raft_consensus.cc:1081] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 LEADER]: Signalling peer 3c997d7a5c7f499abf3cb90c38f0ebe6 to start an election
I20260709 13:22:25.565896 24426 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "248862fab3014daba862723841b4730a"
dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
 from {username='slave'} at 127.0.0.1:48338
I20260709 13:22:25.566588 24426 raft_consensus.cc:493] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:25.566948 24426 raft_consensus.cc:3060] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:25.573984 24426 raft_consensus.cc:515] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.575688 24426 leader_election.cc:290] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 3 election: Requested vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:25.576483 24721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248862fab3014daba862723841b4730a" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14"
I20260709 13:22:25.576738 24500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "248862fab3014daba862723841b4730a" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
I20260709 13:22:25.577253 24721 raft_consensus.cc:3060] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:25.577348 24500 raft_consensus.cc:3055] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:25.577746 24500 raft_consensus.cc:740] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Running, Role: LEADER
I20260709 13:22:25.578580 24500 consensus_queue.cc:260] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.579674 24500 raft_consensus.cc:3060] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:25.582716 24721 raft_consensus.cc:2468] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 3.
I20260709 13:22:25.583897 24390 leader_election.cc:304] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 3c997d7a5c7f499abf3cb90c38f0ebe6; no voters: 
I20260709 13:22:25.584721 24667 raft_consensus.cc:2804] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:25.585958 24667 raft_consensus.cc:697] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 LEADER]: Becoming Leader. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Running, Role: LEADER
I20260709 13:22:25.585995 24500 raft_consensus.cc:2468] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 3.
I20260709 13:22:25.587044 24667 consensus_queue.cc:237] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.594805 24247 catalog_manager.cc:5697] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 reported cstate change: term changed from 2 to 3, leader changed from 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67) to 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66). New cstate: current_term: 3 leader_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: UNKNOWN } } }
W20260709 13:22:25.640609 24298 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:25.684806 24721 raft_consensus.cc:3060] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:25.695953 24667 raft_consensus.cc:993] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6: : Instructing follower 97364f1f8ccd429e9c9d22c09ecfd168 to start an election
I20260709 13:22:25.696702 24767 raft_consensus.cc:1081] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 LEADER]: Signalling peer 97364f1f8ccd429e9c9d22c09ecfd168 to start an election
I20260709 13:22:25.698927 24500 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a5eeb2f0a1e54fdc8c94f815d9819277"
dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
 from {username='slave'} at 127.0.0.1:39112
I20260709 13:22:25.699515 24500 raft_consensus.cc:493] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:25.699927 24500 raft_consensus.cc:3060] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:25.706810 24500 raft_consensus.cc:515] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.708560 24500 leader_election.cc:290] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 3 election: Requested vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577)
I20260709 13:22:25.709414 24721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5eeb2f0a1e54fdc8c94f815d9819277" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14"
I20260709 13:22:25.709718 24426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5eeb2f0a1e54fdc8c94f815d9819277" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
I20260709 13:22:25.710130 24721 raft_consensus.cc:3060] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:25.710265 24426 raft_consensus.cc:3055] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:25.710639 24426 raft_consensus.cc:740] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Running, Role: LEADER
I20260709 13:22:25.711426 24426 consensus_queue.cc:260] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.712471 24426 raft_consensus.cc:3060] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:25.715727 24721 raft_consensus.cc:2468] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 3.
I20260709 13:22:25.716938 24464 leader_election.cc:304] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 
I20260709 13:22:25.717638 24668 raft_consensus.cc:2804] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:25.718060 24668 raft_consensus.cc:697] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 3 LEADER]: Becoming Leader. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Running, Role: LEADER
I20260709 13:22:25.718304 24426 raft_consensus.cc:2468] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 3.
I20260709 13:22:25.719044 24668 consensus_queue.cc:237] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:25.726681 24247 catalog_manager.cc:5697] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 reported cstate change: term changed from 2 to 3, leader changed from 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66) to 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67). New cstate: current_term: 3 leader_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:25.986522 24721 raft_consensus.cc:3060] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:26.084400 24721 raft_consensus.cc:1275] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 3 FOLLOWER]: Refusing update from remote peer 3c997d7a5c7f499abf3cb90c38f0ebe6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:26.085749 24774 consensus_queue.cc:1048] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:26.093958 24500 raft_consensus.cc:1275] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [term 3 FOLLOWER]: Refusing update from remote peer 3c997d7a5c7f499abf3cb90c38f0ebe6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:26.095309 24767 consensus_queue.cc:1048] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:26.179606 24426 raft_consensus.cc:1275] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 FOLLOWER]: Refusing update from remote peer 97364f1f8ccd429e9c9d22c09ecfd168: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:26.180939 24776 consensus_queue.cc:1048] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:26.190629 24721 raft_consensus.cc:1275] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 3 FOLLOWER]: Refusing update from remote peer 97364f1f8ccd429e9c9d22c09ecfd168: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:26.192276 24668 consensus_queue.cc:1048] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:26.488309 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:26.495368 24426 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "332fefa9cf2f4af28b3888c8620703d9"
dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
mode: GRACEFUL
new_leader_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
 from {username='slave'} at 127.0.0.1:47522
I20260709 13:22:26.495868 24426 raft_consensus.cc:606] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 LEADER]: Received request to transfer leadership to TS 97364f1f8ccd429e9c9d22c09ecfd168
I20260709 13:22:26.498514 24500 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe46fad433bd4a1d8070be4a4f4fdcf6"
dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
mode: GRACEFUL
new_leader_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
 from {username='slave'} at 127.0.0.1:48920
I20260709 13:22:26.499001 24500 raft_consensus.cc:606] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 LEADER]: Received request to transfer leadership to TS 3c997d7a5c7f499abf3cb90c38f0ebe6
I20260709 13:22:26.500062 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:22:26.500520 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:26.501035 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:26.520820 24782 raft_consensus.cc:993] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168: : Instructing follower 3c997d7a5c7f499abf3cb90c38f0ebe6 to start an election
I20260709 13:22:26.521337 24668 raft_consensus.cc:1081] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 LEADER]: Signalling peer 3c997d7a5c7f499abf3cb90c38f0ebe6 to start an election
I20260709 13:22:26.522782 24426 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fe46fad433bd4a1d8070be4a4f4fdcf6"
dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
 from {username='slave'} at 127.0.0.1:48338
I20260709 13:22:26.523324 24426 raft_consensus.cc:493] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:26.523633 24426 raft_consensus.cc:3060] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:26.529341 24426 raft_consensus.cc:515] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:26.531338 24426 leader_election.cc:290] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 3 election: Requested vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:26.532359 24721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe46fad433bd4a1d8070be4a4f4fdcf6" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14"
I20260709 13:22:26.532905 24500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe46fad433bd4a1d8070be4a4f4fdcf6" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
I20260709 13:22:26.533066 24721 raft_consensus.cc:3060] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:26.533568 24500 raft_consensus.cc:3055] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:26.533989 24500 raft_consensus.cc:740] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Running, Role: LEADER
I20260709 13:22:26.534853 24500 consensus_queue.cc:260] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:26.536206 24500 raft_consensus.cc:3060] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:26.540791 24721 raft_consensus.cc:2468] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 3.
I20260709 13:22:26.542192 24390 leader_election.cc:304] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 3c997d7a5c7f499abf3cb90c38f0ebe6; no voters: 
I20260709 13:22:26.543031 24667 raft_consensus.cc:2804] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:26.543440 24667 raft_consensus.cc:697] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 LEADER]: Becoming Leader. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Running, Role: LEADER
I20260709 13:22:26.543887 24500 raft_consensus.cc:2468] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 3.
I20260709 13:22:26.544299 24667 consensus_queue.cc:237] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:26.551867 24248 catalog_manager.cc:5697] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 reported cstate change: term changed from 2 to 3, leader changed from 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67) to 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66). New cstate: current_term: 3 leader_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: UNKNOWN } } }
W20260709 13:22:26.641588 24298 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:26.793536 24767 raft_consensus.cc:993] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6: : Instructing follower 97364f1f8ccd429e9c9d22c09ecfd168 to start an election
I20260709 13:22:26.794126 24667 raft_consensus.cc:1081] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 LEADER]: Signalling peer 97364f1f8ccd429e9c9d22c09ecfd168 to start an election
I20260709 13:22:26.795706 24500 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "332fefa9cf2f4af28b3888c8620703d9"
dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
 from {username='slave'} at 127.0.0.1:39112
I20260709 13:22:26.796190 24500 raft_consensus.cc:493] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:26.796480 24500 raft_consensus.cc:3060] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:26.802083 24500 raft_consensus.cc:515] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:26.803666 24500 leader_election.cc:290] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 election: Requested vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577)
I20260709 13:22:26.804608 24721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "332fefa9cf2f4af28b3888c8620703d9" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14"
I20260709 13:22:26.804950 24426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "332fefa9cf2f4af28b3888c8620703d9" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
I20260709 13:22:26.805303 24721 raft_consensus.cc:3060] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:26.805644 24426 raft_consensus.cc:3055] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:26.805936 24426 raft_consensus.cc:740] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Running, Role: LEADER
I20260709 13:22:26.806731 24426 consensus_queue.cc:260] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:26.807761 24426 raft_consensus.cc:3060] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:26.810868 24721 raft_consensus.cc:2468] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 2.
I20260709 13:22:26.812223 24464 leader_election.cc:304] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 
I20260709 13:22:26.813038 24668 raft_consensus.cc:2804] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:26.813089 24426 raft_consensus.cc:2468] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 2.
I20260709 13:22:26.813548 24668 raft_consensus.cc:697] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 LEADER]: Becoming Leader. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Running, Role: LEADER
I20260709 13:22:26.814587 24668 consensus_queue.cc:237] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:26.821223 24248 catalog_manager.cc:5697] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 reported cstate change: term changed from 1 to 2, leader changed from 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66) to 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:27.080983 24721 raft_consensus.cc:1275] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 3 FOLLOWER]: Refusing update from remote peer 3c997d7a5c7f499abf3cb90c38f0ebe6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:27.082221 24667 consensus_queue.cc:1048] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:27.094328 24500 raft_consensus.cc:1275] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 3 FOLLOWER]: Refusing update from remote peer 3c997d7a5c7f499abf3cb90c38f0ebe6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:27.096204 24667 consensus_queue.cc:1048] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:27.248497 24722 raft_consensus.cc:1275] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Refusing update from remote peer 97364f1f8ccd429e9c9d22c09ecfd168: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:27.249694 24668 consensus_queue.cc:1048] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:27.259583 24426 raft_consensus.cc:1275] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Refusing update from remote peer 97364f1f8ccd429e9c9d22c09ecfd168: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:27.261417 24782 consensus_queue.cc:1048] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:27.507438 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:27.514881 24500 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "332fefa9cf2f4af28b3888c8620703d9"
dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
mode: GRACEFUL
new_leader_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
 from {username='slave'} at 127.0.0.1:48920
I20260709 13:22:27.515352 24500 raft_consensus.cc:606] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 LEADER]: Received request to transfer leadership to TS 3c997d7a5c7f499abf3cb90c38f0ebe6
I20260709 13:22:27.517887 24426 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cefb0660f2f84bdfb7e9b47cb002b525"
dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
mode: GRACEFUL
new_leader_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
 from {username='slave'} at 127.0.0.1:47522
I20260709 13:22:27.518594 24426 raft_consensus.cc:606] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 LEADER]: Received request to transfer leadership to TS 97364f1f8ccd429e9c9d22c09ecfd168
I20260709 13:22:27.520678 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:22:27.521377 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:27.521994 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:27.619160 24767 raft_consensus.cc:993] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6: : Instructing follower 97364f1f8ccd429e9c9d22c09ecfd168 to start an election
I20260709 13:22:27.619673 24798 raft_consensus.cc:1081] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 LEADER]: Signalling peer 97364f1f8ccd429e9c9d22c09ecfd168 to start an election
I20260709 13:22:27.621224 24500 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cefb0660f2f84bdfb7e9b47cb002b525"
dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
 from {username='slave'} at 127.0.0.1:39112
I20260709 13:22:27.621665 24500 raft_consensus.cc:493] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:27.621965 24500 raft_consensus.cc:3060] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.626962 24500 raft_consensus.cc:515] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:27.629253 24500 leader_election.cc:290] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 election: Requested vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577)
I20260709 13:22:27.629830 24426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cefb0660f2f84bdfb7e9b47cb002b525" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
I20260709 13:22:27.630514 24426 raft_consensus.cc:3055] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:27.630613 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cefb0660f2f84bdfb7e9b47cb002b525" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14"
I20260709 13:22:27.630764 24426 raft_consensus.cc:740] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Running, Role: LEADER
I20260709 13:22:27.631357 24722 raft_consensus.cc:3060] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.631765 24426 consensus_queue.cc:260] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:27.632800 24426 raft_consensus.cc:3060] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.636775 24722 raft_consensus.cc:2468] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 2.
I20260709 13:22:27.637496 24426 raft_consensus.cc:2468] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 2.
I20260709 13:22:27.637856 24464 leader_election.cc:304] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 
I20260709 13:22:27.638695 24782 raft_consensus.cc:2804] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:27.639192 24782 raft_consensus.cc:697] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 LEADER]: Becoming Leader. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Running, Role: LEADER
I20260709 13:22:27.639966 24782 consensus_queue.cc:237] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
W20260709 13:22:27.642599 24298 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:27.647704 24248 catalog_manager.cc:5697] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168 reported cstate change: term changed from 1 to 2, leader changed from 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66) to 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:27.773507 24796 raft_consensus.cc:993] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168: : Instructing follower 3c997d7a5c7f499abf3cb90c38f0ebe6 to start an election
I20260709 13:22:27.774020 24804 raft_consensus.cc:1081] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 LEADER]: Signalling peer 3c997d7a5c7f499abf3cb90c38f0ebe6 to start an election
I20260709 13:22:27.775452 24426 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "332fefa9cf2f4af28b3888c8620703d9"
dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
 from {username='slave'} at 127.0.0.1:48338
I20260709 13:22:27.775892 24426 raft_consensus.cc:493] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:27.776163 24426 raft_consensus.cc:3060] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:27.781037 24426 raft_consensus.cc:515] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:27.782492 24426 leader_election.cc:290] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 3 election: Requested vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:27.783430 24722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "332fefa9cf2f4af28b3888c8620703d9" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14"
I20260709 13:22:27.783537 24500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "332fefa9cf2f4af28b3888c8620703d9" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
I20260709 13:22:27.784195 24722 raft_consensus.cc:3060] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:27.784273 24500 raft_consensus.cc:3055] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:27.784649 24500 raft_consensus.cc:740] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Running, Role: LEADER
I20260709 13:22:27.785283 24500 consensus_queue.cc:260] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:27.786175 24500 raft_consensus.cc:3060] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:27.789529 24722 raft_consensus.cc:2468] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 3.
I20260709 13:22:27.790735 24500 raft_consensus.cc:2468] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 3.
I20260709 13:22:27.790745 24390 leader_election.cc:304] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 3c997d7a5c7f499abf3cb90c38f0ebe6; no voters: 
I20260709 13:22:27.791755 24803 raft_consensus.cc:2804] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:27.792201 24803 raft_consensus.cc:697] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 LEADER]: Becoming Leader. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Running, Role: LEADER
I20260709 13:22:27.792923 24803 consensus_queue.cc:237] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:27.799347 24248 catalog_manager.cc:5697] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 reported cstate change: term changed from 2 to 3, leader changed from 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67) to 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66). New cstate: current_term: 3 leader_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:28.151252 24426 raft_consensus.cc:1275] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Refusing update from remote peer 97364f1f8ccd429e9c9d22c09ecfd168: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.152520 24804 consensus_queue.cc:1048] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:28.162040 24722 raft_consensus.cc:1275] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Refusing update from remote peer 97364f1f8ccd429e9c9d22c09ecfd168: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.164311 24796 consensus_queue.cc:1048] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:28.242225 24500 raft_consensus.cc:1275] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 3 FOLLOWER]: Refusing update from remote peer 3c997d7a5c7f499abf3cb90c38f0ebe6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:28.243402 24803 consensus_queue.cc:1048] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:28.253664 24722 raft_consensus.cc:1275] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 3 FOLLOWER]: Refusing update from remote peer 3c997d7a5c7f499abf3cb90c38f0ebe6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:28.255182 24811 consensus_queue.cc:1048] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:28.528683 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:28.535835 24499 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee0eadd86f434502b7ccabf5024d1253"
dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
mode: GRACEFUL
new_leader_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
 from {username='slave'} at 127.0.0.1:48920
I20260709 13:22:28.536365 24499 raft_consensus.cc:606] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 LEADER]: Received request to transfer leadership to TS 3c997d7a5c7f499abf3cb90c38f0ebe6
I20260709 13:22:28.538874 24426 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe46fad433bd4a1d8070be4a4f4fdcf6"
dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
mode: GRACEFUL
new_leader_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
 from {username='slave'} at 127.0.0.1:47522
I20260709 13:22:28.539346 24426 raft_consensus.cc:606] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 LEADER]: Received request to transfer leadership to TS 97364f1f8ccd429e9c9d22c09ecfd168
I20260709 13:22:28.540427 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:22:28.540874 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:28.541487 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:22:28.643577 24298 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:28.695379 24812 raft_consensus.cc:993] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6: : Instructing follower 97364f1f8ccd429e9c9d22c09ecfd168 to start an election
I20260709 13:22:28.695914 24811 raft_consensus.cc:1081] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 LEADER]: Signalling peer 97364f1f8ccd429e9c9d22c09ecfd168 to start an election
I20260709 13:22:28.697481 24499 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fe46fad433bd4a1d8070be4a4f4fdcf6"
dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
 from {username='slave'} at 127.0.0.1:39112
I20260709 13:22:28.698094 24499 raft_consensus.cc:493] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:28.698431 24499 raft_consensus.cc:3060] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:28.705294 24499 raft_consensus.cc:515] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:28.706984 24499 leader_election.cc:290] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 4 election: Requested vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577)
I20260709 13:22:28.707652 24721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe46fad433bd4a1d8070be4a4f4fdcf6" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14"
I20260709 13:22:28.707979 24426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe46fad433bd4a1d8070be4a4f4fdcf6" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
I20260709 13:22:28.708345 24721 raft_consensus.cc:3060] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:28.708531 24426 raft_consensus.cc:3055] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:22:28.708889 24426 raft_consensus.cc:740] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Running, Role: LEADER
I20260709 13:22:28.709620 24426 consensus_queue.cc:260] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:28.710918 24426 raft_consensus.cc:3060] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:28.715943 24721 raft_consensus.cc:2468] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 4.
I20260709 13:22:28.717358 24464 leader_election.cc:304] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 
I20260709 13:22:28.717700 24426 raft_consensus.cc:2468] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 4.
I20260709 13:22:28.718254 24796 raft_consensus.cc:2804] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:22:28.718721 24796 raft_consensus.cc:697] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 4 LEADER]: Becoming Leader. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Running, Role: LEADER
I20260709 13:22:28.719521 24796 consensus_queue.cc:237] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:28.726119 24247 catalog_manager.cc:5697] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 reported cstate change: term changed from 3 to 4, leader changed from 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66) to 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67). New cstate: current_term: 4 leader_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:28.861987 24804 raft_consensus.cc:993] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168: : Instructing follower 3c997d7a5c7f499abf3cb90c38f0ebe6 to start an election
I20260709 13:22:28.862423 24796 raft_consensus.cc:1081] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 LEADER]: Signalling peer 3c997d7a5c7f499abf3cb90c38f0ebe6 to start an election
I20260709 13:22:28.863940 24426 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee0eadd86f434502b7ccabf5024d1253"
dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
 from {username='slave'} at 127.0.0.1:48338
I20260709 13:22:28.864394 24426 raft_consensus.cc:493] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:28.864661 24426 raft_consensus.cc:3060] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.869632 24426 raft_consensus.cc:515] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:28.871136 24426 leader_election.cc:290] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 election: Requested vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:28.872084 24721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee0eadd86f434502b7ccabf5024d1253" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14"
I20260709 13:22:28.872087 24499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee0eadd86f434502b7ccabf5024d1253" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
I20260709 13:22:28.872817 24721 raft_consensus.cc:3060] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.872900 24499 raft_consensus.cc:3055] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:28.873297 24499 raft_consensus.cc:740] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Running, Role: LEADER
I20260709 13:22:28.874163 24499 consensus_queue.cc:260] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:28.875193 24499 raft_consensus.cc:3060] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.878296 24721 raft_consensus.cc:2468] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 2.
I20260709 13:22:28.879492 24390 leader_election.cc:304] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 3c997d7a5c7f499abf3cb90c38f0ebe6; no voters: 
I20260709 13:22:28.880164 24818 raft_consensus.cc:2804] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:28.880553 24818 raft_consensus.cc:697] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 LEADER]: Becoming Leader. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Running, Role: LEADER
I20260709 13:22:28.881156 24499 raft_consensus.cc:2468] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 2.
I20260709 13:22:28.881425 24818 consensus_queue.cc:237] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:28.888787 24247 catalog_manager.cc:5697] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6 reported cstate change: term changed from 1 to 2, leader changed from 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67) to 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:29.141723 24426 raft_consensus.cc:1275] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 4 FOLLOWER]: Refusing update from remote peer 97364f1f8ccd429e9c9d22c09ecfd168: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:22:29.143013 24796 consensus_queue.cc:1048] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:29.154610 24721 raft_consensus.cc:1275] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 4 FOLLOWER]: Refusing update from remote peer 97364f1f8ccd429e9c9d22c09ecfd168: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:22:29.156692 24822 consensus_queue.cc:1048] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:29.316843 24499 raft_consensus.cc:1275] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Refusing update from remote peer 3c997d7a5c7f499abf3cb90c38f0ebe6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:29.318061 24818 consensus_queue.cc:1048] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:29.327095 24721 raft_consensus.cc:1275] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Refusing update from remote peer 3c997d7a5c7f499abf3cb90c38f0ebe6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:29.328457 24828 consensus_queue.cc:1048] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:29.547912 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:29.555279 24426 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "332fefa9cf2f4af28b3888c8620703d9"
dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
mode: GRACEFUL
new_leader_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
 from {username='slave'} at 127.0.0.1:47522
I20260709 13:22:29.555763 24426 raft_consensus.cc:606] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 LEADER]: Received request to transfer leadership to TS 97364f1f8ccd429e9c9d22c09ecfd168
I20260709 13:22:29.558209 24499 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "968525e758744fb0b8b1a80822467255"
dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
mode: GRACEFUL
new_leader_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
 from {username='slave'} at 127.0.0.1:48920
I20260709 13:22:29.558671 24499 raft_consensus.cc:606] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 LEADER]: Received request to transfer leadership to TS 3c997d7a5c7f499abf3cb90c38f0ebe6
I20260709 13:22:29.559743 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:22:29.560247 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:29.560801 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:22:29.644574 24298 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:29.750418 24818 raft_consensus.cc:993] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6: : Instructing follower 97364f1f8ccd429e9c9d22c09ecfd168 to start an election
I20260709 13:22:29.750931 24828 raft_consensus.cc:1081] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 LEADER]: Signalling peer 97364f1f8ccd429e9c9d22c09ecfd168 to start an election
I20260709 13:22:29.752345 24499 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "332fefa9cf2f4af28b3888c8620703d9"
dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
 from {username='slave'} at 127.0.0.1:39112
I20260709 13:22:29.752837 24499 raft_consensus.cc:493] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:29.753127 24499 raft_consensus.cc:3060] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:29.758572 24499 raft_consensus.cc:515] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:29.760488 24499 leader_election.cc:290] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 4 election: Requested vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577)
I20260709 13:22:29.761691 24426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "332fefa9cf2f4af28b3888c8620703d9" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
I20260709 13:22:29.762435 24426 raft_consensus.cc:3055] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:22:29.762652 24426 raft_consensus.cc:740] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Running, Role: LEADER
I20260709 13:22:29.761703 24721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "332fefa9cf2f4af28b3888c8620703d9" candidate_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14"
I20260709 13:22:29.763252 24721 raft_consensus.cc:3060] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:29.763495 24426 consensus_queue.cc:260] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:29.764730 24426 raft_consensus.cc:3060] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:29.770229 24721 raft_consensus.cc:2468] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 4.
I20260709 13:22:29.771380 24426 raft_consensus.cc:2468] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97364f1f8ccd429e9c9d22c09ecfd168 in term 4.
I20260709 13:22:29.771546 24464 leader_election.cc:304] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 97364f1f8ccd429e9c9d22c09ecfd168; no voters: 
I20260709 13:22:29.772377 24831 raft_consensus.cc:2804] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:22:29.772856 24831 raft_consensus.cc:697] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 4 LEADER]: Becoming Leader. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Running, Role: LEADER
I20260709 13:22:29.773612 24831 consensus_queue.cc:237] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:29.780045 24248 catalog_manager.cc:5697] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 reported cstate change: term changed from 3 to 4, leader changed from 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66) to 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67). New cstate: current_term: 4 leader_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:29.977313 24830 raft_consensus.cc:993] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168: : Instructing follower 3c997d7a5c7f499abf3cb90c38f0ebe6 to start an election
I20260709 13:22:29.977773 24831 raft_consensus.cc:1081] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 LEADER]: Signalling peer 3c997d7a5c7f499abf3cb90c38f0ebe6 to start an election
I20260709 13:22:29.979200 24426 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "968525e758744fb0b8b1a80822467255"
dest_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6"
 from {username='slave'} at 127.0.0.1:48338
I20260709 13:22:29.979650 24426 raft_consensus.cc:493] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:29.979892 24426 raft_consensus.cc:3060] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:29.985316 24426 raft_consensus.cc:515] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:29.986768 24426 leader_election.cc:290] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 election: Requested vote from peers 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105), 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67:44867)
I20260709 13:22:29.987687 24721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968525e758744fb0b8b1a80822467255" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14"
I20260709 13:22:29.987824 24499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "968525e758744fb0b8b1a80822467255" candidate_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "97364f1f8ccd429e9c9d22c09ecfd168"
I20260709 13:22:29.988384 24721 raft_consensus.cc:3060] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:29.988457 24499 raft_consensus.cc:3055] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:29.988828 24499 raft_consensus.cc:740] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 97364f1f8ccd429e9c9d22c09ecfd168, State: Running, Role: LEADER
I20260709 13:22:29.989549 24499 consensus_queue.cc:260] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:29.990609 24499 raft_consensus.cc:3060] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:29.993480 24721 raft_consensus.cc:2468] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 2.
I20260709 13:22:29.994691 24390 leader_election.cc:304] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bef6e7eea474e7e8f3ec0e55f5b9d14, 3c997d7a5c7f499abf3cb90c38f0ebe6; no voters: 
I20260709 13:22:29.995389 24828 raft_consensus.cc:2804] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:29.995893 24828 raft_consensus.cc:697] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 LEADER]: Becoming Leader. State: Replica: 3c997d7a5c7f499abf3cb90c38f0ebe6, State: Running, Role: LEADER
I20260709 13:22:29.996184 24499 raft_consensus.cc:2468] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c997d7a5c7f499abf3cb90c38f0ebe6 in term 2.
I20260709 13:22:29.996903 24828 consensus_queue.cc:237] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } }
I20260709 13:22:30.003907 24248 catalog_manager.cc:5697] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6 reported cstate change: term changed from 1 to 2, leader changed from 97364f1f8ccd429e9c9d22c09ecfd168 (127.14.23.67) to 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:30.223220 24721 raft_consensus.cc:1275] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 4 FOLLOWER]: Refusing update from remote peer 97364f1f8ccd429e9c9d22c09ecfd168: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:22:30.224545 24831 consensus_queue.cc:1048] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:30.238036 24426 raft_consensus.cc:1275] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 4 FOLLOWER]: Refusing update from remote peer 97364f1f8ccd429e9c9d22c09ecfd168: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:22:30.240257 24830 consensus_queue.cc:1048] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c997d7a5c7f499abf3cb90c38f0ebe6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 41577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:30.567008 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:30.583160 24721 raft_consensus.cc:1275] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Refusing update from remote peer 3c997d7a5c7f499abf3cb90c38f0ebe6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:30.583712 24499 raft_consensus.cc:1275] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Refusing update from remote peer 3c997d7a5c7f499abf3cb90c38f0ebe6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:30.584306 24828 consensus_queue.cc:1048] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bef6e7eea474e7e8f3ec0e55f5b9d14" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 40105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:30.585120 24846 consensus_queue.cc:1048] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97364f1f8ccd429e9c9d22c09ecfd168" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 44867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:30.608017 14429 tablet_server.cc:179] TabletServer@127.14.23.65:40105 shutting down...
I20260709 13:22:30.637423 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:30.638106 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:30.638684 14429 raft_consensus.cc:2243] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:30.639110 14429 raft_consensus.cc:2272] T 688b155b91274e1c9102292eb67cc0c6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.640982 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:30.641530 14429 raft_consensus.cc:2243] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:30.641980 14429 raft_consensus.cc:2272] T 968525e758744fb0b8b1a80822467255 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.644204 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:30.644788 14429 raft_consensus.cc:2243] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:30.645272 14429 raft_consensus.cc:2272] T 332fefa9cf2f4af28b3888c8620703d9 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:30.645588 24298 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:30.647882 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:30.648495 14429 raft_consensus.cc:2243] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:30.648926 14429 raft_consensus.cc:2272] T 4e4065825e694d4fa259bbdb5538289f P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.650923 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:30.651468 14429 raft_consensus.cc:2243] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:30.651849 14429 raft_consensus.cc:2272] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.653591 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:30.654155 14429 raft_consensus.cc:2243] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:30.654547 14429 raft_consensus.cc:2272] T 248862fab3014daba862723841b4730a P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.656314 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:30.656823 14429 raft_consensus.cc:2243] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:30.657212 14429 raft_consensus.cc:2272] T ee0eadd86f434502b7ccabf5024d1253 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.659022 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:30.659515 14429 raft_consensus.cc:2243] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:30.659900 14429 raft_consensus.cc:2272] T a5eeb2f0a1e54fdc8c94f815d9819277 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.661648 14429 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:30.662163 14429 raft_consensus.cc:2243] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:30.662544 14429 raft_consensus.cc:2272] T 7c88fba243c24e7db4b3e7530fc9e881 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.664364 14429 tablet_replica.cc:333] stopping tablet replica
W20260709 13:22:30.664594 24390 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 7 similar messages]
I20260709 13:22:30.664870 14429 raft_consensus.cc:2243] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:30.665406 14429 raft_consensus.cc:2272] T cefb0660f2f84bdfb7e9b47cb002b525 P 1bef6e7eea474e7e8f3ec0e55f5b9d14 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:30.667503 24390 consensus_peers.cc:597] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:30.693713 14429 tablet_server.cc:196] TabletServer@127.14.23.65:40105 shutdown complete.
I20260709 13:22:30.712395 14429 tablet_server.cc:179] TabletServer@127.14.23.66:0 shutting down...
W20260709 13:22:30.719426 24465 consensus_peers.cc:597] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 -> Peer 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577): Couldn't send request to peer 3c997d7a5c7f499abf3cb90c38f0ebe6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:30.739113 24464 consensus_peers.cc:597] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:30.742626 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:30.743479 14429 tablet_replica.cc:333] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6: stopping tablet replica
I20260709 13:22:30.744288 14429 raft_consensus.cc:2243] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:30.744794 14429 raft_consensus.cc:2272] T cefb0660f2f84bdfb7e9b47cb002b525 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.747211 14429 tablet_replica.cc:333] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6: stopping tablet replica
I20260709 13:22:30.747838 14429 raft_consensus.cc:2243] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:30.748286 14429 raft_consensus.cc:2272] T a5eeb2f0a1e54fdc8c94f815d9819277 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.750525 14429 tablet_replica.cc:333] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6: stopping tablet replica
I20260709 13:22:30.751112 14429 raft_consensus.cc:2243] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:30.751968 14429 raft_consensus.cc:2272] T 7c88fba243c24e7db4b3e7530fc9e881 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.754289 14429 tablet_replica.cc:333] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6: stopping tablet replica
I20260709 13:22:30.754841 14429 raft_consensus.cc:2243] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:30.755287 14429 raft_consensus.cc:2272] T 688b155b91274e1c9102292eb67cc0c6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.757332 14429 tablet_replica.cc:333] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6: stopping tablet replica
I20260709 13:22:30.757831 14429 raft_consensus.cc:2243] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:30.758641 14429 raft_consensus.cc:2272] T ee0eadd86f434502b7ccabf5024d1253 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.760838 14429 tablet_replica.cc:333] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6: stopping tablet replica
I20260709 13:22:30.761409 14429 raft_consensus.cc:2243] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:30.762215 14429 raft_consensus.cc:2272] T 248862fab3014daba862723841b4730a P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.764199 14429 tablet_replica.cc:333] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6: stopping tablet replica
I20260709 13:22:30.764732 14429 raft_consensus.cc:2243] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:30.765163 14429 raft_consensus.cc:2272] T 332fefa9cf2f4af28b3888c8620703d9 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.767683 14429 tablet_replica.cc:333] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6: stopping tablet replica
I20260709 13:22:30.768210 14429 raft_consensus.cc:2243] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:30.768952 14429 raft_consensus.cc:2272] T 968525e758744fb0b8b1a80822467255 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.771327 14429 tablet_replica.cc:333] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6: stopping tablet replica
I20260709 13:22:30.771800 14429 raft_consensus.cc:2243] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:30.772332 14429 raft_consensus.cc:2272] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.775295 14429 tablet_replica.cc:333] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6: stopping tablet replica
I20260709 13:22:30.775859 14429 raft_consensus.cc:2243] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:22:30.776844 24464 consensus_peers.cc:597] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:30.776943 14429 raft_consensus.cc:2272] T 4e4065825e694d4fa259bbdb5538289f P 3c997d7a5c7f499abf3cb90c38f0ebe6 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:30.784201 24465 consensus_peers.cc:597] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 -> Peer 3c997d7a5c7f499abf3cb90c38f0ebe6 (127.14.23.66:41577): Couldn't send request to peer 3c997d7a5c7f499abf3cb90c38f0ebe6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:30.811110 14429 tablet_server.cc:196] TabletServer@127.14.23.66:0 shutdown complete.
W20260709 13:22:30.817744 24464 consensus_peers.cc:597] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 -> Peer 1bef6e7eea474e7e8f3ec0e55f5b9d14 (127.14.23.65:40105): Couldn't send request to peer 1bef6e7eea474e7e8f3ec0e55f5b9d14. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:40105: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:30.835424 14429 tablet_server.cc:179] TabletServer@127.14.23.67:0 shutting down...
I20260709 13:22:30.859515 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:30.860178 14429 tablet_replica.cc:333] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168: stopping tablet replica
I20260709 13:22:30.861001 14429 raft_consensus.cc:2243] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:30.861788 14429 raft_consensus.cc:2272] T a5eeb2f0a1e54fdc8c94f815d9819277 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.864027 14429 tablet_replica.cc:333] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168: stopping tablet replica
I20260709 13:22:30.864629 14429 raft_consensus.cc:2243] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:30.865414 14429 raft_consensus.cc:2272] T cefb0660f2f84bdfb7e9b47cb002b525 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.867578 14429 tablet_replica.cc:333] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168: stopping tablet replica
I20260709 13:22:30.868171 14429 raft_consensus.cc:2243] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:30.868642 14429 raft_consensus.cc:2272] T 7c88fba243c24e7db4b3e7530fc9e881 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.870883 14429 tablet_replica.cc:333] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168: stopping tablet replica
I20260709 13:22:30.871471 14429 raft_consensus.cc:2243] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:30.871903 14429 raft_consensus.cc:2272] T 248862fab3014daba862723841b4730a P 97364f1f8ccd429e9c9d22c09ecfd168 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.873980 14429 tablet_replica.cc:333] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168: stopping tablet replica
I20260709 13:22:30.874517 14429 raft_consensus.cc:2243] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:30.874920 14429 raft_consensus.cc:2272] T ee0eadd86f434502b7ccabf5024d1253 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.876951 14429 tablet_replica.cc:333] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168: stopping tablet replica
I20260709 13:22:30.877511 14429 raft_consensus.cc:2243] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:22:30.878357 14429 raft_consensus.cc:2272] T 332fefa9cf2f4af28b3888c8620703d9 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.881057 14429 tablet_replica.cc:333] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168: stopping tablet replica
I20260709 13:22:30.881583 14429 raft_consensus.cc:2243] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:30.882037 14429 raft_consensus.cc:2272] T 968525e758744fb0b8b1a80822467255 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.887661 14429 tablet_replica.cc:333] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168: stopping tablet replica
I20260709 13:22:30.888334 14429 raft_consensus.cc:2243] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:30.889170 14429 raft_consensus.cc:2272] T 688b155b91274e1c9102292eb67cc0c6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.891238 14429 tablet_replica.cc:333] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168: stopping tablet replica
I20260709 13:22:30.891722 14429 raft_consensus.cc:2243] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:22:30.892488 14429 raft_consensus.cc:2272] T fe46fad433bd4a1d8070be4a4f4fdcf6 P 97364f1f8ccd429e9c9d22c09ecfd168 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.894662 14429 tablet_replica.cc:333] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168: stopping tablet replica
I20260709 13:22:30.895157 14429 raft_consensus.cc:2243] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:30.895673 14429 raft_consensus.cc:2272] T 4e4065825e694d4fa259bbdb5538289f P 97364f1f8ccd429e9c9d22c09ecfd168 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.927850 14429 tablet_server.cc:196] TabletServer@127.14.23.67:0 shutdown complete.
I20260709 13:22:30.951809 14429 master.cc:562] Master@127.14.23.126:39899 shutting down...
I20260709 13:22:30.972465 14429 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:30.973009 14429 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:30.973354 14429 tablet_replica.cc:333] T 00000000000000000000000000000000 P ff69dc99e4c14bbbb85e0b2b71152203: stopping tablet replica
I20260709 13:22:31.013877 14429 master.cc:584] Master@127.14.23.126:39899 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (14559 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:22:31.056314 14429 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.23.126:38457
I20260709 13:22:31.057436 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:31.062686 24857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:31.062732 24856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:31.063694 24859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:31.064571 14429 server_base.cc:1061] running on GCE node
I20260709 13:22:31.065490 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:31.065687 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:31.065840 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603351065816 us; error 0 us; skew 500 ppm
I20260709 13:22:31.066407 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:31.068778 14429 webserver.cc:533] Webserver started at http://127.14.23.126:41013/ using document root <none> and password file <none>
I20260709 13:22:31.069268 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:31.069451 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:31.069715 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:31.070851 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603055563165-14429-0/minicluster-data/master-0-root/instance:
uuid: "35aaaee4d4404c66911ad50eb8b84a6c"
format_stamp: "Formatted at 2026-07-09 13:22:31 on dist-test-slave-twc1"
I20260709 13:22:31.075260 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:22:31.078694 24864 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:31.079507 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:22:31.079784 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603055563165-14429-0/minicluster-data/master-0-root
uuid: "35aaaee4d4404c66911ad50eb8b84a6c"
format_stamp: "Formatted at 2026-07-09 13:22:31 on dist-test-slave-twc1"
I20260709 13:22:31.080044 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603055563165-14429-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603055563165-14429-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603055563165-14429-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:31.117611 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:31.118793 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:31.156399 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.126:38457
I20260709 13:22:31.156545 24915 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.126:38457 every 8 connection(s)
I20260709 13:22:31.160391 24916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.172158 24916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c: Bootstrap starting.
I20260709 13:22:31.177350 24916 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.182070 24916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c: No bootstrap required, opened a new log
I20260709 13:22:31.184274 24916 raft_consensus.cc:359] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35aaaee4d4404c66911ad50eb8b84a6c" member_type: VOTER }
I20260709 13:22:31.184674 24916 raft_consensus.cc:385] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.184916 24916 raft_consensus.cc:740] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35aaaee4d4404c66911ad50eb8b84a6c, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.185498 24916 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35aaaee4d4404c66911ad50eb8b84a6c" member_type: VOTER }
I20260709 13:22:31.185985 24916 raft_consensus.cc:399] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:22:31.186201 24916 raft_consensus.cc:493] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:22:31.186470 24916 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.191251 24916 raft_consensus.cc:515] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35aaaee4d4404c66911ad50eb8b84a6c" member_type: VOTER }
I20260709 13:22:31.191820 24916 leader_election.cc:304] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35aaaee4d4404c66911ad50eb8b84a6c; no voters: 
I20260709 13:22:31.192981 24916 leader_election.cc:290] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:22:31.193380 24919 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:31.195019 24919 raft_consensus.cc:697] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c [term 1 LEADER]: Becoming Leader. State: Replica: 35aaaee4d4404c66911ad50eb8b84a6c, State: Running, Role: LEADER
I20260709 13:22:31.195823 24916 sys_catalog.cc:565] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:22:31.195791 24919 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35aaaee4d4404c66911ad50eb8b84a6c" member_type: VOTER }
I20260709 13:22:31.201025 24920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35aaaee4d4404c66911ad50eb8b84a6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35aaaee4d4404c66911ad50eb8b84a6c" member_type: VOTER } }
I20260709 13:22:31.201036 24921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35aaaee4d4404c66911ad50eb8b84a6c. Latest consensus state: current_term: 1 leader_uuid: "35aaaee4d4404c66911ad50eb8b84a6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35aaaee4d4404c66911ad50eb8b84a6c" member_type: VOTER } }
I20260709 13:22:31.201776 24920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:31.201859 24921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:31.206465 24926 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:22:31.211571 24926 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:22:31.213318 14429 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:22:31.221056 24926 catalog_manager.cc:1383] Generated new cluster ID: 7426b8c19f5449efa2e9b3e01b2bfb39
I20260709 13:22:31.221403 24926 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:22:31.260175 24926 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:22:31.261502 24926 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:22:31.277415 24926 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c: Generated new TSK 0
I20260709 13:22:31.278064 24926 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:22:31.344544 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:31.350771 24937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:31.351521 24938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:31.352712 24940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:31.353294 14429 server_base.cc:1061] running on GCE node
I20260709 13:22:31.354184 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:31.354367 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:31.354485 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603351354474 us; error 0 us; skew 500 ppm
I20260709 13:22:31.354939 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:31.357183 14429 webserver.cc:533] Webserver started at http://127.14.23.65:43999/ using document root <none> and password file <none>
I20260709 13:22:31.357625 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:31.357789 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:31.358040 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:31.359133 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603055563165-14429-0/minicluster-data/ts-0-root/instance:
uuid: "9d3e51cf592246a4a46c18fe23e74245"
format_stamp: "Formatted at 2026-07-09 13:22:31 on dist-test-slave-twc1"
I20260709 13:22:31.363615 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:22:31.367323 24945 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:31.368125 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 13:22:31.368448 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603055563165-14429-0/minicluster-data/ts-0-root
uuid: "9d3e51cf592246a4a46c18fe23e74245"
format_stamp: "Formatted at 2026-07-09 13:22:31 on dist-test-slave-twc1"
I20260709 13:22:31.368710 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603055563165-14429-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603055563165-14429-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603055563165-14429-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:31.402275 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:31.403509 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:31.404983 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:31.407310 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:31.407511 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:31.407739 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:31.407888 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:31.448144 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.65:37983
I20260709 13:22:31.448257 25007 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.65:37983 every 8 connection(s)
I20260709 13:22:31.453132 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:31.460749 25012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:31.461903 25013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:31.463829 25015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:31.466362 14429 server_base.cc:1061] running on GCE node
I20260709 13:22:31.466732 25008 heartbeater.cc:344] Connected to a master server at 127.14.23.126:38457
I20260709 13:22:31.467178 25008 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:31.467828 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:31.468139 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:31.468103 25008 heartbeater.cc:507] Master 127.14.23.126:38457 requested a full tablet report, sending...
I20260709 13:22:31.468575 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603351468550 us; error 0 us; skew 500 ppm
I20260709 13:22:31.469352 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:31.470811 24881 ts_manager.cc:194] Registered new tserver with Master: 9d3e51cf592246a4a46c18fe23e74245 (127.14.23.65:37983)
I20260709 13:22:31.472041 14429 webserver.cc:533] Webserver started at http://127.14.23.66:41651/ using document root <none> and password file <none>
I20260709 13:22:31.472564 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:31.472759 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:31.472805 24881 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44762
I20260709 13:22:31.473003 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:31.474514 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603055563165-14429-0/minicluster-data/ts-1-root/instance:
uuid: "c473984489f84c71bb0438b1433525aa"
format_stamp: "Formatted at 2026-07-09 13:22:31 on dist-test-slave-twc1"
I20260709 13:22:31.479295 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:22:31.482834 25020 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:31.483575 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:22:31.483881 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603055563165-14429-0/minicluster-data/ts-1-root
uuid: "c473984489f84c71bb0438b1433525aa"
format_stamp: "Formatted at 2026-07-09 13:22:31 on dist-test-slave-twc1"
I20260709 13:22:31.484158 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603055563165-14429-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603055563165-14429-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603055563165-14429-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:31.509140 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:31.510387 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:31.511893 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:31.514158 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:31.514364 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:31.514585 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:31.514732 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:31.555024 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.66:34827
I20260709 13:22:31.555130 25082 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.66:34827 every 8 connection(s)
I20260709 13:22:31.559695 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:31.566905 25086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:31.569289 25087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:31.570066 25089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:31.572520 25083 heartbeater.cc:344] Connected to a master server at 127.14.23.126:38457
I20260709 13:22:31.572940 25083 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:31.573653 25083 heartbeater.cc:507] Master 127.14.23.126:38457 requested a full tablet report, sending...
I20260709 13:22:31.575861 24881 ts_manager.cc:194] Registered new tserver with Master: c473984489f84c71bb0438b1433525aa (127.14.23.66:34827)
I20260709 13:22:31.576259 14429 server_base.cc:1061] running on GCE node
I20260709 13:22:31.577176 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:31.577374 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:31.577387 24881 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44772
I20260709 13:22:31.577520 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603351577508 us; error 0 us; skew 500 ppm
I20260709 13:22:31.578517 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:31.581044 14429 webserver.cc:533] Webserver started at http://127.14.23.67:33497/ using document root <none> and password file <none>
I20260709 13:22:31.581557 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:31.581768 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:31.582055 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:31.583101 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603055563165-14429-0/minicluster-data/ts-2-root/instance:
uuid: "4a1d671c7b7842d2acd7153d8c5ac466"
format_stamp: "Formatted at 2026-07-09 13:22:31 on dist-test-slave-twc1"
I20260709 13:22:31.587536 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:22:31.590979 25094 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:31.591725 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:22:31.592015 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603055563165-14429-0/minicluster-data/ts-2-root
uuid: "4a1d671c7b7842d2acd7153d8c5ac466"
format_stamp: "Formatted at 2026-07-09 13:22:31 on dist-test-slave-twc1"
I20260709 13:22:31.592304 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603055563165-14429-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603055563165-14429-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603055563165-14429-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:31.602157 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:31.603363 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:31.604813 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:31.607103 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:31.607314 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:31.607537 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:31.607686 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:31.647363 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.67:38101
I20260709 13:22:31.647462 25156 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.67:38101 every 8 connection(s)
I20260709 13:22:31.660948 25157 heartbeater.cc:344] Connected to a master server at 127.14.23.126:38457
I20260709 13:22:31.661430 25157 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:31.662237 25157 heartbeater.cc:507] Master 127.14.23.126:38457 requested a full tablet report, sending...
I20260709 13:22:31.664219 24881 ts_manager.cc:194] Registered new tserver with Master: 4a1d671c7b7842d2acd7153d8c5ac466 (127.14.23.67:38101)
I20260709 13:22:31.665066 14429 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014593851s
I20260709 13:22:31.665902 24881 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44774
I20260709 13:22:31.690912 24881 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44788:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:22:31.693029 24881 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:22:31.743688 25048 tablet_service.cc:1511] Processing CreateTablet for tablet fab8fd8145934e1886fcdcefecf7f1af (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=3a7061057ca34e18a9af58b4cc280764]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:22:31.745217 25048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fab8fd8145934e1886fcdcefecf7f1af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.745854 25047 tablet_service.cc:1511] Processing CreateTablet for tablet b959d6cfc29b40f29b6d6a8752307a9c (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=3a7061057ca34e18a9af58b4cc280764]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:22:31.747241 25047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b959d6cfc29b40f29b6d6a8752307a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.752002 25121 tablet_service.cc:1511] Processing CreateTablet for tablet b959d6cfc29b40f29b6d6a8752307a9c (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=3a7061057ca34e18a9af58b4cc280764]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:22:31.752213 25122 tablet_service.cc:1511] Processing CreateTablet for tablet fab8fd8145934e1886fcdcefecf7f1af (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=3a7061057ca34e18a9af58b4cc280764]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:22:31.752941 24973 tablet_service.cc:1511] Processing CreateTablet for tablet fab8fd8145934e1886fcdcefecf7f1af (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=3a7061057ca34e18a9af58b4cc280764]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:22:31.753301 24972 tablet_service.cc:1511] Processing CreateTablet for tablet b959d6cfc29b40f29b6d6a8752307a9c (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=3a7061057ca34e18a9af58b4cc280764]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:22:31.753582 25121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b959d6cfc29b40f29b6d6a8752307a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.754381 24973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fab8fd8145934e1886fcdcefecf7f1af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.754882 25122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fab8fd8145934e1886fcdcefecf7f1af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.755429 24972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b959d6cfc29b40f29b6d6a8752307a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.780519 25172 tablet_bootstrap.cc:492] T fab8fd8145934e1886fcdcefecf7f1af P c473984489f84c71bb0438b1433525aa: Bootstrap starting.
I20260709 13:22:31.782191 25174 tablet_bootstrap.cc:492] T fab8fd8145934e1886fcdcefecf7f1af P 4a1d671c7b7842d2acd7153d8c5ac466: Bootstrap starting.
I20260709 13:22:31.788251 25173 tablet_bootstrap.cc:492] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245: Bootstrap starting.
I20260709 13:22:31.790901 25172 tablet_bootstrap.cc:654] T fab8fd8145934e1886fcdcefecf7f1af P c473984489f84c71bb0438b1433525aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.793450 25174 tablet_bootstrap.cc:654] T fab8fd8145934e1886fcdcefecf7f1af P 4a1d671c7b7842d2acd7153d8c5ac466: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.794865 25173 tablet_bootstrap.cc:654] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.806633 25172 tablet_bootstrap.cc:492] T fab8fd8145934e1886fcdcefecf7f1af P c473984489f84c71bb0438b1433525aa: No bootstrap required, opened a new log
I20260709 13:22:31.807296 25172 ts_tablet_manager.cc:1403] T fab8fd8145934e1886fcdcefecf7f1af P c473984489f84c71bb0438b1433525aa: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.004s
I20260709 13:22:31.810657 25174 tablet_bootstrap.cc:492] T fab8fd8145934e1886fcdcefecf7f1af P 4a1d671c7b7842d2acd7153d8c5ac466: No bootstrap required, opened a new log
I20260709 13:22:31.810600 25172 raft_consensus.cc:359] T fab8fd8145934e1886fcdcefecf7f1af P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:31.811336 25172 raft_consensus.cc:385] T fab8fd8145934e1886fcdcefecf7f1af P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.811371 25174 ts_tablet_manager.cc:1403] T fab8fd8145934e1886fcdcefecf7f1af P 4a1d671c7b7842d2acd7153d8c5ac466: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.008s
I20260709 13:22:31.811614 25172 raft_consensus.cc:740] T fab8fd8145934e1886fcdcefecf7f1af P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c473984489f84c71bb0438b1433525aa, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.811753 25173 tablet_bootstrap.cc:492] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245: No bootstrap required, opened a new log
I20260709 13:22:31.812320 25173 ts_tablet_manager.cc:1403] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.009s
I20260709 13:22:31.812376 25172 consensus_queue.cc:260] T fab8fd8145934e1886fcdcefecf7f1af P c473984489f84c71bb0438b1433525aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:31.814674 25083 heartbeater.cc:499] Master 127.14.23.126:38457 was elected leader, sending a full tablet report...
I20260709 13:22:31.814462 25174 raft_consensus.cc:359] T fab8fd8145934e1886fcdcefecf7f1af P 4a1d671c7b7842d2acd7153d8c5ac466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:31.815312 25174 raft_consensus.cc:385] T fab8fd8145934e1886fcdcefecf7f1af P 4a1d671c7b7842d2acd7153d8c5ac466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.815676 25174 raft_consensus.cc:740] T fab8fd8145934e1886fcdcefecf7f1af P 4a1d671c7b7842d2acd7153d8c5ac466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a1d671c7b7842d2acd7153d8c5ac466, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.815475 25173 raft_consensus.cc:359] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:31.816293 25173 raft_consensus.cc:385] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.816619 25173 raft_consensus.cc:740] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d3e51cf592246a4a46c18fe23e74245, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.816486 25174 consensus_queue.cc:260] T fab8fd8145934e1886fcdcefecf7f1af P 4a1d671c7b7842d2acd7153d8c5ac466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:31.817552 25173 consensus_queue.cc:260] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:31.819262 25157 heartbeater.cc:499] Master 127.14.23.126:38457 was elected leader, sending a full tablet report...
I20260709 13:22:31.822415 25174 ts_tablet_manager.cc:1434] T fab8fd8145934e1886fcdcefecf7f1af P 4a1d671c7b7842d2acd7153d8c5ac466: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.002s
I20260709 13:22:31.823702 25174 tablet_bootstrap.cc:492] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466: Bootstrap starting.
I20260709 13:22:31.824967 25172 ts_tablet_manager.cc:1434] T fab8fd8145934e1886fcdcefecf7f1af P c473984489f84c71bb0438b1433525aa: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.002s
I20260709 13:22:31.826155 25008 heartbeater.cc:499] Master 127.14.23.126:38457 was elected leader, sending a full tablet report...
I20260709 13:22:31.827270 25173 ts_tablet_manager.cc:1434] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.004s
I20260709 13:22:31.828428 25173 tablet_bootstrap.cc:492] T b959d6cfc29b40f29b6d6a8752307a9c P 9d3e51cf592246a4a46c18fe23e74245: Bootstrap starting.
I20260709 13:22:31.828959 25172 tablet_bootstrap.cc:492] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa: Bootstrap starting.
I20260709 13:22:31.835351 25173 tablet_bootstrap.cc:654] T b959d6cfc29b40f29b6d6a8752307a9c P 9d3e51cf592246a4a46c18fe23e74245: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.836045 25174 tablet_bootstrap.cc:654] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.842756 25173 tablet_bootstrap.cc:492] T b959d6cfc29b40f29b6d6a8752307a9c P 9d3e51cf592246a4a46c18fe23e74245: No bootstrap required, opened a new log
I20260709 13:22:31.843444 25173 ts_tablet_manager.cc:1403] T b959d6cfc29b40f29b6d6a8752307a9c P 9d3e51cf592246a4a46c18fe23e74245: Time spent bootstrapping tablet: real 0.015s	user 0.014s	sys 0.000s
I20260709 13:22:31.844883 25172 tablet_bootstrap.cc:654] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.846959 25173 raft_consensus.cc:359] T b959d6cfc29b40f29b6d6a8752307a9c P 9d3e51cf592246a4a46c18fe23e74245 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:31.847640 25173 raft_consensus.cc:385] T b959d6cfc29b40f29b6d6a8752307a9c P 9d3e51cf592246a4a46c18fe23e74245 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.847995 25173 raft_consensus.cc:740] T b959d6cfc29b40f29b6d6a8752307a9c P 9d3e51cf592246a4a46c18fe23e74245 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d3e51cf592246a4a46c18fe23e74245, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.849150 25174 tablet_bootstrap.cc:492] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466: No bootstrap required, opened a new log
I20260709 13:22:31.848820 25173 consensus_queue.cc:260] T b959d6cfc29b40f29b6d6a8752307a9c P 9d3e51cf592246a4a46c18fe23e74245 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:31.849726 25174 ts_tablet_manager.cc:1403] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.001s
I20260709 13:22:31.850057 25172 tablet_bootstrap.cc:492] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa: No bootstrap required, opened a new log
I20260709 13:22:31.850654 25172 ts_tablet_manager.cc:1403] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.000s
I20260709 13:22:31.851040 25173 ts_tablet_manager.cc:1434] T b959d6cfc29b40f29b6d6a8752307a9c P 9d3e51cf592246a4a46c18fe23e74245: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:22:31.852875 25174 raft_consensus.cc:359] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:31.853492 25174 raft_consensus.cc:385] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.853762 25174 raft_consensus.cc:740] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a1d671c7b7842d2acd7153d8c5ac466, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.853564 25172 raft_consensus.cc:359] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:31.854693 25172 raft_consensus.cc:385] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.854674 25174 consensus_queue.cc:260] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:31.855185 25172 raft_consensus.cc:740] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c473984489f84c71bb0438b1433525aa, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.856519 25174 ts_tablet_manager.cc:1434] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:22:31.857568 25172 consensus_queue.cc:260] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:31.859779 25172 ts_tablet_manager.cc:1434] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:22:32.030444 25180 raft_consensus.cc:493] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.030997 25180 raft_consensus.cc:515] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.033082 25180 leader_election.cc:290] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c473984489f84c71bb0438b1433525aa (127.14.23.66:34827), 4a1d671c7b7842d2acd7153d8c5ac466 (127.14.23.67:38101)
I20260709 13:22:32.041972 25178 raft_consensus.cc:493] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.042583 25178 raft_consensus.cc:515] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.042829 25058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab8fd8145934e1886fcdcefecf7f1af" candidate_uuid: "9d3e51cf592246a4a46c18fe23e74245" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c473984489f84c71bb0438b1433525aa" is_pre_election: true
I20260709 13:22:32.043641 25058 raft_consensus.cc:2468] T fab8fd8145934e1886fcdcefecf7f1af P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d3e51cf592246a4a46c18fe23e74245 in term 0.
I20260709 13:22:32.044309 25132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab8fd8145934e1886fcdcefecf7f1af" candidate_uuid: "9d3e51cf592246a4a46c18fe23e74245" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" is_pre_election: true
I20260709 13:22:32.045058 25132 raft_consensus.cc:2468] T fab8fd8145934e1886fcdcefecf7f1af P 4a1d671c7b7842d2acd7153d8c5ac466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d3e51cf592246a4a46c18fe23e74245 in term 0.
I20260709 13:22:32.045264 24947 leader_election.cc:304] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d3e51cf592246a4a46c18fe23e74245, c473984489f84c71bb0438b1433525aa; no voters: 
I20260709 13:22:32.046923 25180 raft_consensus.cc:2804] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:32.047310 25180 raft_consensus.cc:493] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:32.047524 25178 leader_election.cc:290] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d3e51cf592246a4a46c18fe23e74245 (127.14.23.65:37983), 4a1d671c7b7842d2acd7153d8c5ac466 (127.14.23.67:38101)
I20260709 13:22:32.047645 25180 raft_consensus.cc:3060] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:32.055606 25180 raft_consensus.cc:515] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.055703 24983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b959d6cfc29b40f29b6d6a8752307a9c" candidate_uuid: "c473984489f84c71bb0438b1433525aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d3e51cf592246a4a46c18fe23e74245" is_pre_election: true
I20260709 13:22:32.056591 24983 raft_consensus.cc:2468] T b959d6cfc29b40f29b6d6a8752307a9c P 9d3e51cf592246a4a46c18fe23e74245 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c473984489f84c71bb0438b1433525aa in term 0.
I20260709 13:22:32.057650 25021 leader_election.cc:304] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d3e51cf592246a4a46c18fe23e74245, c473984489f84c71bb0438b1433525aa; no voters: 
I20260709 13:22:32.058413 25178 raft_consensus.cc:2804] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:32.058519 25180 leader_election.cc:290] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245 [CANDIDATE]: Term 1 election: Requested vote from peers c473984489f84c71bb0438b1433525aa (127.14.23.66:34827), 4a1d671c7b7842d2acd7153d8c5ac466 (127.14.23.67:38101)
I20260709 13:22:32.058929 25178 raft_consensus.cc:493] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:32.059123 25132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b959d6cfc29b40f29b6d6a8752307a9c" candidate_uuid: "c473984489f84c71bb0438b1433525aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" is_pre_election: true
I20260709 13:22:32.059316 25178 raft_consensus.cc:3060] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:32.059456 25058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab8fd8145934e1886fcdcefecf7f1af" candidate_uuid: "9d3e51cf592246a4a46c18fe23e74245" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c473984489f84c71bb0438b1433525aa"
I20260709 13:22:32.059844 25132 raft_consensus.cc:2468] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c473984489f84c71bb0438b1433525aa in term 0.
I20260709 13:22:32.060118 25058 raft_consensus.cc:3060] T fab8fd8145934e1886fcdcefecf7f1af P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:32.061498 25132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab8fd8145934e1886fcdcefecf7f1af" candidate_uuid: "9d3e51cf592246a4a46c18fe23e74245" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a1d671c7b7842d2acd7153d8c5ac466"
I20260709 13:22:32.062031 25132 raft_consensus.cc:3060] T fab8fd8145934e1886fcdcefecf7f1af P 4a1d671c7b7842d2acd7153d8c5ac466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:32.067492 25132 raft_consensus.cc:2468] T fab8fd8145934e1886fcdcefecf7f1af P 4a1d671c7b7842d2acd7153d8c5ac466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d3e51cf592246a4a46c18fe23e74245 in term 1.
I20260709 13:22:32.067703 25178 raft_consensus.cc:515] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.068060 25058 raft_consensus.cc:2468] T fab8fd8145934e1886fcdcefecf7f1af P c473984489f84c71bb0438b1433525aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d3e51cf592246a4a46c18fe23e74245 in term 1.
I20260709 13:22:32.068539 24947 leader_election.cc:304] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a1d671c7b7842d2acd7153d8c5ac466, 9d3e51cf592246a4a46c18fe23e74245; no voters: 
I20260709 13:22:32.069334 25180 raft_consensus.cc:2804] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:32.070617 25178 leader_election.cc:290] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [CANDIDATE]: Term 1 election: Requested vote from peers 9d3e51cf592246a4a46c18fe23e74245 (127.14.23.65:37983), 4a1d671c7b7842d2acd7153d8c5ac466 (127.14.23.67:38101)
I20260709 13:22:32.070657 24983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b959d6cfc29b40f29b6d6a8752307a9c" candidate_uuid: "c473984489f84c71bb0438b1433525aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d3e51cf592246a4a46c18fe23e74245"
I20260709 13:22:32.071539 24983 raft_consensus.cc:3060] T b959d6cfc29b40f29b6d6a8752307a9c P 9d3e51cf592246a4a46c18fe23e74245 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:32.071842 25180 raft_consensus.cc:697] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245 [term 1 LEADER]: Becoming Leader. State: Replica: 9d3e51cf592246a4a46c18fe23e74245, State: Running, Role: LEADER
I20260709 13:22:32.072616 25132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b959d6cfc29b40f29b6d6a8752307a9c" candidate_uuid: "c473984489f84c71bb0438b1433525aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a1d671c7b7842d2acd7153d8c5ac466"
I20260709 13:22:32.073285 25132 raft_consensus.cc:3060] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:32.072894 25180 consensus_queue.cc:237] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.078984 24983 raft_consensus.cc:2468] T b959d6cfc29b40f29b6d6a8752307a9c P 9d3e51cf592246a4a46c18fe23e74245 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c473984489f84c71bb0438b1433525aa in term 1.
I20260709 13:22:32.080159 25021 leader_election.cc:304] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d3e51cf592246a4a46c18fe23e74245, c473984489f84c71bb0438b1433525aa; no voters: 
I20260709 13:22:32.080862 25178 raft_consensus.cc:2804] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:32.081318 25132 raft_consensus.cc:2468] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c473984489f84c71bb0438b1433525aa in term 1.
I20260709 13:22:32.082136 25178 raft_consensus.cc:697] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [term 1 LEADER]: Becoming Leader. State: Replica: c473984489f84c71bb0438b1433525aa, State: Running, Role: LEADER
I20260709 13:22:32.083079 25178 consensus_queue.cc:237] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.086781 24881 catalog_manager.cc:5697] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d3e51cf592246a4a46c18fe23e74245 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "9d3e51cf592246a4a46c18fe23e74245" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:32.091740 24880 catalog_manager.cc:5697] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa reported cstate change: term changed from 0 to 1, leader changed from <none> to c473984489f84c71bb0438b1433525aa (127.14.23.66). New cstate: current_term: 1 leader_uuid: "c473984489f84c71bb0438b1433525aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:32.142010 24880 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41300:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:22:32.143994 24880 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:22:32.167241 24972 tablet_service.cc:1511] Processing CreateTablet for tablet 729243f232ce416788915c104c990735 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=d3e68ec8167a4ef8a94fec9f8b195756]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:22:32.168574 24972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 729243f232ce416788915c104c990735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:32.168322 25047 tablet_service.cc:1511] Processing CreateTablet for tablet 729243f232ce416788915c104c990735 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=d3e68ec8167a4ef8a94fec9f8b195756]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:22:32.169608 25047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 729243f232ce416788915c104c990735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:32.170511 25121 tablet_service.cc:1511] Processing CreateTablet for tablet 729243f232ce416788915c104c990735 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=d3e68ec8167a4ef8a94fec9f8b195756]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:22:32.171278 24973 tablet_service.cc:1511] Processing CreateTablet for tablet 334c2584466b48e98903b63551f8caa2 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=d3e68ec8167a4ef8a94fec9f8b195756]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:22:32.171828 25121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 729243f232ce416788915c104c990735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:32.172626 24973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 334c2584466b48e98903b63551f8caa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:32.182948 25048 tablet_service.cc:1511] Processing CreateTablet for tablet 334c2584466b48e98903b63551f8caa2 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=d3e68ec8167a4ef8a94fec9f8b195756]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:22:32.184352 25048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 334c2584466b48e98903b63551f8caa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:32.185773 25173 tablet_bootstrap.cc:492] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245: Bootstrap starting.
I20260709 13:22:32.188148 25172 tablet_bootstrap.cc:492] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa: Bootstrap starting.
I20260709 13:22:32.189633 25174 tablet_bootstrap.cc:492] T 729243f232ce416788915c104c990735 P 4a1d671c7b7842d2acd7153d8c5ac466: Bootstrap starting.
I20260709 13:22:32.192451 25173 tablet_bootstrap.cc:654] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:32.195178 25172 tablet_bootstrap.cc:654] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:32.196879 25174 tablet_bootstrap.cc:654] T 729243f232ce416788915c104c990735 P 4a1d671c7b7842d2acd7153d8c5ac466: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:32.207855 25121 tablet_service.cc:1511] Processing CreateTablet for tablet 334c2584466b48e98903b63551f8caa2 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=d3e68ec8167a4ef8a94fec9f8b195756]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
W20260709 13:22:32.208478 24929 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:32.209218 25121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 334c2584466b48e98903b63551f8caa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:32.213982 25172 tablet_bootstrap.cc:492] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa: No bootstrap required, opened a new log
I20260709 13:22:32.214993 25172 ts_tablet_manager.cc:1403] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa: Time spent bootstrapping tablet: real 0.027s	user 0.019s	sys 0.000s
I20260709 13:22:32.216687 25173 tablet_bootstrap.cc:492] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245: No bootstrap required, opened a new log
I20260709 13:22:32.218596 25172 raft_consensus.cc:359] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.220892 25173 ts_tablet_manager.cc:1403] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245: Time spent bootstrapping tablet: real 0.035s	user 0.016s	sys 0.006s
I20260709 13:22:32.221275 25172 raft_consensus.cc:385] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:32.221733 25172 raft_consensus.cc:740] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c473984489f84c71bb0438b1433525aa, State: Initialized, Role: FOLLOWER
I20260709 13:22:32.221985 25174 tablet_bootstrap.cc:492] T 729243f232ce416788915c104c990735 P 4a1d671c7b7842d2acd7153d8c5ac466: No bootstrap required, opened a new log
I20260709 13:22:32.222715 25174 ts_tablet_manager.cc:1403] T 729243f232ce416788915c104c990735 P 4a1d671c7b7842d2acd7153d8c5ac466: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.007s
I20260709 13:22:32.222738 25172 consensus_queue.cc:260] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.224332 25173 raft_consensus.cc:359] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.225210 25173 raft_consensus.cc:385] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:32.225659 25172 ts_tablet_manager.cc:1434] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:22:32.226030 25174 raft_consensus.cc:359] T 729243f232ce416788915c104c990735 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.225545 25173 raft_consensus.cc:740] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d3e51cf592246a4a46c18fe23e74245, State: Initialized, Role: FOLLOWER
I20260709 13:22:32.226847 25172 tablet_bootstrap.cc:492] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa: Bootstrap starting.
I20260709 13:22:32.227628 25173 consensus_queue.cc:260] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.226854 25174 raft_consensus.cc:385] T 729243f232ce416788915c104c990735 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:32.232412 25174 raft_consensus.cc:740] T 729243f232ce416788915c104c990735 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a1d671c7b7842d2acd7153d8c5ac466, State: Initialized, Role: FOLLOWER
I20260709 13:22:32.234354 25172 tablet_bootstrap.cc:654] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:32.234935 25174 consensus_queue.cc:260] T 729243f232ce416788915c104c990735 P 4a1d671c7b7842d2acd7153d8c5ac466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.237613 25174 ts_tablet_manager.cc:1434] T 729243f232ce416788915c104c990735 P 4a1d671c7b7842d2acd7153d8c5ac466: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:22:32.238706 25174 tablet_bootstrap.cc:492] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466: Bootstrap starting.
I20260709 13:22:32.234488 25173 ts_tablet_manager.cc:1434] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:22:32.245096 25172 tablet_bootstrap.cc:492] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa: No bootstrap required, opened a new log
I20260709 13:22:32.245368 25174 tablet_bootstrap.cc:654] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:32.245761 25172 ts_tablet_manager.cc:1403] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.008s
I20260709 13:22:32.249703 25173 tablet_bootstrap.cc:492] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245: Bootstrap starting.
I20260709 13:22:32.248970 25172 raft_consensus.cc:359] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.250377 25172 raft_consensus.cc:385] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:32.250756 25172 raft_consensus.cc:740] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c473984489f84c71bb0438b1433525aa, State: Initialized, Role: FOLLOWER
I20260709 13:22:32.251259 25174 tablet_bootstrap.cc:492] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466: No bootstrap required, opened a new log
I20260709 13:22:32.251869 25174 ts_tablet_manager.cc:1403] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466: Time spent bootstrapping tablet: real 0.013s	user 0.008s	sys 0.004s
I20260709 13:22:32.252650 25172 consensus_queue.cc:260] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.255230 25172 ts_tablet_manager.cc:1434] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:22:32.256968 25173 tablet_bootstrap.cc:654] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:32.256541 25174 raft_consensus.cc:359] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.257421 25174 raft_consensus.cc:385] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:32.257752 25174 raft_consensus.cc:740] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a1d671c7b7842d2acd7153d8c5ac466, State: Initialized, Role: FOLLOWER
I20260709 13:22:32.258605 25174 consensus_queue.cc:260] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.260835 25174 ts_tablet_manager.cc:1434] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.000s
I20260709 13:22:32.264437 25173 tablet_bootstrap.cc:492] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245: No bootstrap required, opened a new log
I20260709 13:22:32.265084 25173 ts_tablet_manager.cc:1403] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245: Time spent bootstrapping tablet: real 0.016s	user 0.006s	sys 0.007s
I20260709 13:22:32.268168 25173 raft_consensus.cc:359] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.268923 25173 raft_consensus.cc:385] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:32.269167 25173 raft_consensus.cc:740] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d3e51cf592246a4a46c18fe23e74245, State: Initialized, Role: FOLLOWER
I20260709 13:22:32.269811 25173 consensus_queue.cc:260] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.271795 25173 ts_tablet_manager.cc:1434] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:22:32.350163 25178 raft_consensus.cc:493] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.350699 25178 raft_consensus.cc:515] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.352305 25178 leader_election.cc:290] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d3e51cf592246a4a46c18fe23e74245 (127.14.23.65:37983), 4a1d671c7b7842d2acd7153d8c5ac466 (127.14.23.67:38101)
I20260709 13:22:32.353109 24983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "729243f232ce416788915c104c990735" candidate_uuid: "c473984489f84c71bb0438b1433525aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d3e51cf592246a4a46c18fe23e74245" is_pre_election: true
I20260709 13:22:32.353470 25132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "729243f232ce416788915c104c990735" candidate_uuid: "c473984489f84c71bb0438b1433525aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" is_pre_election: true
I20260709 13:22:32.353857 24983 raft_consensus.cc:2468] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c473984489f84c71bb0438b1433525aa in term 0.
I20260709 13:22:32.354156 25132 raft_consensus.cc:2468] T 729243f232ce416788915c104c990735 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c473984489f84c71bb0438b1433525aa in term 0.
I20260709 13:22:32.354964 25021 leader_election.cc:304] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d3e51cf592246a4a46c18fe23e74245, c473984489f84c71bb0438b1433525aa; no voters: 
I20260709 13:22:32.355615 25178 raft_consensus.cc:2804] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:32.355958 25178 raft_consensus.cc:493] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:32.356215 25178 raft_consensus.cc:3060] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:32.361200 25178 raft_consensus.cc:515] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.363063 25178 leader_election.cc:290] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [CANDIDATE]: Term 1 election: Requested vote from peers 9d3e51cf592246a4a46c18fe23e74245 (127.14.23.65:37983), 4a1d671c7b7842d2acd7153d8c5ac466 (127.14.23.67:38101)
I20260709 13:22:32.363803 24983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "729243f232ce416788915c104c990735" candidate_uuid: "c473984489f84c71bb0438b1433525aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d3e51cf592246a4a46c18fe23e74245"
I20260709 13:22:32.364172 25132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "729243f232ce416788915c104c990735" candidate_uuid: "c473984489f84c71bb0438b1433525aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a1d671c7b7842d2acd7153d8c5ac466"
I20260709 13:22:32.364392 24983 raft_consensus.cc:3060] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:32.364697 25132 raft_consensus.cc:3060] T 729243f232ce416788915c104c990735 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:32.369668 25132 raft_consensus.cc:2468] T 729243f232ce416788915c104c990735 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c473984489f84c71bb0438b1433525aa in term 1.
I20260709 13:22:32.369704 24983 raft_consensus.cc:2468] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c473984489f84c71bb0438b1433525aa in term 1.
I20260709 13:22:32.370900 25022 leader_election.cc:304] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a1d671c7b7842d2acd7153d8c5ac466, c473984489f84c71bb0438b1433525aa; no voters: 
I20260709 13:22:32.371655 25178 raft_consensus.cc:2804] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:32.372071 25178 raft_consensus.cc:697] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [term 1 LEADER]: Becoming Leader. State: Replica: c473984489f84c71bb0438b1433525aa, State: Running, Role: LEADER
I20260709 13:22:32.372816 25178 consensus_queue.cc:237] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.379448 24880 catalog_manager.cc:5697] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa reported cstate change: term changed from 0 to 1, leader changed from <none> to c473984489f84c71bb0438b1433525aa (127.14.23.66). New cstate: current_term: 1 leader_uuid: "c473984489f84c71bb0438b1433525aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:32.391113 25179 raft_consensus.cc:493] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.391638 25179 raft_consensus.cc:515] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.393749 25179 leader_election.cc:290] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d3e51cf592246a4a46c18fe23e74245 (127.14.23.65:37983), c473984489f84c71bb0438b1433525aa (127.14.23.66:34827)
I20260709 13:22:32.405483 25058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334c2584466b48e98903b63551f8caa2" candidate_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c473984489f84c71bb0438b1433525aa" is_pre_election: true
I20260709 13:22:32.405800 24983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334c2584466b48e98903b63551f8caa2" candidate_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d3e51cf592246a4a46c18fe23e74245" is_pre_election: true
I20260709 13:22:32.406328 25058 raft_consensus.cc:2468] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a1d671c7b7842d2acd7153d8c5ac466 in term 0.
I20260709 13:22:32.406481 24983 raft_consensus.cc:2468] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a1d671c7b7842d2acd7153d8c5ac466 in term 0.
I20260709 13:22:32.407641 25096 leader_election.cc:304] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a1d671c7b7842d2acd7153d8c5ac466, c473984489f84c71bb0438b1433525aa; no voters: 
I20260709 13:22:32.408452 25179 raft_consensus.cc:2804] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:32.408774 25179 raft_consensus.cc:493] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:32.409032 25179 raft_consensus.cc:3060] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:32.414350 25179 raft_consensus.cc:515] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.415982 25179 leader_election.cc:290] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [CANDIDATE]: Term 1 election: Requested vote from peers 9d3e51cf592246a4a46c18fe23e74245 (127.14.23.65:37983), c473984489f84c71bb0438b1433525aa (127.14.23.66:34827)
I20260709 13:22:32.416833 24983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334c2584466b48e98903b63551f8caa2" candidate_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d3e51cf592246a4a46c18fe23e74245"
I20260709 13:22:32.417044 25058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334c2584466b48e98903b63551f8caa2" candidate_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c473984489f84c71bb0438b1433525aa"
I20260709 13:22:32.417371 24983 raft_consensus.cc:3060] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:32.417625 25058 raft_consensus.cc:3060] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:32.422626 24983 raft_consensus.cc:2468] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a1d671c7b7842d2acd7153d8c5ac466 in term 1.
I20260709 13:22:32.422792 25058 raft_consensus.cc:2468] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a1d671c7b7842d2acd7153d8c5ac466 in term 1.
I20260709 13:22:32.423827 25095 leader_election.cc:304] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a1d671c7b7842d2acd7153d8c5ac466, 9d3e51cf592246a4a46c18fe23e74245; no voters: 
I20260709 13:22:32.424549 25179 raft_consensus.cc:2804] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:32.425478 25179 raft_consensus.cc:697] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 1 LEADER]: Becoming Leader. State: Replica: 4a1d671c7b7842d2acd7153d8c5ac466, State: Running, Role: LEADER
I20260709 13:22:32.426193 25179 consensus_queue.cc:237] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.433805 24880 catalog_manager.cc:5697] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a1d671c7b7842d2acd7153d8c5ac466 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:32.465183 25058 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b959d6cfc29b40f29b6d6a8752307a9c"
dest_uuid: "c473984489f84c71bb0438b1433525aa"
mode: GRACEFUL
new_leader_uuid: "4a1d671c7b7842d2acd7153d8c5ac466"
 from {username='slave'} at 127.0.0.1:53128
I20260709 13:22:32.465750 25058 raft_consensus.cc:606] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [term 1 LEADER]: Received request to transfer leadership to TS 4a1d671c7b7842d2acd7153d8c5ac466
I20260709 13:22:32.471434 25058 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "729243f232ce416788915c104c990735"
dest_uuid: "c473984489f84c71bb0438b1433525aa"
mode: GRACEFUL
new_leader_uuid: "9d3e51cf592246a4a46c18fe23e74245"
 from {username='slave'} at 127.0.0.1:53128
I20260709 13:22:32.471935 25058 raft_consensus.cc:606] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [term 1 LEADER]: Received request to transfer leadership to TS 9d3e51cf592246a4a46c18fe23e74245
I20260709 13:22:32.484519 25132 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "334c2584466b48e98903b63551f8caa2"
dest_uuid: "4a1d671c7b7842d2acd7153d8c5ac466"
mode: GRACEFUL
new_leader_uuid: "9d3e51cf592246a4a46c18fe23e74245"
 from {username='slave'} at 127.0.0.1:32926
I20260709 13:22:32.485042 25132 raft_consensus.cc:606] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 1 LEADER]: Received request to transfer leadership to TS 9d3e51cf592246a4a46c18fe23e74245
I20260709 13:22:32.523640 25180 consensus_queue.cc:1048] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:32.538836 25187 consensus_queue.cc:1048] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:32.600920 25178 consensus_queue.cc:1048] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:32.608425 25189 raft_consensus.cc:993] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa: : Instructing follower 4a1d671c7b7842d2acd7153d8c5ac466 to start an election
I20260709 13:22:32.609118 25178 raft_consensus.cc:1081] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [term 1 LEADER]: Signalling peer 4a1d671c7b7842d2acd7153d8c5ac466 to start an election
I20260709 13:22:32.611727 25131 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b959d6cfc29b40f29b6d6a8752307a9c"
dest_uuid: "4a1d671c7b7842d2acd7153d8c5ac466"
 from {username='slave'} at 127.0.0.1:32918
I20260709 13:22:32.612360 25131 raft_consensus.cc:493] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:32.612738 25131 raft_consensus.cc:3060] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:32.616331 25178 consensus_queue.cc:1048] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:32.621548 25131 raft_consensus.cc:515] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.622897 25132 raft_consensus.cc:1240] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466 [term 2 FOLLOWER]: Rejecting Update request from peer c473984489f84c71bb0438b1433525aa for earlier term 1. Current term is 2. Ops: []
I20260709 13:22:32.624847 24983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b959d6cfc29b40f29b6d6a8752307a9c" candidate_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d3e51cf592246a4a46c18fe23e74245"
I20260709 13:22:32.625803 25131 leader_election.cc:290] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466 [CANDIDATE]: Term 2 election: Requested vote from peers 9d3e51cf592246a4a46c18fe23e74245 (127.14.23.65:37983), c473984489f84c71bb0438b1433525aa (127.14.23.66:34827)
I20260709 13:22:32.627028 24983 raft_consensus.cc:3060] T b959d6cfc29b40f29b6d6a8752307a9c P 9d3e51cf592246a4a46c18fe23e74245 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:32.627111 25178 consensus_queue.cc:1059] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:32.627946 25058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b959d6cfc29b40f29b6d6a8752307a9c" candidate_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c473984489f84c71bb0438b1433525aa"
I20260709 13:22:32.628829 25058 raft_consensus.cc:3055] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:32.629191 25058 raft_consensus.cc:740] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c473984489f84c71bb0438b1433525aa, State: Running, Role: LEADER
I20260709 13:22:32.630066 25058 consensus_queue.cc:260] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.631431 25058 raft_consensus.cc:3060] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:32.637089 24983 raft_consensus.cc:2468] T b959d6cfc29b40f29b6d6a8752307a9c P 9d3e51cf592246a4a46c18fe23e74245 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a1d671c7b7842d2acd7153d8c5ac466 in term 2.
I20260709 13:22:32.639487 25058 raft_consensus.cc:2468] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a1d671c7b7842d2acd7153d8c5ac466 in term 2.
W20260709 13:22:32.640241 25189 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:22:32.640511 25095 leader_election.cc:304] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a1d671c7b7842d2acd7153d8c5ac466, 9d3e51cf592246a4a46c18fe23e74245; no voters: 
I20260709 13:22:32.642145 25179 raft_consensus.cc:2804] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:32.642663 25179 raft_consensus.cc:697] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466 [term 2 LEADER]: Becoming Leader. State: Replica: 4a1d671c7b7842d2acd7153d8c5ac466, State: Running, Role: LEADER
I20260709 13:22:32.643393 25179 consensus_queue.cc:237] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.651291 24881 catalog_manager.cc:5697] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466 reported cstate change: term changed from 1 to 2, leader changed from c473984489f84c71bb0438b1433525aa (127.14.23.66) to 4a1d671c7b7842d2acd7153d8c5ac466 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:32.827708 25189 consensus_queue.cc:1048] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:32.838512 25189 consensus_queue.cc:1048] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:32.845862 25214 raft_consensus.cc:993] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa: : Instructing follower 9d3e51cf592246a4a46c18fe23e74245 to start an election
I20260709 13:22:32.846295 25189 raft_consensus.cc:1081] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [term 1 LEADER]: Signalling peer 9d3e51cf592246a4a46c18fe23e74245 to start an election
I20260709 13:22:32.847780 24983 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "729243f232ce416788915c104c990735"
dest_uuid: "9d3e51cf592246a4a46c18fe23e74245"
 from {username='slave'} at 127.0.0.1:59500
I20260709 13:22:32.848242 24983 raft_consensus.cc:493] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:32.848529 24983 raft_consensus.cc:3060] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:32.853442 24983 raft_consensus.cc:515] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.855120 24983 leader_election.cc:290] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245 [CANDIDATE]: Term 2 election: Requested vote from peers c473984489f84c71bb0438b1433525aa (127.14.23.66:34827), 4a1d671c7b7842d2acd7153d8c5ac466 (127.14.23.67:38101)
I20260709 13:22:32.856190 25058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "729243f232ce416788915c104c990735" candidate_uuid: "9d3e51cf592246a4a46c18fe23e74245" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c473984489f84c71bb0438b1433525aa"
I20260709 13:22:32.856572 25131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "729243f232ce416788915c104c990735" candidate_uuid: "9d3e51cf592246a4a46c18fe23e74245" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4a1d671c7b7842d2acd7153d8c5ac466"
I20260709 13:22:32.856890 25058 raft_consensus.cc:3055] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:32.857177 25131 raft_consensus.cc:3060] T 729243f232ce416788915c104c990735 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:32.857237 25058 raft_consensus.cc:740] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c473984489f84c71bb0438b1433525aa, State: Running, Role: LEADER
I20260709 13:22:32.858196 25058 consensus_queue.cc:260] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.859480 25058 raft_consensus.cc:3060] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:32.864665 25131 raft_consensus.cc:2468] T 729243f232ce416788915c104c990735 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d3e51cf592246a4a46c18fe23e74245 in term 2.
I20260709 13:22:32.866158 24947 leader_election.cc:304] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a1d671c7b7842d2acd7153d8c5ac466, 9d3e51cf592246a4a46c18fe23e74245; no voters: 
I20260709 13:22:32.866393 25058 raft_consensus.cc:2468] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d3e51cf592246a4a46c18fe23e74245 in term 2.
I20260709 13:22:32.866922 25180 raft_consensus.cc:2804] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:32.867405 25180 raft_consensus.cc:697] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245 [term 2 LEADER]: Becoming Leader. State: Replica: 9d3e51cf592246a4a46c18fe23e74245, State: Running, Role: LEADER
I20260709 13:22:32.868127 25180 consensus_queue.cc:237] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.873852 24880 catalog_manager.cc:5697] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245 reported cstate change: term changed from 1 to 2, leader changed from c473984489f84c71bb0438b1433525aa (127.14.23.66) to 9d3e51cf592246a4a46c18fe23e74245 (127.14.23.65). New cstate: current_term: 2 leader_uuid: "9d3e51cf592246a4a46c18fe23e74245" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:32.880338 25179 consensus_queue.cc:1048] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:32.887279 25205 raft_consensus.cc:993] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466: : Instructing follower 9d3e51cf592246a4a46c18fe23e74245 to start an election
I20260709 13:22:32.887950 25179 raft_consensus.cc:1081] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 1 LEADER]: Signalling peer 9d3e51cf592246a4a46c18fe23e74245 to start an election
I20260709 13:22:32.889741 24982 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "334c2584466b48e98903b63551f8caa2"
dest_uuid: "9d3e51cf592246a4a46c18fe23e74245"
 from {username='slave'} at 127.0.0.1:59502
I20260709 13:22:32.890410 24982 raft_consensus.cc:493] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:32.890801 24982 raft_consensus.cc:3060] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:32.893458 25219 consensus_queue.cc:1048] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:32.898159 24982 raft_consensus.cc:515] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.899389 24983 raft_consensus.cc:1240] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [term 2 FOLLOWER]: Rejecting Update request from peer 4a1d671c7b7842d2acd7153d8c5ac466 for earlier term 1. Current term is 2. Ops: []
I20260709 13:22:32.901080 24982 leader_election.cc:290] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [CANDIDATE]: Term 2 election: Requested vote from peers c473984489f84c71bb0438b1433525aa (127.14.23.66:34827), 4a1d671c7b7842d2acd7153d8c5ac466 (127.14.23.67:38101)
I20260709 13:22:32.901403 25057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334c2584466b48e98903b63551f8caa2" candidate_uuid: "9d3e51cf592246a4a46c18fe23e74245" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c473984489f84c71bb0438b1433525aa"
I20260709 13:22:32.902864 25219 consensus_queue.cc:1059] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:32.902830 25131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334c2584466b48e98903b63551f8caa2" candidate_uuid: "9d3e51cf592246a4a46c18fe23e74245" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4a1d671c7b7842d2acd7153d8c5ac466"
I20260709 13:22:32.904752 25205 raft_consensus.cc:3055] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:32.905109 25205 raft_consensus.cc:740] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4a1d671c7b7842d2acd7153d8c5ac466, State: Running, Role: LEADER
I20260709 13:22:32.905881 25205 consensus_queue.cc:260] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.907174 25205 raft_consensus.cc:3060] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:32.917667 25131 raft_consensus.cc:2468] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d3e51cf592246a4a46c18fe23e74245 in term 2.
I20260709 13:22:32.919857 24947 leader_election.cc:304] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4a1d671c7b7842d2acd7153d8c5ac466, 9d3e51cf592246a4a46c18fe23e74245; no voters: c473984489f84c71bb0438b1433525aa
I20260709 13:22:32.920605 25180 raft_consensus.cc:2804] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:32.921044 25180 raft_consensus.cc:697] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [term 2 LEADER]: Becoming Leader. State: Replica: 9d3e51cf592246a4a46c18fe23e74245, State: Running, Role: LEADER
I20260709 13:22:32.921826 25180 consensus_queue.cc:237] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:32.928051 24880 catalog_manager.cc:5697] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 reported cstate change: term changed from 1 to 2, leader changed from 4a1d671c7b7842d2acd7153d8c5ac466 (127.14.23.67) to 9d3e51cf592246a4a46c18fe23e74245 (127.14.23.65). New cstate: current_term: 2 leader_uuid: "9d3e51cf592246a4a46c18fe23e74245" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:32.943455 14429 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:22:32.945036 14429 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:22:32.945420 14429 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:22:32.957046 24982 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "334c2584466b48e98903b63551f8caa2"
dest_uuid: "9d3e51cf592246a4a46c18fe23e74245"
mode: GRACEFUL
new_leader_uuid: "c473984489f84c71bb0438b1433525aa"
 from {username='slave'} at 127.0.0.1:59514
I20260709 13:22:32.957592 24982 raft_consensus.cc:606] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [term 2 LEADER]: Received request to transfer leadership to TS c473984489f84c71bb0438b1433525aa
I20260709 13:22:32.958813 14429 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:22:33.101116 24982 raft_consensus.cc:1275] T b959d6cfc29b40f29b6d6a8752307a9c P 9d3e51cf592246a4a46c18fe23e74245 [term 2 FOLLOWER]: Refusing update from remote peer 4a1d671c7b7842d2acd7153d8c5ac466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:33.102506 25205 consensus_queue.cc:1048] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:33.111513 25057 raft_consensus.cc:1275] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [term 2 FOLLOWER]: Refusing update from remote peer 4a1d671c7b7842d2acd7153d8c5ac466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:33.113176 25205 consensus_queue.cc:1048] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:33.327538 25131 raft_consensus.cc:1275] T 729243f232ce416788915c104c990735 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 2 FOLLOWER]: Refusing update from remote peer 9d3e51cf592246a4a46c18fe23e74245: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:33.328044 25057 raft_consensus.cc:3060] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:33.328979 25223 consensus_queue.cc:1048] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:33.336009 25057 raft_consensus.cc:1275] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa [term 2 FOLLOWER]: Refusing update from remote peer 9d3e51cf592246a4a46c18fe23e74245: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:33.336953 25058 raft_consensus.cc:1275] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [term 2 FOLLOWER]: Refusing update from remote peer 9d3e51cf592246a4a46c18fe23e74245: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:33.338668 25180 consensus_queue.cc:1048] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:33.339015 25187 consensus_queue.cc:1048] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:33.345950 25180 raft_consensus.cc:993] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245: : Instructing follower c473984489f84c71bb0438b1433525aa to start an election
I20260709 13:22:33.346419 25180 raft_consensus.cc:1081] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [term 2 LEADER]: Signalling peer c473984489f84c71bb0438b1433525aa to start an election
I20260709 13:22:33.349303 25057 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "334c2584466b48e98903b63551f8caa2"
dest_uuid: "c473984489f84c71bb0438b1433525aa"
 from {username='slave'} at 127.0.0.1:53112
I20260709 13:22:33.351565 25057 raft_consensus.cc:493] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:33.352087 25057 raft_consensus.cc:3060] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:33.352896 25131 raft_consensus.cc:1275] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 2 FOLLOWER]: Refusing update from remote peer 9d3e51cf592246a4a46c18fe23e74245: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:33.354377 25187 consensus_queue.cc:1048] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:33.359436 25057 raft_consensus.cc:515] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:33.366420 24982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334c2584466b48e98903b63551f8caa2" candidate_uuid: "c473984489f84c71bb0438b1433525aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9d3e51cf592246a4a46c18fe23e74245"
I20260709 13:22:33.366891 25131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "334c2584466b48e98903b63551f8caa2" candidate_uuid: "c473984489f84c71bb0438b1433525aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4a1d671c7b7842d2acd7153d8c5ac466"
I20260709 13:22:33.367074 24982 raft_consensus.cc:3055] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:33.367527 25131 raft_consensus.cc:3060] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:33.367503 24982 raft_consensus.cc:740] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9d3e51cf592246a4a46c18fe23e74245, State: Running, Role: LEADER
I20260709 13:22:33.368440 24982 consensus_queue.cc:260] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:33.369233 24982 raft_consensus.cc:3060] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:33.372959 25057 leader_election.cc:290] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa [CANDIDATE]: Term 3 election: Requested vote from peers 9d3e51cf592246a4a46c18fe23e74245 (127.14.23.65:37983), 4a1d671c7b7842d2acd7153d8c5ac466 (127.14.23.67:38101)
I20260709 13:22:33.373224 25131 raft_consensus.cc:2468] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c473984489f84c71bb0438b1433525aa in term 3.
I20260709 13:22:33.374682 24982 raft_consensus.cc:2468] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c473984489f84c71bb0438b1433525aa in term 3.
I20260709 13:22:33.374924 25022 leader_election.cc:304] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a1d671c7b7842d2acd7153d8c5ac466, c473984489f84c71bb0438b1433525aa; no voters: 
I20260709 13:22:33.375741 25214 raft_consensus.cc:2804] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:33.376196 25214 raft_consensus.cc:697] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa [term 3 LEADER]: Becoming Leader. State: Replica: c473984489f84c71bb0438b1433525aa, State: Running, Role: LEADER
I20260709 13:22:33.376890 25214 consensus_queue.cc:237] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } }
I20260709 13:22:33.383104 24879 catalog_manager.cc:5697] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa reported cstate change: term changed from 2 to 3, leader changed from 9d3e51cf592246a4a46c18fe23e74245 (127.14.23.65) to c473984489f84c71bb0438b1433525aa (127.14.23.66). New cstate: current_term: 3 leader_uuid: "c473984489f84c71bb0438b1433525aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3e51cf592246a4a46c18fe23e74245" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 37983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c473984489f84c71bb0438b1433525aa" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 34827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a1d671c7b7842d2acd7153d8c5ac466" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38101 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:33.571375 14429 tablet_server.cc:179] TabletServer@127.14.23.65:0 shutting down...
I20260709 13:22:33.590916 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:33.591571 14429 tablet_replica.cc:333] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245: stopping tablet replica
I20260709 13:22:33.592124 14429 raft_consensus.cc:2243] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:33.592581 14429 raft_consensus.cc:2272] T 334c2584466b48e98903b63551f8caa2 P 9d3e51cf592246a4a46c18fe23e74245 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:33.595389 14429 tablet_replica.cc:333] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245: stopping tablet replica
I20260709 13:22:33.596107 14429 raft_consensus.cc:2243] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:33.597039 14429 raft_consensus.cc:2272] T 729243f232ce416788915c104c990735 P 9d3e51cf592246a4a46c18fe23e74245 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:33.599453 14429 tablet_replica.cc:333] T b959d6cfc29b40f29b6d6a8752307a9c P 9d3e51cf592246a4a46c18fe23e74245: stopping tablet replica
I20260709 13:22:33.599958 14429 raft_consensus.cc:2243] T b959d6cfc29b40f29b6d6a8752307a9c P 9d3e51cf592246a4a46c18fe23e74245 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:33.600344 14429 raft_consensus.cc:2272] T b959d6cfc29b40f29b6d6a8752307a9c P 9d3e51cf592246a4a46c18fe23e74245 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:33.602512 14429 tablet_replica.cc:333] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245: stopping tablet replica
I20260709 13:22:33.602967 14429 raft_consensus.cc:2243] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:33.603695 14429 raft_consensus.cc:2272] T fab8fd8145934e1886fcdcefecf7f1af P 9d3e51cf592246a4a46c18fe23e74245 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:33.626317 14429 tablet_server.cc:196] TabletServer@127.14.23.65:0 shutdown complete.
I20260709 13:22:33.638489 14429 tablet_server.cc:179] TabletServer@127.14.23.66:0 shutting down...
I20260709 13:22:33.656209 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:33.656857 14429 tablet_replica.cc:333] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa: stopping tablet replica
I20260709 13:22:33.657407 14429 raft_consensus.cc:2243] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:33.658087 14429 pending_rounds.cc:70] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa: Trying to abort 1 pending ops.
I20260709 13:22:33.658286 14429 pending_rounds.cc:77] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305639335439216640 op_type: NO_OP noop_request { }
I20260709 13:22:33.658566 14429 raft_consensus.cc:2889] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:33.658849 14429 raft_consensus.cc:2272] T 334c2584466b48e98903b63551f8caa2 P c473984489f84c71bb0438b1433525aa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:33.661077 14429 tablet_replica.cc:333] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa: stopping tablet replica
I20260709 13:22:33.661600 14429 raft_consensus.cc:2243] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:33.662106 14429 raft_consensus.cc:2272] T 729243f232ce416788915c104c990735 P c473984489f84c71bb0438b1433525aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:33.664350 14429 tablet_replica.cc:333] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa: stopping tablet replica
I20260709 13:22:33.664875 14429 raft_consensus.cc:2243] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:33.665302 14429 raft_consensus.cc:2272] T b959d6cfc29b40f29b6d6a8752307a9c P c473984489f84c71bb0438b1433525aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:33.667423 14429 tablet_replica.cc:333] T fab8fd8145934e1886fcdcefecf7f1af P c473984489f84c71bb0438b1433525aa: stopping tablet replica
I20260709 13:22:33.667898 14429 raft_consensus.cc:2243] T fab8fd8145934e1886fcdcefecf7f1af P c473984489f84c71bb0438b1433525aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:33.668285 14429 raft_consensus.cc:2272] T fab8fd8145934e1886fcdcefecf7f1af P c473984489f84c71bb0438b1433525aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:33.689893 14429 tablet_server.cc:196] TabletServer@127.14.23.66:0 shutdown complete.
I20260709 13:22:33.704174 14429 tablet_server.cc:179] TabletServer@127.14.23.67:0 shutting down...
I20260709 13:22:33.722746 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:33.723354 14429 tablet_replica.cc:333] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466: stopping tablet replica
I20260709 13:22:33.723875 14429 raft_consensus.cc:2243] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:33.724334 14429 raft_consensus.cc:2272] T 334c2584466b48e98903b63551f8caa2 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:33.726673 14429 tablet_replica.cc:333] T 729243f232ce416788915c104c990735 P 4a1d671c7b7842d2acd7153d8c5ac466: stopping tablet replica
I20260709 13:22:33.727170 14429 raft_consensus.cc:2243] T 729243f232ce416788915c104c990735 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:33.727599 14429 raft_consensus.cc:2272] T 729243f232ce416788915c104c990735 P 4a1d671c7b7842d2acd7153d8c5ac466 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:33.729766 14429 tablet_replica.cc:333] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466: stopping tablet replica
I20260709 13:22:33.730338 14429 raft_consensus.cc:2243] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:33.731148 14429 raft_consensus.cc:2272] T b959d6cfc29b40f29b6d6a8752307a9c P 4a1d671c7b7842d2acd7153d8c5ac466 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:33.733371 14429 tablet_replica.cc:333] T fab8fd8145934e1886fcdcefecf7f1af P 4a1d671c7b7842d2acd7153d8c5ac466: stopping tablet replica
I20260709 13:22:33.733839 14429 raft_consensus.cc:2243] T fab8fd8145934e1886fcdcefecf7f1af P 4a1d671c7b7842d2acd7153d8c5ac466 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:33.734398 14429 raft_consensus.cc:2272] T fab8fd8145934e1886fcdcefecf7f1af P 4a1d671c7b7842d2acd7153d8c5ac466 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:33.747009 14429 tablet_server.cc:196] TabletServer@127.14.23.67:0 shutdown complete.
I20260709 13:22:33.758702 14429 master.cc:562] Master@127.14.23.126:38457 shutting down...
I20260709 13:22:33.772974 14429 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:33.773517 14429 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:33.773828 14429 tablet_replica.cc:333] T 00000000000000000000000000000000 P 35aaaee4d4404c66911ad50eb8b84a6c: stopping tablet replica
I20260709 13:22:33.793269 14429 master.cc:584] Master@127.14.23.126:38457 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2767 ms)
[----------] 6 tests from LeaderRebalancerTest (298096 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:22:33.823856 14429 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.23.126:39227
I20260709 13:22:33.824946 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:33.831591 25225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:33.832649 25224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:33.834724 25227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:33.837101 14429 server_base.cc:1061] running on GCE node
I20260709 13:22:33.838007 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:33.838219 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:33.838357 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603353838340 us; error 0 us; skew 500 ppm
I20260709 13:22:33.838868 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:33.841599 14429 webserver.cc:533] Webserver started at http://127.14.23.126:46259/ using document root <none> and password file <none>
I20260709 13:22:33.842104 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:33.842278 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:33.842486 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:33.843525 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603055563165-14429-0/minicluster-data/master-0-root/instance:
uuid: "64cc9b71002c4a708e70fdba2f0dfb4f"
format_stamp: "Formatted at 2026-07-09 13:22:33 on dist-test-slave-twc1"
I20260709 13:22:33.848062 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:22:33.851701 25232 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:33.852465 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:22:33.852787 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603055563165-14429-0/minicluster-data/master-0-root
uuid: "64cc9b71002c4a708e70fdba2f0dfb4f"
format_stamp: "Formatted at 2026-07-09 13:22:33 on dist-test-slave-twc1"
I20260709 13:22:33.853072 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603055563165-14429-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603055563165-14429-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603055563165-14429-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:33.869836 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:33.871244 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:33.908885 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.126:39227
I20260709 13:22:33.908942 25283 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.126:39227 every 8 connection(s)
I20260709 13:22:33.912690 25284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:33.923732 25284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f: Bootstrap starting.
I20260709 13:22:33.928208 25284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:33.932304 25284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f: No bootstrap required, opened a new log
I20260709 13:22:33.934439 25284 raft_consensus.cc:359] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64cc9b71002c4a708e70fdba2f0dfb4f" member_type: VOTER }
I20260709 13:22:33.934825 25284 raft_consensus.cc:385] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:33.935001 25284 raft_consensus.cc:740] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64cc9b71002c4a708e70fdba2f0dfb4f, State: Initialized, Role: FOLLOWER
I20260709 13:22:33.935492 25284 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64cc9b71002c4a708e70fdba2f0dfb4f" member_type: VOTER }
I20260709 13:22:33.935918 25284 raft_consensus.cc:399] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:22:33.936103 25284 raft_consensus.cc:493] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:22:33.936323 25284 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:33.941385 25284 raft_consensus.cc:515] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64cc9b71002c4a708e70fdba2f0dfb4f" member_type: VOTER }
I20260709 13:22:33.941900 25284 leader_election.cc:304] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64cc9b71002c4a708e70fdba2f0dfb4f; no voters: 
I20260709 13:22:33.943068 25284 leader_election.cc:290] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:22:33.943394 25287 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:33.944792 25287 raft_consensus.cc:697] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f [term 1 LEADER]: Becoming Leader. State: Replica: 64cc9b71002c4a708e70fdba2f0dfb4f, State: Running, Role: LEADER
I20260709 13:22:33.945495 25287 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64cc9b71002c4a708e70fdba2f0dfb4f" member_type: VOTER }
I20260709 13:22:33.945883 25284 sys_catalog.cc:565] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:22:33.948204 25289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64cc9b71002c4a708e70fdba2f0dfb4f. Latest consensus state: current_term: 1 leader_uuid: "64cc9b71002c4a708e70fdba2f0dfb4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64cc9b71002c4a708e70fdba2f0dfb4f" member_type: VOTER } }
I20260709 13:22:33.948146 25288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64cc9b71002c4a708e70fdba2f0dfb4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64cc9b71002c4a708e70fdba2f0dfb4f" member_type: VOTER } }
I20260709 13:22:33.948935 25289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:33.949064 25288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:33.953289 25292 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:22:33.960233 25292 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:22:33.961303 14429 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:22:33.969100 25292 catalog_manager.cc:1383] Generated new cluster ID: f162911d9ab44d5c8171bd260f652234
I20260709 13:22:33.969411 25292 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:22:34.001566 25292 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:22:34.002885 25292 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:22:34.015410 25292 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f: Generated new TSK 0
I20260709 13:22:34.015991 25292 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:22:34.028250 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:34.034039 25305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:34.035207 25306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:34.036986 14429 server_base.cc:1061] running on GCE node
W20260709 13:22:34.036922 25308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:34.038142 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:34.038350 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:34.038472 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603354038461 us; error 0 us; skew 500 ppm
I20260709 13:22:34.038959 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:34.041313 14429 webserver.cc:533] Webserver started at http://127.14.23.65:36599/ using document root <none> and password file <none>
I20260709 13:22:34.041774 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:34.041960 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:34.042225 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:34.043267 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603055563165-14429-0/minicluster-data/ts-0-root/instance:
uuid: "01445466116342929df00ee716f54e3f"
format_stamp: "Formatted at 2026-07-09 13:22:34 on dist-test-slave-twc1"
I20260709 13:22:34.047750 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:22:34.051133 25313 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:34.051852 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:22:34.052134 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603055563165-14429-0/minicluster-data/ts-0-root
uuid: "01445466116342929df00ee716f54e3f"
format_stamp: "Formatted at 2026-07-09 13:22:34 on dist-test-slave-twc1"
I20260709 13:22:34.052407 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603055563165-14429-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603055563165-14429-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603055563165-14429-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:34.072490 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:34.073637 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:34.075181 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:34.077505 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:34.077728 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:34.077975 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:34.078125 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:34.115355 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.65:43051
I20260709 13:22:34.115455 25375 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.65:43051 every 8 connection(s)
I20260709 13:22:34.120088 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:34.127357 25380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:34.129374 25381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:34.130492 14429 server_base.cc:1061] running on GCE node
W20260709 13:22:34.131825 25383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:34.132977 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:34.133204 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:34.133381 25376 heartbeater.cc:344] Connected to a master server at 127.14.23.126:39227
I20260709 13:22:34.133384 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603354133364 us; error 0 us; skew 500 ppm
I20260709 13:22:34.133992 25376 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:34.134466 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:34.134950 25376 heartbeater.cc:507] Master 127.14.23.126:39227 requested a full tablet report, sending...
I20260709 13:22:34.137244 25249 ts_manager.cc:194] Registered new tserver with Master: 01445466116342929df00ee716f54e3f (127.14.23.65:43051)
I20260709 13:22:34.139007 25249 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33584
I20260709 13:22:34.140401 14429 webserver.cc:533] Webserver started at http://127.14.23.66:44409/ using document root <none> and password file <none>
I20260709 13:22:34.141000 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:34.141181 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:34.141489 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:34.142611 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603055563165-14429-0/minicluster-data/ts-1-root/instance:
uuid: "1c893d888632407db50d5d53a2f4edb6"
format_stamp: "Formatted at 2026-07-09 13:22:34 on dist-test-slave-twc1"
I20260709 13:22:34.147114 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:22:34.150564 25388 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:34.151309 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:22:34.151600 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603055563165-14429-0/minicluster-data/ts-1-root
uuid: "1c893d888632407db50d5d53a2f4edb6"
format_stamp: "Formatted at 2026-07-09 13:22:34 on dist-test-slave-twc1"
I20260709 13:22:34.151880 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603055563165-14429-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603055563165-14429-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603055563165-14429-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:34.162933 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:34.164016 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:34.165480 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:34.167852 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:34.168063 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:34.168294 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:34.168443 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:34.206207 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.66:46657
I20260709 13:22:34.206329 25450 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.66:46657 every 8 connection(s)
I20260709 13:22:34.211076 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:34.218678 25454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:34.222481 25455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:34.225020 25457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:34.225696 14429 server_base.cc:1061] running on GCE node
I20260709 13:22:34.226361 25451 heartbeater.cc:344] Connected to a master server at 127.14.23.126:39227
I20260709 13:22:34.226737 25451 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:34.226827 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:34.227142 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:34.227315 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603354227298 us; error 0 us; skew 500 ppm
I20260709 13:22:34.227412 25451 heartbeater.cc:507] Master 127.14.23.126:39227 requested a full tablet report, sending...
I20260709 13:22:34.227948 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:34.229753 25249 ts_manager.cc:194] Registered new tserver with Master: 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657)
I20260709 13:22:34.230857 14429 webserver.cc:533] Webserver started at http://127.14.23.67:40123/ using document root <none> and password file <none>
I20260709 13:22:34.231570 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:34.231694 25249 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33588
I20260709 13:22:34.231837 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:34.232321 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:34.233510 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603055563165-14429-0/minicluster-data/ts-2-root/instance:
uuid: "a724fa0654fa4809b29444f707a88767"
format_stamp: "Formatted at 2026-07-09 13:22:34 on dist-test-slave-twc1"
I20260709 13:22:34.238049 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.001s
I20260709 13:22:34.241412 25462 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:34.242251 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:22:34.242559 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603055563165-14429-0/minicluster-data/ts-2-root
uuid: "a724fa0654fa4809b29444f707a88767"
format_stamp: "Formatted at 2026-07-09 13:22:34 on dist-test-slave-twc1"
I20260709 13:22:34.242851 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603055563165-14429-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603055563165-14429-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603055563165-14429-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:34.260761 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:34.261883 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:34.263356 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:34.265606 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:34.265820 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:34.266088 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:34.266247 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:34.303325 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.67:42881
I20260709 13:22:34.303460 25524 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.67:42881 every 8 connection(s)
I20260709 13:22:34.316408 25525 heartbeater.cc:344] Connected to a master server at 127.14.23.126:39227
I20260709 13:22:34.316867 25525 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:34.317557 25525 heartbeater.cc:507] Master 127.14.23.126:39227 requested a full tablet report, sending...
I20260709 13:22:34.319515 25249 ts_manager.cc:194] Registered new tserver with Master: a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:34.319995 14429 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013539418s
I20260709 13:22:34.321276 25249 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33592
I20260709 13:22:34.345311 25249 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33596:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:22:34.347980 25249 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:22:34.423506 25416 tablet_service.cc:1511] Processing CreateTablet for tablet 01c94152f0c44421b2c2e3560785b3e6 (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:34.424957 25416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01c94152f0c44421b2c2e3560785b3e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.430227 25415 tablet_service.cc:1511] Processing CreateTablet for tablet 30912c2e960f4e5c85dfe927e98a1e9f (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:34.431726 25415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30912c2e960f4e5c85dfe927e98a1e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.432610 25490 tablet_service.cc:1511] Processing CreateTablet for tablet 01c94152f0c44421b2c2e3560785b3e6 (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:34.433336 25489 tablet_service.cc:1511] Processing CreateTablet for tablet 30912c2e960f4e5c85dfe927e98a1e9f (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:34.434051 25490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01c94152f0c44421b2c2e3560785b3e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.434738 25489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30912c2e960f4e5c85dfe927e98a1e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.434991 25414 tablet_service.cc:1511] Processing CreateTablet for tablet 8b34ba27e9a24cab9530218fa592c6ae (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:34.436380 25414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b34ba27e9a24cab9530218fa592c6ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.438998 25488 tablet_service.cc:1511] Processing CreateTablet for tablet 8b34ba27e9a24cab9530218fa592c6ae (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:34.440470 25488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b34ba27e9a24cab9530218fa592c6ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.443044 25487 tablet_service.cc:1511] Processing CreateTablet for tablet 5fead9ecfeb04d0d94509484c7234118 (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:34.444446 25487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fead9ecfeb04d0d94509484c7234118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.447031 25412 tablet_service.cc:1511] Processing CreateTablet for tablet efc2e95552a142ea8f26983d2b49e0c5 (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:34.448051 25341 tablet_service.cc:1511] Processing CreateTablet for tablet 01c94152f0c44421b2c2e3560785b3e6 (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:34.448498 25412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efc2e95552a142ea8f26983d2b49e0c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.449406 25341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01c94152f0c44421b2c2e3560785b3e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.451012 25486 tablet_service.cc:1511] Processing CreateTablet for tablet efc2e95552a142ea8f26983d2b49e0c5 (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:34.452085 25339 tablet_service.cc:1511] Processing CreateTablet for tablet 8b34ba27e9a24cab9530218fa592c6ae (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:34.453465 25339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b34ba27e9a24cab9530218fa592c6ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.453794 25485 tablet_service.cc:1511] Processing CreateTablet for tablet 1cefd5d18235480d9e47c7bf0377e4f0 (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:34.454999 25338 tablet_service.cc:1511] Processing CreateTablet for tablet 5fead9ecfeb04d0d94509484c7234118 (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:34.456349 25338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fead9ecfeb04d0d94509484c7234118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.456037 25411 tablet_service.cc:1511] Processing CreateTablet for tablet 1cefd5d18235480d9e47c7bf0377e4f0 (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:34.457361 25411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cefd5d18235480d9e47c7bf0377e4f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.459004 25337 tablet_service.cc:1511] Processing CreateTablet for tablet efc2e95552a142ea8f26983d2b49e0c5 (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:34.459827 25413 tablet_service.cc:1511] Processing CreateTablet for tablet 5fead9ecfeb04d0d94509484c7234118 (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:34.460362 25337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efc2e95552a142ea8f26983d2b49e0c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.461191 25413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fead9ecfeb04d0d94509484c7234118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.451012 25340 tablet_service.cc:1511] Processing CreateTablet for tablet 30912c2e960f4e5c85dfe927e98a1e9f (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:34.452513 25486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efc2e95552a142ea8f26983d2b49e0c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.470446 25485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cefd5d18235480d9e47c7bf0377e4f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.479771 25540 tablet_bootstrap.cc:492] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6: Bootstrap starting.
I20260709 13:22:34.480427 25340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30912c2e960f4e5c85dfe927e98a1e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.485134 25336 tablet_service.cc:1511] Processing CreateTablet for tablet 1cefd5d18235480d9e47c7bf0377e4f0 (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:34.486677 25336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cefd5d18235480d9e47c7bf0377e4f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.488569 25540 tablet_bootstrap.cc:654] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.509053 25335 tablet_service.cc:1511] Processing CreateTablet for tablet 0768383f820c4e4cb8d8595901456ae1 (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:34.510815 25335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0768383f820c4e4cb8d8595901456ae1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.517141 25490 tablet_service.cc:1511] Processing CreateTablet for tablet 0768383f820c4e4cb8d8595901456ae1 (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:34.518674 25490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0768383f820c4e4cb8d8595901456ae1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:22:34.547004 25233 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.14.23.66:46657, user_credentials={real_user=slave}} blocked reactor thread for 63179.1us
I20260709 13:22:34.554185 25416 tablet_service.cc:1511] Processing CreateTablet for tablet 0768383f820c4e4cb8d8595901456ae1 (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:34.555641 25416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0768383f820c4e4cb8d8595901456ae1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.559245 25489 tablet_service.cc:1511] Processing CreateTablet for tablet 6729a5f0b78e4d148483f7b5d427304a (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:34.560782 25489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6729a5f0b78e4d148483f7b5d427304a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.561710 25334 tablet_service.cc:1511] Processing CreateTablet for tablet 28a64081a0c94a92a11015ec646adf0c (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:34.563103 25334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28a64081a0c94a92a11015ec646adf0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.565501 25543 tablet_bootstrap.cc:492] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f: Bootstrap starting.
I20260709 13:22:34.568734 25540 tablet_bootstrap.cc:492] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6: No bootstrap required, opened a new log
I20260709 13:22:34.568959 25410 tablet_service.cc:1511] Processing CreateTablet for tablet 6729a5f0b78e4d148483f7b5d427304a (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:34.570474 25410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6729a5f0b78e4d148483f7b5d427304a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.580209 25409 tablet_service.cc:1511] Processing CreateTablet for tablet 28a64081a0c94a92a11015ec646adf0c (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:34.582762 25409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28a64081a0c94a92a11015ec646adf0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.585994 25540 ts_tablet_manager.cc:1403] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6: Time spent bootstrapping tablet: real 0.107s	user 0.013s	sys 0.054s
I20260709 13:22:34.589278 25541 tablet_bootstrap.cc:492] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767: Bootstrap starting.
I20260709 13:22:34.589008 25540 raft_consensus.cc:359] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } } peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } }
I20260709 13:22:34.594977 25335 tablet_service.cc:1511] Processing CreateTablet for tablet 6729a5f0b78e4d148483f7b5d427304a (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:34.596500 25335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6729a5f0b78e4d148483f7b5d427304a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.599675 25543 tablet_bootstrap.cc:654] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.601794 25485 tablet_service.cc:1511] Processing CreateTablet for tablet 28a64081a0c94a92a11015ec646adf0c (DEFAULT_TABLE table=test-workload [id=3e352943450d47979bdd5d05eef69295]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:34.614853 25485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28a64081a0c94a92a11015ec646adf0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.589769 25540 raft_consensus.cc:385] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.616910 25540 raft_consensus.cc:740] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.617890 25540 consensus_queue.cc:260] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } } peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } }
I20260709 13:22:34.622545 25541 tablet_bootstrap.cc:654] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.629478 25451 heartbeater.cc:499] Master 127.14.23.126:39227 was elected leader, sending a full tablet report...
I20260709 13:22:34.628319 25540 ts_tablet_manager.cc:1434] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.000s
I20260709 13:22:34.632342 25540 tablet_bootstrap.cc:492] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6: Bootstrap starting.
I20260709 13:22:34.655689 25541 tablet_bootstrap.cc:492] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767: No bootstrap required, opened a new log
I20260709 13:22:34.656270 25541 ts_tablet_manager.cc:1403] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767: Time spent bootstrapping tablet: real 0.067s	user 0.008s	sys 0.014s
I20260709 13:22:34.661314 25541 raft_consensus.cc:359] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.662109 25540 tablet_bootstrap.cc:654] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.662073 25541 raft_consensus.cc:385] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.662721 25541 raft_consensus.cc:740] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a724fa0654fa4809b29444f707a88767, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.664631 25543 tablet_bootstrap.cc:492] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f: No bootstrap required, opened a new log
I20260709 13:22:34.667105 25543 ts_tablet_manager.cc:1403] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f: Time spent bootstrapping tablet: real 0.118s	user 0.012s	sys 0.018s
I20260709 13:22:34.663827 25541 consensus_queue.cc:260] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.670074 25543 raft_consensus.cc:359] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.670807 25543 raft_consensus.cc:385] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.672744 25543 raft_consensus.cc:740] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.673724 25543 consensus_queue.cc:260] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.674379 25540 tablet_bootstrap.cc:492] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6: No bootstrap required, opened a new log
I20260709 13:22:34.674981 25540 ts_tablet_manager.cc:1403] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6: Time spent bootstrapping tablet: real 0.043s	user 0.004s	sys 0.013s
I20260709 13:22:34.675393 25541 ts_tablet_manager.cc:1434] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.008s
I20260709 13:22:34.676254 25376 heartbeater.cc:499] Master 127.14.23.126:39227 was elected leader, sending a full tablet report...
I20260709 13:22:34.676676 25543 ts_tablet_manager.cc:1434] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:22:34.677240 25525 heartbeater.cc:499] Master 127.14.23.126:39227 was elected leader, sending a full tablet report...
I20260709 13:22:34.677855 25543 tablet_bootstrap.cc:492] T efc2e95552a142ea8f26983d2b49e0c5 P 01445466116342929df00ee716f54e3f: Bootstrap starting.
I20260709 13:22:34.679180 25541 tablet_bootstrap.cc:492] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767: Bootstrap starting.
I20260709 13:22:34.677829 25540 raft_consensus.cc:359] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.684736 25540 raft_consensus.cc:385] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.685261 25540 raft_consensus.cc:740] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.684479 25543 tablet_bootstrap.cc:654] T efc2e95552a142ea8f26983d2b49e0c5 P 01445466116342929df00ee716f54e3f: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.686324 25540 consensus_queue.cc:260] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.685403 25541 tablet_bootstrap.cc:654] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.690824 25540 ts_tablet_manager.cc:1434] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:22:34.691941 25540 tablet_bootstrap.cc:492] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6: Bootstrap starting.
I20260709 13:22:34.692539 25543 tablet_bootstrap.cc:492] T efc2e95552a142ea8f26983d2b49e0c5 P 01445466116342929df00ee716f54e3f: No bootstrap required, opened a new log
I20260709 13:22:34.693195 25543 ts_tablet_manager.cc:1403] T efc2e95552a142ea8f26983d2b49e0c5 P 01445466116342929df00ee716f54e3f: Time spent bootstrapping tablet: real 0.016s	user 0.005s	sys 0.008s
I20260709 13:22:34.699472 25540 tablet_bootstrap.cc:654] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.701819 25543 raft_consensus.cc:359] T efc2e95552a142ea8f26983d2b49e0c5 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.702667 25543 raft_consensus.cc:385] T efc2e95552a142ea8f26983d2b49e0c5 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.702986 25543 raft_consensus.cc:740] T efc2e95552a142ea8f26983d2b49e0c5 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.703532 25541 tablet_bootstrap.cc:492] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767: No bootstrap required, opened a new log
I20260709 13:22:34.704092 25541 ts_tablet_manager.cc:1403] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767: Time spent bootstrapping tablet: real 0.025s	user 0.006s	sys 0.011s
I20260709 13:22:34.703768 25543 consensus_queue.cc:260] T efc2e95552a142ea8f26983d2b49e0c5 P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.706418 25543 ts_tablet_manager.cc:1434] T efc2e95552a142ea8f26983d2b49e0c5 P 01445466116342929df00ee716f54e3f: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:22:34.706418 25545 raft_consensus.cc:493] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:34.707047 25545 raft_consensus.cc:515] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } } peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } }
I20260709 13:22:34.707530 25543 tablet_bootstrap.cc:492] T 1cefd5d18235480d9e47c7bf0377e4f0 P 01445466116342929df00ee716f54e3f: Bootstrap starting.
I20260709 13:22:34.707176 25541 raft_consensus.cc:359] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } } peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } }
I20260709 13:22:34.708033 25541 raft_consensus.cc:385] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.708382 25541 raft_consensus.cc:740] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a724fa0654fa4809b29444f707a88767, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.709187 25541 consensus_queue.cc:260] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } } peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } }
I20260709 13:22:34.712633 25541 ts_tablet_manager.cc:1434] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:22:34.713644 25541 tablet_bootstrap.cc:492] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767: Bootstrap starting.
I20260709 13:22:34.714731 25543 tablet_bootstrap.cc:654] T 1cefd5d18235480d9e47c7bf0377e4f0 P 01445466116342929df00ee716f54e3f: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.719766 25541 tablet_bootstrap.cc:654] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.722065 25545 leader_election.cc:290] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881), 01445466116342929df00ee716f54e3f (127.14.23.65:43051)
I20260709 13:22:34.727804 25351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c94152f0c44421b2c2e3560785b3e6" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f" is_pre_election: true
W20260709 13:22:34.732046 25391 leader_election.cc:343] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01445466116342929df00ee716f54e3f (127.14.23.65:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:34.736472 25500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c94152f0c44421b2c2e3560785b3e6" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767" is_pre_election: true
I20260709 13:22:34.737277 25500 raft_consensus.cc:2468] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 0.
I20260709 13:22:34.738173 25543 tablet_bootstrap.cc:492] T 1cefd5d18235480d9e47c7bf0377e4f0 P 01445466116342929df00ee716f54e3f: No bootstrap required, opened a new log
I20260709 13:22:34.738782 25543 ts_tablet_manager.cc:1403] T 1cefd5d18235480d9e47c7bf0377e4f0 P 01445466116342929df00ee716f54e3f: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.014s
I20260709 13:22:34.738775 25390 leader_election.cc:304] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1c893d888632407db50d5d53a2f4edb6, a724fa0654fa4809b29444f707a88767; no voters: 01445466116342929df00ee716f54e3f
I20260709 13:22:34.740272 25545 raft_consensus.cc:2804] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:34.740283 25540 tablet_bootstrap.cc:492] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6: No bootstrap required, opened a new log
I20260709 13:22:34.740414 25541 tablet_bootstrap.cc:492] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767: No bootstrap required, opened a new log
I20260709 13:22:34.741117 25540 ts_tablet_manager.cc:1403] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6: Time spent bootstrapping tablet: real 0.049s	user 0.009s	sys 0.008s
I20260709 13:22:34.740923 25545 raft_consensus.cc:493] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:34.741626 25541 ts_tablet_manager.cc:1403] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.000s
I20260709 13:22:34.741752 25545 raft_consensus.cc:3060] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:34.742260 25543 raft_consensus.cc:359] T 1cefd5d18235480d9e47c7bf0377e4f0 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.743009 25543 raft_consensus.cc:385] T 1cefd5d18235480d9e47c7bf0377e4f0 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.743297 25543 raft_consensus.cc:740] T 1cefd5d18235480d9e47c7bf0377e4f0 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.744024 25543 consensus_queue.cc:260] T 1cefd5d18235480d9e47c7bf0377e4f0 P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.744282 25540 raft_consensus.cc:359] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.745025 25540 raft_consensus.cc:385] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.744623 25541 raft_consensus.cc:359] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.745332 25540 raft_consensus.cc:740] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.745435 25541 raft_consensus.cc:385] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.746364 25548 raft_consensus.cc:493] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:34.746865 25548 raft_consensus.cc:515] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.745851 25541 raft_consensus.cc:740] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a724fa0654fa4809b29444f707a88767, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.754288 25543 ts_tablet_manager.cc:1434] T 1cefd5d18235480d9e47c7bf0377e4f0 P 01445466116342929df00ee716f54e3f: Time spent starting tablet: real 0.015s	user 0.011s	sys 0.003s
I20260709 13:22:34.754390 25540 consensus_queue.cc:260] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.755749 25543 tablet_bootstrap.cc:492] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f: Bootstrap starting.
I20260709 13:22:34.762517 25543 tablet_bootstrap.cc:654] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.766855 25545 raft_consensus.cc:515] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } } peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } }
I20260709 13:22:34.768303 25351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30912c2e960f4e5c85dfe927e98a1e9f" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f" is_pre_election: true
W20260709 13:22:34.769836 25465 leader_election.cc:343] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01445466116342929df00ee716f54e3f (127.14.23.65:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:34.770104 25500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c94152f0c44421b2c2e3560785b3e6" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:34.770704 25500 raft_consensus.cc:3060] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:34.774583 25543 tablet_bootstrap.cc:492] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f: No bootstrap required, opened a new log
I20260709 13:22:34.775249 25543 ts_tablet_manager.cc:1403] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260709 13:22:34.778800 25500 raft_consensus.cc:2468] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 1.
I20260709 13:22:34.778614 25543 raft_consensus.cc:359] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.779610 25548 leader_election.cc:290] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657)
I20260709 13:22:34.780279 25541 consensus_queue.cc:260] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.781747 25543 raft_consensus.cc:385] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.782146 25543 raft_consensus.cc:740] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.782941 25543 consensus_queue.cc:260] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.784888 25426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30912c2e960f4e5c85dfe927e98a1e9f" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6" is_pre_election: true
I20260709 13:22:34.785300 25543 ts_tablet_manager.cc:1434] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 13:22:34.786257 25540 ts_tablet_manager.cc:1434] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6: Time spent starting tablet: real 0.045s	user 0.026s	sys 0.015s
W20260709 13:22:34.786630 25463 leader_election.cc:343] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:34.787168 25463 leader_election.cc:304] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a724fa0654fa4809b29444f707a88767; no voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6
I20260709 13:22:34.787642 25390 leader_election.cc:304] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c893d888632407db50d5d53a2f4edb6, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:34.786677 25543 tablet_bootstrap.cc:492] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f: Bootstrap starting.
I20260709 13:22:34.788604 25548 raft_consensus.cc:2749] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:34.781067 25351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c94152f0c44421b2c2e3560785b3e6" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:34.783499 25541 ts_tablet_manager.cc:1434] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.000s
I20260709 13:22:34.791508 25541 tablet_bootstrap.cc:492] T 5fead9ecfeb04d0d94509484c7234118 P a724fa0654fa4809b29444f707a88767: Bootstrap starting.
I20260709 13:22:34.792280 25540 tablet_bootstrap.cc:492] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6: Bootstrap starting.
W20260709 13:22:34.794212 25391 leader_election.cc:343] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 01445466116342929df00ee716f54e3f (127.14.23.65:43051): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:34.794575 25543 tablet_bootstrap.cc:654] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.797811 25541 tablet_bootstrap.cc:654] T 5fead9ecfeb04d0d94509484c7234118 P a724fa0654fa4809b29444f707a88767: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.800107 25558 raft_consensus.cc:2804] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:34.802865 25558 raft_consensus.cc:697] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Becoming Leader. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Running, Role: LEADER
I20260709 13:22:34.803814 25558 consensus_queue.cc:237] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } } peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } }
I20260709 13:22:34.798957 25540 tablet_bootstrap.cc:654] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.799340 25545 leader_election.cc:290] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 election: Requested vote from peers a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881), 01445466116342929df00ee716f54e3f (127.14.23.65:43051)
I20260709 13:22:34.809217 25541 tablet_bootstrap.cc:492] T 5fead9ecfeb04d0d94509484c7234118 P a724fa0654fa4809b29444f707a88767: No bootstrap required, opened a new log
I20260709 13:22:34.809870 25541 ts_tablet_manager.cc:1403] T 5fead9ecfeb04d0d94509484c7234118 P a724fa0654fa4809b29444f707a88767: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.004s
I20260709 13:22:34.812661 25543 tablet_bootstrap.cc:492] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f: No bootstrap required, opened a new log
I20260709 13:22:34.813203 25543 ts_tablet_manager.cc:1403] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.013s
I20260709 13:22:34.813078 25541 raft_consensus.cc:359] T 5fead9ecfeb04d0d94509484c7234118 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.814059 25541 raft_consensus.cc:385] T 5fead9ecfeb04d0d94509484c7234118 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.814424 25541 raft_consensus.cc:740] T 5fead9ecfeb04d0d94509484c7234118 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a724fa0654fa4809b29444f707a88767, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.815586 25540 tablet_bootstrap.cc:492] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6: No bootstrap required, opened a new log
I20260709 13:22:34.815430 25541 consensus_queue.cc:260] T 5fead9ecfeb04d0d94509484c7234118 P a724fa0654fa4809b29444f707a88767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.816169 25540 ts_tablet_manager.cc:1403] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.000s
I20260709 13:22:34.816507 25543 raft_consensus.cc:359] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } } peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } }
I20260709 13:22:34.817181 25543 raft_consensus.cc:385] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.817413 25543 raft_consensus.cc:740] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.818107 25543 consensus_queue.cc:260] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } } peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } }
I20260709 13:22:34.820268 25543 ts_tablet_manager.cc:1434] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:22:34.821597 25543 tablet_bootstrap.cc:492] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f: Bootstrap starting.
I20260709 13:22:34.824586 25541 ts_tablet_manager.cc:1434] T 5fead9ecfeb04d0d94509484c7234118 P a724fa0654fa4809b29444f707a88767: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:22:34.824342 25248 catalog_manager.cc:5697] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "1c893d888632407db50d5d53a2f4edb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:34.825755 25541 tablet_bootstrap.cc:492] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767: Bootstrap starting.
I20260709 13:22:34.827677 25543 tablet_bootstrap.cc:654] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.833461 25540 raft_consensus.cc:359] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.834299 25540 raft_consensus.cc:385] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.834558 25541 tablet_bootstrap.cc:654] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.834605 25540 raft_consensus.cc:740] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.835638 25540 consensus_queue.cc:260] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.839416 25540 ts_tablet_manager.cc:1434] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6: Time spent starting tablet: real 0.023s	user 0.010s	sys 0.000s
I20260709 13:22:34.840451 25540 tablet_bootstrap.cc:492] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6: Bootstrap starting.
I20260709 13:22:34.848820 25541 tablet_bootstrap.cc:492] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767: No bootstrap required, opened a new log
I20260709 13:22:34.849535 25541 ts_tablet_manager.cc:1403] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.001s
I20260709 13:22:34.850488 25543 tablet_bootstrap.cc:492] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f: No bootstrap required, opened a new log
I20260709 13:22:34.851073 25543 ts_tablet_manager.cc:1403] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.011s
I20260709 13:22:34.852205 25540 tablet_bootstrap.cc:654] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.852943 25541 raft_consensus.cc:359] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.853807 25541 raft_consensus.cc:385] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.854261 25541 raft_consensus.cc:740] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a724fa0654fa4809b29444f707a88767, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.854054 25543 raft_consensus.cc:359] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.855049 25543 raft_consensus.cc:385] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.855356 25543 raft_consensus.cc:740] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.855748 25541 consensus_queue.cc:260] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.856215 25543 consensus_queue.cc:260] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.859107 25541 ts_tablet_manager.cc:1434] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:22:34.859824 25540 tablet_bootstrap.cc:492] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6: No bootstrap required, opened a new log
I20260709 13:22:34.860381 25540 ts_tablet_manager.cc:1403] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.003s
I20260709 13:22:34.861976 25541 tablet_bootstrap.cc:492] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767: Bootstrap starting.
I20260709 13:22:34.863260 25540 raft_consensus.cc:359] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.864049 25540 raft_consensus.cc:385] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.864351 25540 raft_consensus.cc:740] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.864780 25543 ts_tablet_manager.cc:1434] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:22:34.865209 25540 consensus_queue.cc:260] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.868677 25541 tablet_bootstrap.cc:654] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.870083 25543 tablet_bootstrap.cc:492] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f: Bootstrap starting.
I20260709 13:22:34.870434 25540 ts_tablet_manager.cc:1434] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
I20260709 13:22:34.871508 25540 tablet_bootstrap.cc:492] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6: Bootstrap starting.
I20260709 13:22:34.876793 25543 tablet_bootstrap.cc:654] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.881516 25540 tablet_bootstrap.cc:654] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.882853 25541 tablet_bootstrap.cc:492] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767: No bootstrap required, opened a new log
I20260709 13:22:34.883414 25541 ts_tablet_manager.cc:1403] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.004s
I20260709 13:22:34.886439 25541 raft_consensus.cc:359] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.886976 25541 raft_consensus.cc:385] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.887202 25541 raft_consensus.cc:740] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a724fa0654fa4809b29444f707a88767, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.887382 25540 tablet_bootstrap.cc:492] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6: No bootstrap required, opened a new log
I20260709 13:22:34.887821 25543 tablet_bootstrap.cc:492] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f: No bootstrap required, opened a new log
I20260709 13:22:34.888087 25540 ts_tablet_manager.cc:1403] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:22:34.887898 25541 consensus_queue.cc:260] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.888437 25543 ts_tablet_manager.cc:1403] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f: Time spent bootstrapping tablet: real 0.019s	user 0.017s	sys 0.001s
I20260709 13:22:34.890282 25541 ts_tablet_manager.cc:1434] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:22:34.891324 25541 tablet_bootstrap.cc:492] T 8b34ba27e9a24cab9530218fa592c6ae P a724fa0654fa4809b29444f707a88767: Bootstrap starting.
I20260709 13:22:34.891070 25540 raft_consensus.cc:359] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.891865 25540 raft_consensus.cc:385] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.891467 25543 raft_consensus.cc:359] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.892184 25540 raft_consensus.cc:740] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.892375 25543 raft_consensus.cc:385] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.892788 25543 raft_consensus.cc:740] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.893142 25540 consensus_queue.cc:260] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.893644 25543 consensus_queue.cc:260] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.896143 25543 ts_tablet_manager.cc:1434] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.005s
I20260709 13:22:34.897209 25543 tablet_bootstrap.cc:492] T 28a64081a0c94a92a11015ec646adf0c P 01445466116342929df00ee716f54e3f: Bootstrap starting.
I20260709 13:22:34.897548 25549 raft_consensus.cc:493] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:34.908293 25541 tablet_bootstrap.cc:654] T 8b34ba27e9a24cab9530218fa592c6ae P a724fa0654fa4809b29444f707a88767: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.908318 25543 tablet_bootstrap.cc:654] T 28a64081a0c94a92a11015ec646adf0c P 01445466116342929df00ee716f54e3f: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.908170 25549 raft_consensus.cc:515] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.900667 25540 ts_tablet_manager.cc:1434] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.004s
I20260709 13:22:34.917243 25549 leader_election.cc:290] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:34.919425 25549 raft_consensus.cc:493] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:34.922514 25549 raft_consensus.cc:515] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } } peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } }
I20260709 13:22:34.930358 25540 tablet_bootstrap.cc:492] T 8b34ba27e9a24cab9530218fa592c6ae P 1c893d888632407db50d5d53a2f4edb6: Bootstrap starting.
I20260709 13:22:34.931223 25558 raft_consensus.cc:493] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:34.931890 25558 raft_consensus.cc:515] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.932650 25543 tablet_bootstrap.cc:492] T 28a64081a0c94a92a11015ec646adf0c P 01445466116342929df00ee716f54e3f: No bootstrap required, opened a new log
I20260709 13:22:34.933255 25543 ts_tablet_manager.cc:1403] T 28a64081a0c94a92a11015ec646adf0c P 01445466116342929df00ee716f54e3f: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.019s
I20260709 13:22:34.935892 25351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fead9ecfeb04d0d94509484c7234118" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f" is_pre_election: true
I20260709 13:22:34.936793 25351 raft_consensus.cc:2468] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 0.
I20260709 13:22:34.938000 25549 leader_election.cc:290] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:34.938280 25391 leader_election.cc:304] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:34.939369 25545 raft_consensus.cc:2804] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:34.939740 25545 raft_consensus.cc:493] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:34.939973 25558 leader_election.cc:290] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:34.940824 25549 raft_consensus.cc:493] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:34.940460 25500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fead9ecfeb04d0d94509484c7234118" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767" is_pre_election: true
I20260709 13:22:34.941561 25549 raft_consensus.cc:515] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.940064 25545 raft_consensus.cc:3060] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:34.941820 25500 raft_consensus.cc:2468] T 5fead9ecfeb04d0d94509484c7234118 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 0.
I20260709 13:22:34.945941 25549 leader_election.cc:290] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:34.948673 25540 tablet_bootstrap.cc:654] T 8b34ba27e9a24cab9530218fa592c6ae P 1c893d888632407db50d5d53a2f4edb6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.949227 25543 raft_consensus.cc:359] T 28a64081a0c94a92a11015ec646adf0c P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.950085 25543 raft_consensus.cc:385] T 28a64081a0c94a92a11015ec646adf0c P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.950368 25543 raft_consensus.cc:740] T 28a64081a0c94a92a11015ec646adf0c P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.951117 25543 consensus_queue.cc:260] T 28a64081a0c94a92a11015ec646adf0c P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.951290 25545 raft_consensus.cc:515] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.951649 25549 raft_consensus.cc:493] T 1cefd5d18235480d9e47c7bf0377e4f0 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:34.954577 25549 raft_consensus.cc:515] T 1cefd5d18235480d9e47c7bf0377e4f0 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.955492 25541 tablet_bootstrap.cc:492] T 8b34ba27e9a24cab9530218fa592c6ae P a724fa0654fa4809b29444f707a88767: No bootstrap required, opened a new log
I20260709 13:22:34.956166 25541 ts_tablet_manager.cc:1403] T 8b34ba27e9a24cab9530218fa592c6ae P a724fa0654fa4809b29444f707a88767: Time spent bootstrapping tablet: real 0.065s	user 0.026s	sys 0.030s
I20260709 13:22:34.956751 25549 leader_election.cc:290] T 1cefd5d18235480d9e47c7bf0377e4f0 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
W20260709 13:22:34.957062 25297 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:34.953289 25543 ts_tablet_manager.cc:1434] T 28a64081a0c94a92a11015ec646adf0c P 01445466116342929df00ee716f54e3f: Time spent starting tablet: real 0.020s	user 0.001s	sys 0.002s
I20260709 13:22:34.958523 25543 tablet_bootstrap.cc:492] T 6729a5f0b78e4d148483f7b5d427304a P 01445466116342929df00ee716f54e3f: Bootstrap starting.
I20260709 13:22:34.959527 25541 raft_consensus.cc:359] T 8b34ba27e9a24cab9530218fa592c6ae P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.960395 25541 raft_consensus.cc:385] T 8b34ba27e9a24cab9530218fa592c6ae P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.960757 25541 raft_consensus.cc:740] T 8b34ba27e9a24cab9530218fa592c6ae P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a724fa0654fa4809b29444f707a88767, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.961673 25541 consensus_queue.cc:260] T 8b34ba27e9a24cab9530218fa592c6ae P a724fa0654fa4809b29444f707a88767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.963243 25351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fead9ecfeb04d0d94509484c7234118" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:34.963923 25351 raft_consensus.cc:3060] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:34.964515 25543 tablet_bootstrap.cc:654] T 6729a5f0b78e4d148483f7b5d427304a P 01445466116342929df00ee716f54e3f: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.966147 25548 raft_consensus.cc:493] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:34.966674 25548 raft_consensus.cc:515] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.970175 25548 leader_election.cc:290] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657)
I20260709 13:22:34.970486 25350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cefd5d18235480d9e47c7bf0377e4f0" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f" is_pre_election: true
I20260709 13:22:34.971195 25350 raft_consensus.cc:2468] T 1cefd5d18235480d9e47c7bf0377e4f0 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 0.
I20260709 13:22:34.971793 25558 raft_consensus.cc:493] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:34.972376 25541 ts_tablet_manager.cc:1434] T 8b34ba27e9a24cab9530218fa592c6ae P a724fa0654fa4809b29444f707a88767: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.003s
I20260709 13:22:34.972333 25558 raft_consensus.cc:515] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.973104 25465 leader_election.cc:304] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:34.972241 25426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cefd5d18235480d9e47c7bf0377e4f0" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6" is_pre_election: true
I20260709 13:22:34.974404 25426 raft_consensus.cc:2468] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 0.
I20260709 13:22:34.974756 25558 leader_election.cc:290] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:34.976197 25350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a64081a0c94a92a11015ec646adf0c" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f" is_pre_election: true
I20260709 13:22:34.976856 25350 raft_consensus.cc:2468] T 28a64081a0c94a92a11015ec646adf0c P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 0.
I20260709 13:22:34.977036 25500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a64081a0c94a92a11015ec646adf0c" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767" is_pre_election: true
I20260709 13:22:34.978170 25391 leader_election.cc:304] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:34.978868 25548 raft_consensus.cc:2804] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:34.979017 25558 raft_consensus.cc:2804] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:34.979314 25548 raft_consensus.cc:493] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:34.979416 25558 raft_consensus.cc:493] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:34.979727 25548 raft_consensus.cc:3060] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:34.979804 25558 raft_consensus.cc:3060] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:34.982020 25390 leader_election.cc:343] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:34.984100 25541 tablet_bootstrap.cc:492] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767: Bootstrap starting.
I20260709 13:22:34.992209 25351 raft_consensus.cc:2468] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 1.
I20260709 13:22:34.992427 25499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c94152f0c44421b2c2e3560785b3e6" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767" is_pre_election: true
I20260709 13:22:34.992889 25500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768383f820c4e4cb8d8595901456ae1" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767" is_pre_election: true
I20260709 13:22:34.993521 25499 raft_consensus.cc:2393] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01445466116342929df00ee716f54e3f in current term 1: Already voted for candidate 1c893d888632407db50d5d53a2f4edb6 in this term.
I20260709 13:22:34.993592 25391 leader_election.cc:304] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:34.989490 25558 raft_consensus.cc:515] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:34.994926 25556 raft_consensus.cc:2804] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:34.994316 25498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fead9ecfeb04d0d94509484c7234118" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767" is_pre_election: true
I20260709 13:22:34.995219 25497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cefd5d18235480d9e47c7bf0377e4f0" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767" is_pre_election: true
I20260709 13:22:34.996272 25498 raft_consensus.cc:2468] T 5fead9ecfeb04d0d94509484c7234118 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 0.
W20260709 13:22:34.999114 25315 leader_election.cc:343] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:34.989722 25548 raft_consensus.cc:515] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.000259 25315 leader_election.cc:304] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:35.006748 25424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fead9ecfeb04d0d94509484c7234118" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6" is_pre_election: true
I20260709 13:22:35.007560 25497 raft_consensus.cc:2393] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01445466116342929df00ee716f54e3f in current term 1: Already voted for candidate a724fa0654fa4809b29444f707a88767 in this term.
I20260709 13:22:35.007725 25572 raft_consensus.cc:2764] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:22:35.008563 25426 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768383f820c4e4cb8d8595901456ae1" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6" is_pre_election: true
I20260709 13:22:35.005705 25425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c94152f0c44421b2c2e3560785b3e6" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6" is_pre_election: true
W20260709 13:22:35.012722 25314 leader_election.cc:343] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:35.013200 25314 leader_election.cc:304] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01445466116342929df00ee716f54e3f; no voters: 1c893d888632407db50d5d53a2f4edb6, a724fa0654fa4809b29444f707a88767
I20260709 13:22:35.013788 25548 leader_election.cc:290] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 election: Requested vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657)
I20260709 13:22:35.014071 25572 raft_consensus.cc:2749] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:35.014592 25314 leader_election.cc:304] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01445466116342929df00ee716f54e3f; no voters: 1c893d888632407db50d5d53a2f4edb6, a724fa0654fa4809b29444f707a88767
I20260709 13:22:35.008028 25423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cefd5d18235480d9e47c7bf0377e4f0" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6" is_pre_election: true
I20260709 13:22:35.016515 25423 raft_consensus.cc:2468] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 0.
I20260709 13:22:35.017654 25541 tablet_bootstrap.cc:654] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:35.017721 25425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cefd5d18235480d9e47c7bf0377e4f0" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:35.019614 25350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cefd5d18235480d9e47c7bf0377e4f0" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:35.020155 25572 raft_consensus.cc:3060] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.020663 25351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a64081a0c94a92a11015ec646adf0c" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:35.021528 25545 leader_election.cc:290] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 election: Requested vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:35.025098 25314 leader_election.cc:304] T 1cefd5d18235480d9e47c7bf0377e4f0 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: a724fa0654fa4809b29444f707a88767
I20260709 13:22:35.025967 25577 raft_consensus.cc:493] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:35.023138 25556 raft_consensus.cc:697] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Becoming Leader. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Running, Role: LEADER
I20260709 13:22:35.028864 25577 raft_consensus.cc:515] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.029783 25540 tablet_bootstrap.cc:492] T 8b34ba27e9a24cab9530218fa592c6ae P 1c893d888632407db50d5d53a2f4edb6: No bootstrap required, opened a new log
I20260709 13:22:35.030354 25540 ts_tablet_manager.cc:1403] T 8b34ba27e9a24cab9530218fa592c6ae P 1c893d888632407db50d5d53a2f4edb6: Time spent bootstrapping tablet: real 0.100s	user 0.009s	sys 0.003s
I20260709 13:22:35.031229 25577 leader_election.cc:290] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:35.026433 25558 leader_election.cc:290] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 election: Requested vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:35.032042 25498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a64081a0c94a92a11015ec646adf0c" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:35.034392 25349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc2e95552a142ea8f26983d2b49e0c5" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f" is_pre_election: true
I20260709 13:22:35.035152 25349 raft_consensus.cc:2468] T efc2e95552a142ea8f26983d2b49e0c5 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 0.
I20260709 13:22:35.032377 25351 raft_consensus.cc:3060] T 28a64081a0c94a92a11015ec646adf0c P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.031406 25497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fead9ecfeb04d0d94509484c7234118" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:35.036140 25497 raft_consensus.cc:3060] T 5fead9ecfeb04d0d94509484c7234118 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.036342 25391 leader_election.cc:304] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:35.037302 25558 raft_consensus.cc:2804] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:35.037663 25558 raft_consensus.cc:493] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:35.023885 25425 raft_consensus.cc:3060] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.026851 25350 raft_consensus.cc:3060] T 1cefd5d18235480d9e47c7bf0377e4f0 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.039171 25498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc2e95552a142ea8f26983d2b49e0c5" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767" is_pre_election: true
I20260709 13:22:35.039966 25498 raft_consensus.cc:2468] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 0.
I20260709 13:22:35.038026 25558 raft_consensus.cc:3060] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:35.038297 25390 leader_election.cc:343] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:35.054927 25540 raft_consensus.cc:359] T 8b34ba27e9a24cab9530218fa592c6ae P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.056023 25540 raft_consensus.cc:385] T 8b34ba27e9a24cab9530218fa592c6ae P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:35.056316 25540 raft_consensus.cc:740] T 8b34ba27e9a24cab9530218fa592c6ae P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Initialized, Role: FOLLOWER
I20260709 13:22:35.057286 25543 tablet_bootstrap.cc:492] T 6729a5f0b78e4d148483f7b5d427304a P 01445466116342929df00ee716f54e3f: No bootstrap required, opened a new log
I20260709 13:22:35.057101 25540 consensus_queue.cc:260] T 8b34ba27e9a24cab9530218fa592c6ae P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.057849 25543 ts_tablet_manager.cc:1403] T 6729a5f0b78e4d148483f7b5d427304a P 01445466116342929df00ee716f54e3f: Time spent bootstrapping tablet: real 0.100s	user 0.038s	sys 0.041s
I20260709 13:22:35.055104 25351 raft_consensus.cc:2468] T 28a64081a0c94a92a11015ec646adf0c P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 1.
I20260709 13:22:35.059408 25391 leader_election.cc:304] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: a724fa0654fa4809b29444f707a88767
I20260709 13:22:35.061802 25576 raft_consensus.cc:2804] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:35.063941 25497 raft_consensus.cc:2468] T 5fead9ecfeb04d0d94509484c7234118 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 1.
I20260709 13:22:35.065254 25540 ts_tablet_manager.cc:1434] T 8b34ba27e9a24cab9530218fa592c6ae P 1c893d888632407db50d5d53a2f4edb6: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 13:22:35.062659 25543 raft_consensus.cc:359] T 6729a5f0b78e4d148483f7b5d427304a P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.060171 25556 consensus_queue.cc:237] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.066105 25543 raft_consensus.cc:385] T 6729a5f0b78e4d148483f7b5d427304a P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:35.066437 25540 tablet_bootstrap.cc:492] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6: Bootstrap starting.
I20260709 13:22:35.066417 25543 raft_consensus.cc:740] T 6729a5f0b78e4d148483f7b5d427304a P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Initialized, Role: FOLLOWER
I20260709 13:22:35.068265 25543 consensus_queue.cc:260] T 6729a5f0b78e4d148483f7b5d427304a P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.067008 25558 raft_consensus.cc:515] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.062695 25576 raft_consensus.cc:697] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Becoming Leader. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Running, Role: LEADER
I20260709 13:22:35.063943 25425 raft_consensus.cc:2468] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 1.
I20260709 13:22:35.072065 25463 leader_election.cc:304] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c893d888632407db50d5d53a2f4edb6, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:35.072803 25350 raft_consensus.cc:2468] T 1cefd5d18235480d9e47c7bf0377e4f0 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 1.
I20260709 13:22:35.073159 25548 raft_consensus.cc:2804] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:35.073347 25549 raft_consensus.cc:2764] T 1cefd5d18235480d9e47c7bf0377e4f0 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:22:35.069532 25572 raft_consensus.cc:2749] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:35.075634 25543 ts_tablet_manager.cc:1434] T 6729a5f0b78e4d148483f7b5d427304a P 01445466116342929df00ee716f54e3f: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:22:35.077144 25541 tablet_bootstrap.cc:492] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767: No bootstrap required, opened a new log
I20260709 13:22:35.077695 25541 ts_tablet_manager.cc:1403] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767: Time spent bootstrapping tablet: real 0.094s	user 0.011s	sys 0.007s
I20260709 13:22:35.075670 25548 raft_consensus.cc:697] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Becoming Leader. State: Replica: a724fa0654fa4809b29444f707a88767, State: Running, Role: LEADER
I20260709 13:22:35.078444 25576 consensus_queue.cc:237] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.078848 25548 consensus_queue.cc:237] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.080561 25540 tablet_bootstrap.cc:654] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:35.080533 25541 raft_consensus.cc:359] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.081557 25541 raft_consensus.cc:385] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:35.081857 25541 raft_consensus.cc:740] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a724fa0654fa4809b29444f707a88767, State: Initialized, Role: FOLLOWER
I20260709 13:22:35.082257 25581 raft_consensus.cc:493] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:35.080942 25558 leader_election.cc:290] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 election: Requested vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:35.082729 25541 consensus_queue.cc:260] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.082799 25581 raft_consensus.cc:515] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.088459 25581 leader_election.cc:290] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657)
I20260709 13:22:35.088681 25425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6729a5f0b78e4d148483f7b5d427304a" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6" is_pre_election: true
I20260709 13:22:35.089398 25497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc2e95552a142ea8f26983d2b49e0c5" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767"
W20260709 13:22:35.090243 25463 leader_election.cc:343] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:35.090981 25497 raft_consensus.cc:3060] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.133350 25351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc2e95552a142ea8f26983d2b49e0c5" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:35.133350 25350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6729a5f0b78e4d148483f7b5d427304a" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f" is_pre_election: true
I20260709 13:22:35.135363 25351 raft_consensus.cc:3060] T efc2e95552a142ea8f26983d2b49e0c5 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.135497 25350 raft_consensus.cc:2468] T 6729a5f0b78e4d148483f7b5d427304a P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 0.
I20260709 13:22:35.153129 25465 leader_election.cc:304] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01445466116342929df00ee716f54e3f, a724fa0654fa4809b29444f707a88767; no voters: 1c893d888632407db50d5d53a2f4edb6
I20260709 13:22:35.156561 25541 ts_tablet_manager.cc:1434] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767: Time spent starting tablet: real 0.078s	user 0.014s	sys 0.008s
I20260709 13:22:35.159379 25541 tablet_bootstrap.cc:492] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767: Bootstrap starting.
I20260709 13:22:35.164063 25540 tablet_bootstrap.cc:492] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6: No bootstrap required, opened a new log
I20260709 13:22:35.166147 25585 raft_consensus.cc:2804] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:35.167831 25540 ts_tablet_manager.cc:1403] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6: Time spent bootstrapping tablet: real 0.102s	user 0.023s	sys 0.008s
I20260709 13:22:35.170176 25249 catalog_manager.cc:5697] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "1c893d888632407db50d5d53a2f4edb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:35.171070 25248 catalog_manager.cc:5697] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 reported cstate change: term changed from 0 to 1, leader changed from <none> to a724fa0654fa4809b29444f707a88767 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "a724fa0654fa4809b29444f707a88767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:35.173554 25585 raft_consensus.cc:493] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:35.174440 25585 raft_consensus.cc:3060] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.184294 25497 raft_consensus.cc:2468] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 1.
I20260709 13:22:35.182646 25540 raft_consensus.cc:359] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.185969 25540 raft_consensus.cc:385] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:35.186399 25540 raft_consensus.cc:740] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Initialized, Role: FOLLOWER
I20260709 13:22:35.186879 25390 leader_election.cc:304] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c893d888632407db50d5d53a2f4edb6, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:35.187300 25540 consensus_queue.cc:260] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.187855 25351 raft_consensus.cc:2468] T efc2e95552a142ea8f26983d2b49e0c5 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 1.
I20260709 13:22:35.188633 25556 raft_consensus.cc:2804] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:35.189400 25556 raft_consensus.cc:697] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Becoming Leader. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Running, Role: LEADER
I20260709 13:22:35.191429 25576 raft_consensus.cc:493] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:35.191159 25556 consensus_queue.cc:237] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.191841 25541 tablet_bootstrap.cc:654] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:35.192256 25576 raft_consensus.cc:515] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.193226 25540 ts_tablet_manager.cc:1434] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6: Time spent starting tablet: real 0.025s	user 0.012s	sys 0.001s
I20260709 13:22:35.194989 25540 tablet_bootstrap.cc:492] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6: Bootstrap starting.
I20260709 13:22:35.195215 25576 leader_election.cc:290] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:35.186834 25585 raft_consensus.cc:515] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.196118 25350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30912c2e960f4e5c85dfe927e98a1e9f" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f" is_pre_election: true
I20260709 13:22:35.196980 25350 raft_consensus.cc:2468] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 0.
I20260709 13:22:35.199229 25498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30912c2e960f4e5c85dfe927e98a1e9f" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767" is_pre_election: true
I20260709 13:22:35.200060 25424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6729a5f0b78e4d148483f7b5d427304a" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:35.200826 25424 raft_consensus.cc:3060] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.201279 25391 leader_election.cc:304] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:35.202023 25576 raft_consensus.cc:2804] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:35.202457 25576 raft_consensus.cc:493] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:35.202868 25576 raft_consensus.cc:3060] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.203518 25498 raft_consensus.cc:2468] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 0.
I20260709 13:22:35.205365 25350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6729a5f0b78e4d148483f7b5d427304a" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:35.206125 25350 raft_consensus.cc:3060] T 6729a5f0b78e4d148483f7b5d427304a P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.211232 25585 leader_election.cc:290] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 election: Requested vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657)
I20260709 13:22:35.212494 25540 tablet_bootstrap.cc:654] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:35.212540 25576 raft_consensus.cc:515] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.213503 25558 consensus_queue.cc:1048] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:35.215786 25351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30912c2e960f4e5c85dfe927e98a1e9f" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:35.216435 25351 raft_consensus.cc:3060] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.219396 25541 tablet_bootstrap.cc:492] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767: No bootstrap required, opened a new log
I20260709 13:22:35.220109 25541 ts_tablet_manager.cc:1403] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767: Time spent bootstrapping tablet: real 0.061s	user 0.012s	sys 0.007s
I20260709 13:22:35.222738 25576 leader_election.cc:290] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 election: Requested vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:35.224290 25497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30912c2e960f4e5c85dfe927e98a1e9f" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:35.225076 25497 raft_consensus.cc:3060] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.231113 25540 tablet_bootstrap.cc:492] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6: No bootstrap required, opened a new log
I20260709 13:22:35.231750 25540 ts_tablet_manager.cc:1403] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6: Time spent bootstrapping tablet: real 0.037s	user 0.006s	sys 0.010s
I20260709 13:22:35.231294 25541 raft_consensus.cc:359] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.232278 25541 raft_consensus.cc:385] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:35.232625 25541 raft_consensus.cc:740] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a724fa0654fa4809b29444f707a88767, State: Initialized, Role: FOLLOWER
I20260709 13:22:35.213688 25424 raft_consensus.cc:2468] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 1.
I20260709 13:22:35.233686 25541 consensus_queue.cc:260] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.234483 25497 raft_consensus.cc:2468] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 1.
I20260709 13:22:35.234910 25463 leader_election.cc:304] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c893d888632407db50d5d53a2f4edb6, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:35.235862 25585 raft_consensus.cc:2804] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:35.236279 25390 leader_election.cc:304] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c893d888632407db50d5d53a2f4edb6, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:35.236394 25585 raft_consensus.cc:697] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Becoming Leader. State: Replica: a724fa0654fa4809b29444f707a88767, State: Running, Role: LEADER
I20260709 13:22:35.238767 25585 consensus_queue.cc:237] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.240244 25577 raft_consensus.cc:2804] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:35.240985 25577 raft_consensus.cc:697] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Becoming Leader. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Running, Role: LEADER
I20260709 13:22:35.237257 25548 raft_consensus.cc:493] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:35.242362 25548 raft_consensus.cc:515] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.242523 25577 consensus_queue.cc:237] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.246269 25349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a64081a0c94a92a11015ec646adf0c" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f" is_pre_election: true
I20260709 13:22:35.247309 25349 raft_consensus.cc:2393] T 28a64081a0c94a92a11015ec646adf0c P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a724fa0654fa4809b29444f707a88767 in current term 1: Already voted for candidate 1c893d888632407db50d5d53a2f4edb6 in this term.
I20260709 13:22:35.247666 25548 leader_election.cc:290] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657)
I20260709 13:22:35.248271 25351 raft_consensus.cc:2468] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 1.
I20260709 13:22:35.249804 25541 ts_tablet_manager.cc:1434] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.003s
I20260709 13:22:35.251098 25540 raft_consensus.cc:359] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.252071 25540 raft_consensus.cc:385] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:35.252372 25540 raft_consensus.cc:740] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Initialized, Role: FOLLOWER
I20260709 13:22:35.256841 25350 raft_consensus.cc:2468] T 6729a5f0b78e4d148483f7b5d427304a P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 1.
I20260709 13:22:35.253898 25424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a64081a0c94a92a11015ec646adf0c" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6" is_pre_election: true
I20260709 13:22:35.260241 25577 consensus_queue.cc:1048] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:35.261164 25463 leader_election.cc:304] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a724fa0654fa4809b29444f707a88767; no voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6
I20260709 13:22:35.261291 25248 catalog_manager.cc:5697] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 reported cstate change: term changed from 0 to 1, leader changed from <none> to a724fa0654fa4809b29444f707a88767 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "a724fa0654fa4809b29444f707a88767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:35.262750 25585 raft_consensus.cc:3060] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.257457 25540 consensus_queue.cc:260] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.265940 25540 ts_tablet_manager.cc:1434] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.002s
I20260709 13:22:35.271932 25585 raft_consensus.cc:2749] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:35.282785 25249 catalog_manager.cc:5697] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "1c893d888632407db50d5d53a2f4edb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:35.284690 25249 catalog_manager.cc:5697] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "1c893d888632407db50d5d53a2f4edb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:35.301506 25248 catalog_manager.cc:5697] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "1c893d888632407db50d5d53a2f4edb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:35.371592 25580 raft_consensus.cc:493] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:35.372224 25580 raft_consensus.cc:515] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.374207 25580 leader_election.cc:290] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:35.375037 25424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34ba27e9a24cab9530218fa592c6ae" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6" is_pre_election: true
I20260709 13:22:35.375416 25498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34ba27e9a24cab9530218fa592c6ae" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767" is_pre_election: true
I20260709 13:22:35.375837 25424 raft_consensus.cc:2468] T 8b34ba27e9a24cab9530218fa592c6ae P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 0.
I20260709 13:22:35.376173 25498 raft_consensus.cc:2468] T 8b34ba27e9a24cab9530218fa592c6ae P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 0.
I20260709 13:22:35.377135 25314 leader_election.cc:304] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:35.377832 25580 raft_consensus.cc:2804] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:35.378208 25580 raft_consensus.cc:493] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:35.378461 25580 raft_consensus.cc:3060] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.385118 25580 raft_consensus.cc:515] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.387274 25580 leader_election.cc:290] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 election: Requested vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:35.388057 25424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34ba27e9a24cab9530218fa592c6ae" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:35.388214 25498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34ba27e9a24cab9530218fa592c6ae" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:35.388599 25424 raft_consensus.cc:3060] T 8b34ba27e9a24cab9530218fa592c6ae P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.388763 25498 raft_consensus.cc:3060] T 8b34ba27e9a24cab9530218fa592c6ae P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.394223 25424 raft_consensus.cc:2468] T 8b34ba27e9a24cab9530218fa592c6ae P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 1.
I20260709 13:22:35.395321 25498 raft_consensus.cc:2468] T 8b34ba27e9a24cab9530218fa592c6ae P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 1.
I20260709 13:22:35.395367 25314 leader_election.cc:304] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:35.396445 25580 raft_consensus.cc:2804] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:35.396935 25580 raft_consensus.cc:697] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Becoming Leader. State: Replica: 01445466116342929df00ee716f54e3f, State: Running, Role: LEADER
I20260709 13:22:35.397841 25580 consensus_queue.cc:237] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.405171 25248 catalog_manager.cc:5697] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f reported cstate change: term changed from 0 to 1, leader changed from <none> to 01445466116342929df00ee716f54e3f (127.14.23.65). New cstate: current_term: 1 leader_uuid: "01445466116342929df00ee716f54e3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:35.462164 25556 consensus_queue.cc:1048] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:35.471535 25585 raft_consensus.cc:493] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:35.472127 25585 raft_consensus.cc:515] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.474148 25556 consensus_queue.cc:1048] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:35.475064 25351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768383f820c4e4cb8d8595901456ae1" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f" is_pre_election: true
I20260709 13:22:35.475910 25351 raft_consensus.cc:2468] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 0.
I20260709 13:22:35.477219 25465 leader_election.cc:304] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:35.478514 25581 raft_consensus.cc:2804] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:35.478897 25581 raft_consensus.cc:493] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:35.479259 25581 raft_consensus.cc:3060] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.482635 25585 leader_election.cc:290] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657)
I20260709 13:22:35.484436 25424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768383f820c4e4cb8d8595901456ae1" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6" is_pre_election: true
I20260709 13:22:35.485257 25424 raft_consensus.cc:2468] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 0.
I20260709 13:22:35.488394 25581 raft_consensus.cc:515] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.489993 25585 consensus_queue.cc:1048] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:35.491389 25581 leader_election.cc:290] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 election: Requested vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657)
I20260709 13:22:35.491428 25350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768383f820c4e4cb8d8595901456ae1" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:35.492209 25350 raft_consensus.cc:3060] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.493302 25424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768383f820c4e4cb8d8595901456ae1" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:35.494009 25424 raft_consensus.cc:3060] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.500335 25350 raft_consensus.cc:2468] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 1.
I20260709 13:22:35.502140 25424 raft_consensus.cc:2468] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 1.
I20260709 13:22:35.502681 25465 leader_election.cc:304] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:35.503664 25581 raft_consensus.cc:2804] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:35.504458 25581 raft_consensus.cc:697] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Becoming Leader. State: Replica: a724fa0654fa4809b29444f707a88767, State: Running, Role: LEADER
I20260709 13:22:35.505534 25581 consensus_queue.cc:237] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.516259 25556 consensus_queue.cc:1048] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:35.516533 25585 consensus_queue.cc:1048] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:35.516741 25248 catalog_manager.cc:5697] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 reported cstate change: term changed from 0 to 1, leader changed from <none> to a724fa0654fa4809b29444f707a88767 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "a724fa0654fa4809b29444f707a88767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:35.547585 25576 consensus_queue.cc:1048] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:35.638803 25576 consensus_queue.cc:1048] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:35.650748 25576 consensus_queue.cc:1048] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:35.703997 25576 consensus_queue.cc:1048] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:35.709432 25425 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "01c94152f0c44421b2c2e3560785b3e6"
dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
mode: GRACEFUL
new_leader_uuid: "01445466116342929df00ee716f54e3f"
 from {username='slave'} at 127.0.0.1:35940
I20260709 13:22:35.710108 25425 raft_consensus.cc:606] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Received request to transfer leadership to TS 01445466116342929df00ee716f54e3f
I20260709 13:22:35.715853 25576 consensus_queue.cc:1048] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:35.719116 25425 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30912c2e960f4e5c85dfe927e98a1e9f"
dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
mode: GRACEFUL
new_leader_uuid: "01445466116342929df00ee716f54e3f"
 from {username='slave'} at 127.0.0.1:35940
I20260709 13:22:35.719734 25425 raft_consensus.cc:606] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Received request to transfer leadership to TS 01445466116342929df00ee716f54e3f
I20260709 13:22:35.733186 25576 raft_consensus.cc:993] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6: : Instructing follower 01445466116342929df00ee716f54e3f to start an election
I20260709 13:22:35.733726 25576 raft_consensus.cc:1081] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Signalling peer 01445466116342929df00ee716f54e3f to start an election
I20260709 13:22:35.734274 25425 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5fead9ecfeb04d0d94509484c7234118"
dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
mode: GRACEFUL
new_leader_uuid: "01445466116342929df00ee716f54e3f"
 from {username='slave'} at 127.0.0.1:35940
I20260709 13:22:35.734918 25425 raft_consensus.cc:606] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Received request to transfer leadership to TS 01445466116342929df00ee716f54e3f
I20260709 13:22:35.736662 25350 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "01c94152f0c44421b2c2e3560785b3e6"
dest_uuid: "01445466116342929df00ee716f54e3f"
 from {username='slave'} at 127.0.0.1:35476
I20260709 13:22:35.737372 25350 raft_consensus.cc:493] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:35.737744 25350 raft_consensus.cc:3060] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.744262 25350 raft_consensus.cc:515] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } } peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } }
I20260709 13:22:35.746052 25350 leader_election.cc:290] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 2 election: Requested vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:35.747084 25425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c94152f0c44421b2c2e3560785b3e6" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:35.747243 25498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c94152f0c44421b2c2e3560785b3e6" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:35.748044 25425 raft_consensus.cc:3055] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:35.748121 25498 raft_consensus.cc:3060] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.748445 25425 raft_consensus.cc:740] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Running, Role: LEADER
I20260709 13:22:35.749473 25425 consensus_queue.cc:260] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } } peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } }
I20260709 13:22:35.750474 25425 raft_consensus.cc:3060] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.756309 25498 raft_consensus.cc:2468] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 2.
I20260709 13:22:35.757645 25315 leader_election.cc:304] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:35.758426 25580 raft_consensus.cc:2804] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:35.758769 25425 raft_consensus.cc:2468] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 2.
I20260709 13:22:35.758941 25580 raft_consensus.cc:697] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [term 2 LEADER]: Becoming Leader. State: Replica: 01445466116342929df00ee716f54e3f, State: Running, Role: LEADER
I20260709 13:22:35.760356 25580 consensus_queue.cc:237] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } } peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } }
I20260709 13:22:35.761255 25498 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1cefd5d18235480d9e47c7bf0377e4f0"
dest_uuid: "a724fa0654fa4809b29444f707a88767"
mode: GRACEFUL
new_leader_uuid: "1c893d888632407db50d5d53a2f4edb6"
 from {username='slave'} at 127.0.0.1:36232
I20260709 13:22:35.761835 25498 raft_consensus.cc:606] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Received request to transfer leadership to TS 1c893d888632407db50d5d53a2f4edb6
I20260709 13:22:35.766803 25548 consensus_queue.cc:1048] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:35.767445 25249 catalog_manager.cc:5697] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f reported cstate change: term changed from 1 to 2, leader changed from 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66) to 01445466116342929df00ee716f54e3f (127.14.23.65). New cstate: current_term: 2 leader_uuid: "01445466116342929df00ee716f54e3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:35.770588 25498 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0768383f820c4e4cb8d8595901456ae1"
dest_uuid: "a724fa0654fa4809b29444f707a88767"
mode: GRACEFUL
new_leader_uuid: "1c893d888632407db50d5d53a2f4edb6"
 from {username='slave'} at 127.0.0.1:36232
I20260709 13:22:35.771123 25498 raft_consensus.cc:606] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Received request to transfer leadership to TS 1c893d888632407db50d5d53a2f4edb6
I20260709 13:22:35.780243 25498 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6729a5f0b78e4d148483f7b5d427304a"
dest_uuid: "a724fa0654fa4809b29444f707a88767"
mode: GRACEFUL
new_leader_uuid: "1c893d888632407db50d5d53a2f4edb6"
 from {username='slave'} at 127.0.0.1:36232
I20260709 13:22:35.780840 25498 raft_consensus.cc:606] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Received request to transfer leadership to TS 1c893d888632407db50d5d53a2f4edb6
I20260709 13:22:35.780794 25602 consensus_queue.cc:1048] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:35.783766 25548 raft_consensus.cc:993] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767: : Instructing follower 1c893d888632407db50d5d53a2f4edb6 to start an election
I20260709 13:22:35.784216 25602 raft_consensus.cc:1081] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Signalling peer 1c893d888632407db50d5d53a2f4edb6 to start an election
I20260709 13:22:35.784293 25580 consensus_queue.cc:1048] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:35.790565 25549 consensus_queue.cc:1048] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:35.794590 25423 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28a64081a0c94a92a11015ec646adf0c"
dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
mode: GRACEFUL
new_leader_uuid: "a724fa0654fa4809b29444f707a88767"
 from {username='slave'} at 127.0.0.1:35940
I20260709 13:22:35.795315 25423 raft_consensus.cc:606] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Received request to transfer leadership to TS a724fa0654fa4809b29444f707a88767
I20260709 13:22:35.798281 25425 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6729a5f0b78e4d148483f7b5d427304a"
dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
 from {username='slave'} at 127.0.0.1:35936
I20260709 13:22:35.799265 25425 raft_consensus.cc:493] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:35.799680 25425 raft_consensus.cc:3060] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.807773 25425 raft_consensus.cc:515] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.810789 25351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6729a5f0b78e4d148483f7b5d427304a" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:35.813215 25425 leader_election.cc:290] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 2 election: Requested vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:35.816228 25499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6729a5f0b78e4d148483f7b5d427304a" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:35.816955 25499 raft_consensus.cc:3055] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:35.817298 25499 raft_consensus.cc:740] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a724fa0654fa4809b29444f707a88767, State: Running, Role: LEADER
I20260709 13:22:35.818323 25499 consensus_queue.cc:260] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.819599 25499 raft_consensus.cc:3060] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.831828 25499 raft_consensus.cc:2468] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 2.
I20260709 13:22:35.833029 25390 leader_election.cc:304] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1c893d888632407db50d5d53a2f4edb6, a724fa0654fa4809b29444f707a88767; no voters: 01445466116342929df00ee716f54e3f
I20260709 13:22:35.833863 25576 raft_consensus.cc:2804] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:35.834378 25576 raft_consensus.cc:697] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6 [term 2 LEADER]: Becoming Leader. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Running, Role: LEADER
I20260709 13:22:35.835189 25576 consensus_queue.cc:237] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.842697 25249 catalog_manager.cc:5697] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6 reported cstate change: term changed from 1 to 2, leader changed from a724fa0654fa4809b29444f707a88767 (127.14.23.67) to 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "1c893d888632407db50d5d53a2f4edb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:35.900221 25602 consensus_queue.cc:1048] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:35.911733 25548 consensus_queue.cc:1048] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:35.919155 25585 raft_consensus.cc:993] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767: : Instructing follower 1c893d888632407db50d5d53a2f4edb6 to start an election
I20260709 13:22:35.919701 25548 raft_consensus.cc:1081] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Signalling peer 1c893d888632407db50d5d53a2f4edb6 to start an election
I20260709 13:22:35.921269 25425 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0768383f820c4e4cb8d8595901456ae1"
dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
 from {username='slave'} at 127.0.0.1:35936
I20260709 13:22:35.921729 25425 raft_consensus.cc:493] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:35.922052 25425 raft_consensus.cc:3060] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.928479 25425 raft_consensus.cc:515] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.930158 25425 leader_election.cc:290] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 2 election: Requested vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:35.931188 25499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768383f820c4e4cb8d8595901456ae1" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:35.931816 25499 raft_consensus.cc:3055] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:35.931705 25350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0768383f820c4e4cb8d8595901456ae1" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:35.932231 25499 raft_consensus.cc:740] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a724fa0654fa4809b29444f707a88767, State: Running, Role: LEADER
I20260709 13:22:35.932456 25350 raft_consensus.cc:3060] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.932971 25499 consensus_queue.cc:260] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.933974 25499 raft_consensus.cc:3060] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.938349 25350 raft_consensus.cc:2468] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 2.
I20260709 13:22:35.939463 25391 leader_election.cc:304] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:35.940152 25576 raft_consensus.cc:2804] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:35.940421 25499 raft_consensus.cc:2468] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 2.
I20260709 13:22:35.940562 25576 raft_consensus.cc:697] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6 [term 2 LEADER]: Becoming Leader. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Running, Role: LEADER
I20260709 13:22:35.941524 25576 consensus_queue.cc:237] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.947655 25249 catalog_manager.cc:5697] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6 reported cstate change: term changed from 1 to 2, leader changed from a724fa0654fa4809b29444f707a88767 (127.14.23.67) to 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "1c893d888632407db50d5d53a2f4edb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:35.963973 25558 raft_consensus.cc:993] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6: : Instructing follower 01445466116342929df00ee716f54e3f to start an election
I20260709 13:22:35.964530 25576 raft_consensus.cc:1081] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Signalling peer 01445466116342929df00ee716f54e3f to start an election
I20260709 13:22:35.966043 25350 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5fead9ecfeb04d0d94509484c7234118"
dest_uuid: "01445466116342929df00ee716f54e3f"
 from {username='slave'} at 127.0.0.1:35476
I20260709 13:22:35.966539 25350 raft_consensus.cc:493] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:35.966854 25350 raft_consensus.cc:3060] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.972041 25350 raft_consensus.cc:515] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.973778 25350 leader_election.cc:290] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 2 election: Requested vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:35.974649 25425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fead9ecfeb04d0d94509484c7234118" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:35.974922 25499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fead9ecfeb04d0d94509484c7234118" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:35.975279 25425 raft_consensus.cc:3055] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:35.975481 25499 raft_consensus.cc:3060] T 5fead9ecfeb04d0d94509484c7234118 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.975605 25425 raft_consensus.cc:740] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Running, Role: LEADER
I20260709 13:22:35.976449 25425 consensus_queue.cc:260] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.977453 25425 raft_consensus.cc:3060] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.980755 25499 raft_consensus.cc:2468] T 5fead9ecfeb04d0d94509484c7234118 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 2.
I20260709 13:22:35.982074 25315 leader_election.cc:304] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:35.982784 25617 raft_consensus.cc:2804] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:35.982731 25425 raft_consensus.cc:2468] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 2.
I20260709 13:22:35.983482 25617 raft_consensus.cc:697] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [term 2 LEADER]: Becoming Leader. State: Replica: 01445466116342929df00ee716f54e3f, State: Running, Role: LEADER
I20260709 13:22:35.984273 25617 consensus_queue.cc:237] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:35.990692 25249 catalog_manager.cc:5697] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f reported cstate change: term changed from 1 to 2, leader changed from 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66) to 01445466116342929df00ee716f54e3f (127.14.23.65). New cstate: current_term: 2 leader_uuid: "01445466116342929df00ee716f54e3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:36.079779 25558 raft_consensus.cc:993] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6: : Instructing follower a724fa0654fa4809b29444f707a88767 to start an election
I20260709 13:22:36.080233 25576 raft_consensus.cc:1081] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Signalling peer a724fa0654fa4809b29444f707a88767 to start an election
I20260709 13:22:36.081784 25499 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "28a64081a0c94a92a11015ec646adf0c"
dest_uuid: "a724fa0654fa4809b29444f707a88767"
 from {username='slave'} at 127.0.0.1:36214
I20260709 13:22:36.082280 25499 raft_consensus.cc:493] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:36.082573 25499 raft_consensus.cc:3060] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:36.087656 25499 raft_consensus.cc:515] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:36.089339 25499 leader_election.cc:290] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 2 election: Requested vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657)
I20260709 13:22:36.090200 25350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a64081a0c94a92a11015ec646adf0c" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:36.090493 25425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a64081a0c94a92a11015ec646adf0c" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:36.090827 25350 raft_consensus.cc:3060] T 28a64081a0c94a92a11015ec646adf0c P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:36.091063 25425 raft_consensus.cc:3055] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:36.091377 25425 raft_consensus.cc:740] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Running, Role: LEADER
I20260709 13:22:36.091988 25425 consensus_queue.cc:260] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:36.092936 25425 raft_consensus.cc:3060] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:36.096484 25350 raft_consensus.cc:2468] T 28a64081a0c94a92a11015ec646adf0c P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 2.
I20260709 13:22:36.097590 25465 leader_election.cc:304] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:36.098508 25585 raft_consensus.cc:2804] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:36.098805 25581 raft_consensus.cc:993] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767: : Instructing follower 1c893d888632407db50d5d53a2f4edb6 to start an election
I20260709 13:22:36.099020 25585 raft_consensus.cc:697] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 [term 2 LEADER]: Becoming Leader. State: Replica: a724fa0654fa4809b29444f707a88767, State: Running, Role: LEADER
I20260709 13:22:36.099370 25425 raft_consensus.cc:2468] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 2.
I20260709 13:22:36.099843 25548 raft_consensus.cc:1081] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Signalling peer 1c893d888632407db50d5d53a2f4edb6 to start an election
I20260709 13:22:36.100006 25585 consensus_queue.cc:237] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:36.102135 25425 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1cefd5d18235480d9e47c7bf0377e4f0"
dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
 from {username='slave'} at 127.0.0.1:35936
I20260709 13:22:36.102662 25425 raft_consensus.cc:493] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:36.102990 25425 raft_consensus.cc:3060] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:36.106913 25249 catalog_manager.cc:5697] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 reported cstate change: term changed from 1 to 2, leader changed from 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66) to a724fa0654fa4809b29444f707a88767 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "a724fa0654fa4809b29444f707a88767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:36.108600 25425 raft_consensus.cc:515] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:36.110926 25425 leader_election.cc:290] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 2 election: Requested vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:36.111586 25350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cefd5d18235480d9e47c7bf0377e4f0" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:36.112012 25558 raft_consensus.cc:993] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6: : Instructing follower 01445466116342929df00ee716f54e3f to start an election
I20260709 13:22:36.112205 25350 raft_consensus.cc:3060] T 1cefd5d18235480d9e47c7bf0377e4f0 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:36.112928 25576 raft_consensus.cc:1081] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Signalling peer 01445466116342929df00ee716f54e3f to start an election
I20260709 13:22:36.112321 25499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cefd5d18235480d9e47c7bf0377e4f0" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:36.113899 25499 raft_consensus.cc:3055] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:36.114274 25499 raft_consensus.cc:740] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a724fa0654fa4809b29444f707a88767, State: Running, Role: LEADER
I20260709 13:22:36.114648 25351 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "30912c2e960f4e5c85dfe927e98a1e9f"
dest_uuid: "01445466116342929df00ee716f54e3f"
 from {username='slave'} at 127.0.0.1:35476
I20260709 13:22:36.115947 25351 raft_consensus.cc:493] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:36.116351 25351 raft_consensus.cc:3060] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:36.116495 25499 consensus_queue.cc:260] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:36.117868 25499 raft_consensus.cc:3060] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:36.119029 25350 raft_consensus.cc:2468] T 1cefd5d18235480d9e47c7bf0377e4f0 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 2.
I20260709 13:22:36.120072 25391 leader_election.cc:304] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:36.120720 25576 raft_consensus.cc:2804] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:36.121114 25576 raft_consensus.cc:697] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6 [term 2 LEADER]: Becoming Leader. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Running, Role: LEADER
I20260709 13:22:36.121892 25576 consensus_queue.cc:237] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:36.123802 25351 raft_consensus.cc:515] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:36.126181 25351 leader_election.cc:290] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 2 election: Requested vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:36.126816 25425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30912c2e960f4e5c85dfe927e98a1e9f" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:36.127257 25497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30912c2e960f4e5c85dfe927e98a1e9f" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:36.127485 25425 raft_consensus.cc:3055] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:36.127839 25425 raft_consensus.cc:740] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Running, Role: LEADER
I20260709 13:22:36.127862 25497 raft_consensus.cc:3060] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:36.128690 25425 consensus_queue.cc:260] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:36.129655 25425 raft_consensus.cc:3060] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:36.128998 25249 catalog_manager.cc:5697] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6 reported cstate change: term changed from 1 to 2, leader changed from a724fa0654fa4809b29444f707a88767 (127.14.23.67) to 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "1c893d888632407db50d5d53a2f4edb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:36.134204 25499 raft_consensus.cc:2468] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 2.
I20260709 13:22:36.136215 25425 raft_consensus.cc:2468] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 2.
I20260709 13:22:36.136510 25497 raft_consensus.cc:2468] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 2.
I20260709 13:22:36.137537 25314 leader_election.cc:304] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:36.138607 25617 raft_consensus.cc:2804] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:36.139015 25617 raft_consensus.cc:697] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f [term 2 LEADER]: Becoming Leader. State: Replica: 01445466116342929df00ee716f54e3f, State: Running, Role: LEADER
I20260709 13:22:36.139885 25617 consensus_queue.cc:237] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:36.146194 25249 catalog_manager.cc:5697] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f reported cstate change: term changed from 1 to 2, leader changed from 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66) to 01445466116342929df00ee716f54e3f (127.14.23.65). New cstate: current_term: 2 leader_uuid: "01445466116342929df00ee716f54e3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:36.167869 25425 raft_consensus.cc:1275] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Refusing update from remote peer 01445466116342929df00ee716f54e3f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.169265 25617 consensus_queue.cc:1048] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:36.177538 25497 raft_consensus.cc:1275] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Refusing update from remote peer 01445466116342929df00ee716f54e3f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.179316 25617 consensus_queue.cc:1048] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:36.268296 25351 raft_consensus.cc:3060] T 6729a5f0b78e4d148483f7b5d427304a P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:36.273881 25351 raft_consensus.cc:1275] T 6729a5f0b78e4d148483f7b5d427304a P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Refusing update from remote peer 1c893d888632407db50d5d53a2f4edb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.275381 25576 consensus_queue.cc:1048] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:36.286167 25497 raft_consensus.cc:1275] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Refusing update from remote peer 1c893d888632407db50d5d53a2f4edb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.287956 25620 consensus_queue.cc:1048] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:36.330564 25497 raft_consensus.cc:1275] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Refusing update from remote peer 1c893d888632407db50d5d53a2f4edb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.333457 25620 consensus_queue.cc:1048] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:36.343530 25351 raft_consensus.cc:1275] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Refusing update from remote peer 1c893d888632407db50d5d53a2f4edb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.347991 25558 consensus_queue.cc:1048] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:36.401311 25497 raft_consensus.cc:1275] T 5fead9ecfeb04d0d94509484c7234118 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Refusing update from remote peer 01445466116342929df00ee716f54e3f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.402616 25617 consensus_queue.cc:1048] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:36.410802 25425 raft_consensus.cc:1275] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Refusing update from remote peer 01445466116342929df00ee716f54e3f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.412135 25617 consensus_queue.cc:1048] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:36.521209 25425 raft_consensus.cc:1275] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Refusing update from remote peer a724fa0654fa4809b29444f707a88767: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.522740 25585 consensus_queue.cc:1048] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:36.530460 25351 raft_consensus.cc:1275] T 28a64081a0c94a92a11015ec646adf0c P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Refusing update from remote peer a724fa0654fa4809b29444f707a88767: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.531924 25548 consensus_queue.cc:1048] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:36.560371 25425 raft_consensus.cc:1275] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Refusing update from remote peer 01445466116342929df00ee716f54e3f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.561561 25617 consensus_queue.cc:1048] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:36.564848 25497 raft_consensus.cc:1275] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Refusing update from remote peer 01445466116342929df00ee716f54e3f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.566250 25617 consensus_queue.cc:1048] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:36.680879 25350 raft_consensus.cc:1275] T 1cefd5d18235480d9e47c7bf0377e4f0 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Refusing update from remote peer 1c893d888632407db50d5d53a2f4edb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.682178 25576 consensus_queue.cc:1048] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:36.690651 25497 raft_consensus.cc:1275] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Refusing update from remote peer 1c893d888632407db50d5d53a2f4edb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.692451 25620 consensus_queue.cc:1048] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:38.823962 25247 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33602:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:22:38.826503 25247 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:22:38.878993 25335 tablet_service.cc:1511] Processing CreateTablet for tablet ae34bd035b404eba838fadb2f2d50934 (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:38.880059 25416 tablet_service.cc:1511] Processing CreateTablet for tablet ae34bd035b404eba838fadb2f2d50934 (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:38.880705 25335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae34bd035b404eba838fadb2f2d50934. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.881762 25416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae34bd035b404eba838fadb2f2d50934. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.881773 25485 tablet_service.cc:1511] Processing CreateTablet for tablet ae34bd035b404eba838fadb2f2d50934 (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:38.883402 25485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae34bd035b404eba838fadb2f2d50934. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.884545 25410 tablet_service.cc:1511] Processing CreateTablet for tablet 0526fea4b4214a128677c8f013ea8cc8 (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:38.886005 25410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0526fea4b4214a128677c8f013ea8cc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.887621 25490 tablet_service.cc:1511] Processing CreateTablet for tablet 0526fea4b4214a128677c8f013ea8cc8 (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:38.889040 25490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0526fea4b4214a128677c8f013ea8cc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.891634 25334 tablet_service.cc:1511] Processing CreateTablet for tablet 0526fea4b4214a128677c8f013ea8cc8 (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:38.892000 25339 tablet_service.cc:1511] Processing CreateTablet for tablet 9115d271796f4b18a68999f53eb80e57 (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:38.893187 25334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0526fea4b4214a128677c8f013ea8cc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.894109 25413 tablet_service.cc:1511] Processing CreateTablet for tablet 70e830e6def34b06a3e805e4e9bf0a44 (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:38.895581 25413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70e830e6def34b06a3e805e4e9bf0a44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.895776 25488 tablet_service.cc:1511] Processing CreateTablet for tablet 9115d271796f4b18a68999f53eb80e57 (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:38.897159 25488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9115d271796f4b18a68999f53eb80e57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.897295 25340 tablet_service.cc:1511] Processing CreateTablet for tablet 70e830e6def34b06a3e805e4e9bf0a44 (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:38.898675 25339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9115d271796f4b18a68999f53eb80e57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.894109 25414 tablet_service.cc:1511] Processing CreateTablet for tablet 9115d271796f4b18a68999f53eb80e57 (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:38.902100 25414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9115d271796f4b18a68999f53eb80e57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.903914 25340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70e830e6def34b06a3e805e4e9bf0a44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.934396 25486 tablet_service.cc:1511] Processing CreateTablet for tablet 70e830e6def34b06a3e805e4e9bf0a44 (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:38.935936 25486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70e830e6def34b06a3e805e4e9bf0a44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.936223 25341 tablet_service.cc:1511] Processing CreateTablet for tablet db4c952bd42842479c6e6cc4ac7d2eac (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:38.937599 25341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db4c952bd42842479c6e6cc4ac7d2eac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.943270 25415 tablet_service.cc:1511] Processing CreateTablet for tablet db4c952bd42842479c6e6cc4ac7d2eac (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:38.944777 25415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db4c952bd42842479c6e6cc4ac7d2eac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.948088 25338 tablet_service.cc:1511] Processing CreateTablet for tablet acbfd09f2a2f45c7ac9aca07ee71dc7e (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:38.948763 25409 tablet_service.cc:1511] Processing CreateTablet for tablet acbfd09f2a2f45c7ac9aca07ee71dc7e (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:38.950219 25409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acbfd09f2a2f45c7ac9aca07ee71dc7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.951900 25338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acbfd09f2a2f45c7ac9aca07ee71dc7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.948212 25489 tablet_service.cc:1511] Processing CreateTablet for tablet db4c952bd42842479c6e6cc4ac7d2eac (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:38.955945 25489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db4c952bd42842479c6e6cc4ac7d2eac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.974290 25487 tablet_service.cc:1511] Processing CreateTablet for tablet acbfd09f2a2f45c7ac9aca07ee71dc7e (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:38.975759 25487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acbfd09f2a2f45c7ac9aca07ee71dc7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.977788 25413 tablet_service.cc:1511] Processing CreateTablet for tablet b9f4e02a32cd4b5c955b4f865d872842 (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:38.979244 25413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9f4e02a32cd4b5c955b4f865d872842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.979960 25484 tablet_service.cc:1511] Processing CreateTablet for tablet b9f4e02a32cd4b5c955b4f865d872842 (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:38.981364 25484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9f4e02a32cd4b5c955b4f865d872842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.988734 25339 tablet_service.cc:1511] Processing CreateTablet for tablet d08fd06d7e974780955efa1d052b5da4 (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:38.990195 25339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d08fd06d7e974780955efa1d052b5da4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:22:38.994204 25297 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:38.990274 25638 tablet_bootstrap.cc:492] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f: Bootstrap starting.
I20260709 13:22:38.995774 25340 tablet_service.cc:1511] Processing CreateTablet for tablet b9f4e02a32cd4b5c955b4f865d872842 (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:38.997102 25416 tablet_service.cc:1511] Processing CreateTablet for tablet d08fd06d7e974780955efa1d052b5da4 (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:38.998490 25416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d08fd06d7e974780955efa1d052b5da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:39.001606 25638 tablet_bootstrap.cc:654] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:38.997160 25340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9f4e02a32cd4b5c955b4f865d872842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:39.006456 25335 tablet_service.cc:1511] Processing CreateTablet for tablet ddfd39813ccd423d81843e1bc36a4a76 (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:39.008075 25414 tablet_service.cc:1511] Processing CreateTablet for tablet ddfd39813ccd423d81843e1bc36a4a76 (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:39.009191 25490 tablet_service.cc:1511] Processing CreateTablet for tablet d08fd06d7e974780955efa1d052b5da4 (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:39.016040 25490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d08fd06d7e974780955efa1d052b5da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:39.013063 25488 tablet_service.cc:1511] Processing CreateTablet for tablet ddfd39813ccd423d81843e1bc36a4a76 (DEFAULT_TABLE table=soft_deleted_table [id=59210192c00845278c200bb77ef719a3]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:39.015086 25414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddfd39813ccd423d81843e1bc36a4a76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:39.015060 25335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddfd39813ccd423d81843e1bc36a4a76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:39.034534 25488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddfd39813ccd423d81843e1bc36a4a76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:39.074256 25640 tablet_bootstrap.cc:492] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6: Bootstrap starting.
I20260709 13:22:39.076105 25639 tablet_bootstrap.cc:492] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767: Bootstrap starting.
I20260709 13:22:39.082787 25640 tablet_bootstrap.cc:654] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.090029 25638 tablet_bootstrap.cc:492] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f: No bootstrap required, opened a new log
I20260709 13:22:39.090603 25638 ts_tablet_manager.cc:1403] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f: Time spent bootstrapping tablet: real 0.101s	user 0.032s	sys 0.036s
I20260709 13:22:39.091168 25639 tablet_bootstrap.cc:654] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.093623 25638 raft_consensus.cc:359] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.098420 25638 raft_consensus.cc:385] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.098747 25638 raft_consensus.cc:740] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.099586 25638 consensus_queue.cc:260] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.102171 25638 ts_tablet_manager.cc:1434] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:22:39.103204 25638 tablet_bootstrap.cc:492] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f: Bootstrap starting.
I20260709 13:22:39.109892 25638 tablet_bootstrap.cc:654] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.115964 25638 tablet_bootstrap.cc:492] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f: No bootstrap required, opened a new log
I20260709 13:22:39.116402 25639 tablet_bootstrap.cc:492] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767: No bootstrap required, opened a new log
I20260709 13:22:39.116478 25638 ts_tablet_manager.cc:1403] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f: Time spent bootstrapping tablet: real 0.014s	user 0.006s	sys 0.004s
I20260709 13:22:39.117034 25639 ts_tablet_manager.cc:1403] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767: Time spent bootstrapping tablet: real 0.041s	user 0.011s	sys 0.010s
I20260709 13:22:39.118111 25640 tablet_bootstrap.cc:492] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6: No bootstrap required, opened a new log
I20260709 13:22:39.118638 25640 ts_tablet_manager.cc:1403] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6: Time spent bootstrapping tablet: real 0.045s	user 0.011s	sys 0.009s
I20260709 13:22:39.120127 25638 raft_consensus.cc:359] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.120399 25639 raft_consensus.cc:359] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.120963 25638 raft_consensus.cc:385] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.121146 25639 raft_consensus.cc:385] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.121495 25638 raft_consensus.cc:740] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.121549 25639 raft_consensus.cc:740] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a724fa0654fa4809b29444f707a88767, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.122527 25640 raft_consensus.cc:359] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.122583 25638 consensus_queue.cc:260] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.123395 25640 raft_consensus.cc:385] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.122597 25639 consensus_queue.cc:260] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.123728 25640 raft_consensus.cc:740] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.124514 25640 consensus_queue.cc:260] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.125785 25639 ts_tablet_manager.cc:1434] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 13:22:39.127204 25640 ts_tablet_manager.cc:1434] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 13:22:39.128332 25640 tablet_bootstrap.cc:492] T 9115d271796f4b18a68999f53eb80e57 P 1c893d888632407db50d5d53a2f4edb6: Bootstrap starting.
I20260709 13:22:39.129343 25638 ts_tablet_manager.cc:1434] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:22:39.130509 25638 tablet_bootstrap.cc:492] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f: Bootstrap starting.
I20260709 13:22:39.135183 25640 tablet_bootstrap.cc:654] T 9115d271796f4b18a68999f53eb80e57 P 1c893d888632407db50d5d53a2f4edb6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.136953 25638 tablet_bootstrap.cc:654] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.158504 25639 tablet_bootstrap.cc:492] T ae34bd035b404eba838fadb2f2d50934 P a724fa0654fa4809b29444f707a88767: Bootstrap starting.
I20260709 13:22:39.170915 25639 tablet_bootstrap.cc:654] T ae34bd035b404eba838fadb2f2d50934 P a724fa0654fa4809b29444f707a88767: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.177824 25640 tablet_bootstrap.cc:492] T 9115d271796f4b18a68999f53eb80e57 P 1c893d888632407db50d5d53a2f4edb6: No bootstrap required, opened a new log
I20260709 13:22:39.178097 25638 tablet_bootstrap.cc:492] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f: No bootstrap required, opened a new log
I20260709 13:22:39.178339 25640 ts_tablet_manager.cc:1403] T 9115d271796f4b18a68999f53eb80e57 P 1c893d888632407db50d5d53a2f4edb6: Time spent bootstrapping tablet: real 0.050s	user 0.027s	sys 0.021s
I20260709 13:22:39.178638 25638 ts_tablet_manager.cc:1403] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f: Time spent bootstrapping tablet: real 0.048s	user 0.023s	sys 0.022s
I20260709 13:22:39.179922 25639 tablet_bootstrap.cc:492] T ae34bd035b404eba838fadb2f2d50934 P a724fa0654fa4809b29444f707a88767: No bootstrap required, opened a new log
I20260709 13:22:39.180450 25639 ts_tablet_manager.cc:1403] T ae34bd035b404eba838fadb2f2d50934 P a724fa0654fa4809b29444f707a88767: Time spent bootstrapping tablet: real 0.054s	user 0.009s	sys 0.005s
I20260709 13:22:39.181164 25638 raft_consensus.cc:359] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.181752 25638 raft_consensus.cc:385] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.182082 25638 raft_consensus.cc:740] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.181571 25640 raft_consensus.cc:359] T 9115d271796f4b18a68999f53eb80e57 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.182461 25640 raft_consensus.cc:385] T 9115d271796f4b18a68999f53eb80e57 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.182772 25640 raft_consensus.cc:740] T 9115d271796f4b18a68999f53eb80e57 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.182729 25638 consensus_queue.cc:260] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.183634 25639 raft_consensus.cc:359] T ae34bd035b404eba838fadb2f2d50934 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.183657 25640 consensus_queue.cc:260] T 9115d271796f4b18a68999f53eb80e57 P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.185299 25638 ts_tablet_manager.cc:1434] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:22:39.186223 25640 ts_tablet_manager.cc:1434] T 9115d271796f4b18a68999f53eb80e57 P 1c893d888632407db50d5d53a2f4edb6: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:22:39.186369 25638 tablet_bootstrap.cc:492] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f: Bootstrap starting.
I20260709 13:22:39.187201 25640 tablet_bootstrap.cc:492] T 70e830e6def34b06a3e805e4e9bf0a44 P 1c893d888632407db50d5d53a2f4edb6: Bootstrap starting.
I20260709 13:22:39.184517 25639 raft_consensus.cc:385] T ae34bd035b404eba838fadb2f2d50934 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.188030 25639 raft_consensus.cc:740] T ae34bd035b404eba838fadb2f2d50934 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a724fa0654fa4809b29444f707a88767, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.188830 25639 consensus_queue.cc:260] T ae34bd035b404eba838fadb2f2d50934 P a724fa0654fa4809b29444f707a88767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.194509 25640 tablet_bootstrap.cc:654] T 70e830e6def34b06a3e805e4e9bf0a44 P 1c893d888632407db50d5d53a2f4edb6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.195933 25638 tablet_bootstrap.cc:654] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.195961 25639 ts_tablet_manager.cc:1434] T ae34bd035b404eba838fadb2f2d50934 P a724fa0654fa4809b29444f707a88767: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.001s
I20260709 13:22:39.197252 25639 tablet_bootstrap.cc:492] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767: Bootstrap starting.
I20260709 13:22:39.204607 25639 tablet_bootstrap.cc:654] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.212061 25638 tablet_bootstrap.cc:492] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f: No bootstrap required, opened a new log
I20260709 13:22:39.212718 25638 ts_tablet_manager.cc:1403] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.005s
I20260709 13:22:39.219491 25638 raft_consensus.cc:359] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.220086 25638 raft_consensus.cc:385] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.220386 25638 raft_consensus.cc:740] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.221680 25638 consensus_queue.cc:260] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.223894 25638 ts_tablet_manager.cc:1434] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:22:39.225142 25638 tablet_bootstrap.cc:492] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f: Bootstrap starting.
I20260709 13:22:39.227912 25640 tablet_bootstrap.cc:492] T 70e830e6def34b06a3e805e4e9bf0a44 P 1c893d888632407db50d5d53a2f4edb6: No bootstrap required, opened a new log
I20260709 13:22:39.228685 25640 ts_tablet_manager.cc:1403] T 70e830e6def34b06a3e805e4e9bf0a44 P 1c893d888632407db50d5d53a2f4edb6: Time spent bootstrapping tablet: real 0.042s	user 0.007s	sys 0.028s
I20260709 13:22:39.232232 25640 raft_consensus.cc:359] T 70e830e6def34b06a3e805e4e9bf0a44 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.232950 25640 raft_consensus.cc:385] T 70e830e6def34b06a3e805e4e9bf0a44 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.233255 25640 raft_consensus.cc:740] T 70e830e6def34b06a3e805e4e9bf0a44 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.233255 25639 tablet_bootstrap.cc:492] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767: No bootstrap required, opened a new log
I20260709 13:22:39.234050 25639 ts_tablet_manager.cc:1403] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767: Time spent bootstrapping tablet: real 0.037s	user 0.010s	sys 0.009s
I20260709 13:22:39.234099 25640 consensus_queue.cc:260] T 70e830e6def34b06a3e805e4e9bf0a44 P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.236667 25640 ts_tablet_manager.cc:1434] T 70e830e6def34b06a3e805e4e9bf0a44 P 1c893d888632407db50d5d53a2f4edb6: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:22:39.237910 25640 tablet_bootstrap.cc:492] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6: Bootstrap starting.
I20260709 13:22:39.242650 25627 raft_consensus.cc:493] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.243346 25638 tablet_bootstrap.cc:654] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.243228 25627 raft_consensus.cc:515] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.244818 25640 tablet_bootstrap.cc:654] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.247298 25497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70e830e6def34b06a3e805e4e9bf0a44" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767" is_pre_election: true
W20260709 13:22:39.249425 25315 leader_election.cc:343] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:39.243216 25639 raft_consensus.cc:359] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.251094 25639 raft_consensus.cc:385] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.251471 25639 raft_consensus.cc:740] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a724fa0654fa4809b29444f707a88767, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.252352 25639 consensus_queue.cc:260] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.251223 25425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70e830e6def34b06a3e805e4e9bf0a44" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6" is_pre_election: true
I20260709 13:22:39.253640 25425 raft_consensus.cc:2468] T 70e830e6def34b06a3e805e4e9bf0a44 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 0.
I20260709 13:22:39.255321 25314 leader_election.cc:304] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: a724fa0654fa4809b29444f707a88767
I20260709 13:22:39.256592 25627 leader_election.cc:290] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:39.257181 25627 raft_consensus.cc:2804] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:39.255523 25639 ts_tablet_manager.cc:1434] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.002s
I20260709 13:22:39.258524 25639 tablet_bootstrap.cc:492] T 70e830e6def34b06a3e805e4e9bf0a44 P a724fa0654fa4809b29444f707a88767: Bootstrap starting.
I20260709 13:22:39.257524 25627 raft_consensus.cc:493] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:39.259215 25640 tablet_bootstrap.cc:492] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6: No bootstrap required, opened a new log
I20260709 13:22:39.259150 25627 raft_consensus.cc:3060] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.259833 25640 ts_tablet_manager.cc:1403] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.008s
I20260709 13:22:39.262980 25640 raft_consensus.cc:359] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.263710 25640 raft_consensus.cc:385] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.264005 25640 raft_consensus.cc:740] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.264725 25640 consensus_queue.cc:260] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.267175 25640 ts_tablet_manager.cc:1434] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:22:39.268296 25640 tablet_bootstrap.cc:492] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6: Bootstrap starting.
I20260709 13:22:39.273057 25639 tablet_bootstrap.cc:654] T 70e830e6def34b06a3e805e4e9bf0a44 P a724fa0654fa4809b29444f707a88767: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.272668 25627 raft_consensus.cc:515] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.275655 25425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70e830e6def34b06a3e805e4e9bf0a44" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:39.276259 25425 raft_consensus.cc:3060] T 70e830e6def34b06a3e805e4e9bf0a44 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.277856 25638 tablet_bootstrap.cc:492] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f: No bootstrap required, opened a new log
I20260709 13:22:39.278450 25638 ts_tablet_manager.cc:1403] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f: Time spent bootstrapping tablet: real 0.054s	user 0.018s	sys 0.000s
I20260709 13:22:39.279233 25627 leader_election.cc:290] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 election: Requested vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:39.280330 25497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70e830e6def34b06a3e805e4e9bf0a44" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:39.280695 25640 tablet_bootstrap.cc:654] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.281445 25638 raft_consensus.cc:359] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.283933 25638 raft_consensus.cc:385] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:22:39.283996 25315 leader_election.cc:343] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:39.284312 25638 raft_consensus.cc:740] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.285151 25638 consensus_queue.cc:260] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.286319 25425 raft_consensus.cc:2468] T 70e830e6def34b06a3e805e4e9bf0a44 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 1.
I20260709 13:22:39.287860 25638 ts_tablet_manager.cc:1434] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:22:39.288036 25639 tablet_bootstrap.cc:492] T 70e830e6def34b06a3e805e4e9bf0a44 P a724fa0654fa4809b29444f707a88767: No bootstrap required, opened a new log
I20260709 13:22:39.288573 25639 ts_tablet_manager.cc:1403] T 70e830e6def34b06a3e805e4e9bf0a44 P a724fa0654fa4809b29444f707a88767: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.000s
I20260709 13:22:39.289021 25314 leader_election.cc:304] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: a724fa0654fa4809b29444f707a88767
I20260709 13:22:39.288981 25638 tablet_bootstrap.cc:492] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f: Bootstrap starting.
I20260709 13:22:39.290313 25627 raft_consensus.cc:2804] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:39.290794 25627 raft_consensus.cc:697] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Becoming Leader. State: Replica: 01445466116342929df00ee716f54e3f, State: Running, Role: LEADER
I20260709 13:22:39.290755 25558 raft_consensus.cc:493] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.291546 25558 raft_consensus.cc:515] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.291706 25639 raft_consensus.cc:359] T 70e830e6def34b06a3e805e4e9bf0a44 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.291807 25627 consensus_queue.cc:237] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.292560 25639 raft_consensus.cc:385] T 70e830e6def34b06a3e805e4e9bf0a44 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.292886 25639 raft_consensus.cc:740] T 70e830e6def34b06a3e805e4e9bf0a44 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a724fa0654fa4809b29444f707a88767, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.293777 25558 leader_election.cc:290] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:39.293634 25639 consensus_queue.cc:260] T 70e830e6def34b06a3e805e4e9bf0a44 P a724fa0654fa4809b29444f707a88767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.295480 25350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0526fea4b4214a128677c8f013ea8cc8" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f" is_pre_election: true
I20260709 13:22:39.296056 25639 ts_tablet_manager.cc:1434] T 70e830e6def34b06a3e805e4e9bf0a44 P a724fa0654fa4809b29444f707a88767: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:22:39.296634 25497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0526fea4b4214a128677c8f013ea8cc8" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767" is_pre_election: true
I20260709 13:22:39.297186 25639 tablet_bootstrap.cc:492] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767: Bootstrap starting.
I20260709 13:22:39.296170 25350 raft_consensus.cc:2468] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 0.
I20260709 13:22:39.298911 25391 leader_election.cc:304] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:39.299924 25558 raft_consensus.cc:2804] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:39.300371 25558 raft_consensus.cc:493] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:39.300758 25558 raft_consensus.cc:3060] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.302775 25638 tablet_bootstrap.cc:654] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.304847 25639 tablet_bootstrap.cc:654] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.308084 25640 tablet_bootstrap.cc:492] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6: No bootstrap required, opened a new log
I20260709 13:22:39.308714 25640 ts_tablet_manager.cc:1403] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6: Time spent bootstrapping tablet: real 0.041s	user 0.010s	sys 0.000s
I20260709 13:22:39.297235 25497 raft_consensus.cc:2468] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 0.
I20260709 13:22:39.312193 25640 raft_consensus.cc:359] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.313084 25640 raft_consensus.cc:385] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.313417 25640 raft_consensus.cc:740] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.314335 25640 consensus_queue.cc:260] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.317562 25640 ts_tablet_manager.cc:1434] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:22:39.318873 25640 tablet_bootstrap.cc:492] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6: Bootstrap starting.
I20260709 13:22:39.323927 25245 catalog_manager.cc:5697] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f reported cstate change: term changed from 0 to 1, leader changed from <none> to 01445466116342929df00ee716f54e3f (127.14.23.65). New cstate: current_term: 1 leader_uuid: "01445466116342929df00ee716f54e3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:39.313468 25558 raft_consensus.cc:515] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.327239 25647 raft_consensus.cc:493] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.327927 25647 raft_consensus.cc:515] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.329759 25558 leader_election.cc:290] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 election: Requested vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:39.330294 25647 leader_election.cc:290] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:39.331426 25499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae34bd035b404eba838fadb2f2d50934" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767" is_pre_election: true
I20260709 13:22:39.332465 25350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0526fea4b4214a128677c8f013ea8cc8" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:39.333072 25350 raft_consensus.cc:3060] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.332123 25499 raft_consensus.cc:2468] T ae34bd035b404eba838fadb2f2d50934 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 0.
I20260709 13:22:39.338124 25390 leader_election.cc:304] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c893d888632407db50d5d53a2f4edb6, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:39.338750 25351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae34bd035b404eba838fadb2f2d50934" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f" is_pre_election: true
I20260709 13:22:39.339061 25647 raft_consensus.cc:2804] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:39.339589 25647 raft_consensus.cc:493] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:39.339967 25647 raft_consensus.cc:3060] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.340138 25626 raft_consensus.cc:493] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.340672 25626 raft_consensus.cc:515] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.344012 25626 leader_election.cc:290] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657)
I20260709 13:22:39.348224 25348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9115d271796f4b18a68999f53eb80e57" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f" is_pre_election: true
I20260709 13:22:39.348227 25497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0526fea4b4214a128677c8f013ea8cc8" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:39.349085 25348 raft_consensus.cc:2468] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 0.
I20260709 13:22:39.349653 25497 raft_consensus.cc:3060] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.350546 25465 leader_election.cc:304] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:39.351889 25647 raft_consensus.cc:515] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.366582 25626 raft_consensus.cc:2804] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:39.367055 25626 raft_consensus.cc:493] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:39.367398 25425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9115d271796f4b18a68999f53eb80e57" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6" is_pre_election: true
I20260709 13:22:39.367852 25647 leader_election.cc:290] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 election: Requested vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:39.353498 25640 tablet_bootstrap.cc:654] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.368080 25425 raft_consensus.cc:2468] T 9115d271796f4b18a68999f53eb80e57 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 0.
I20260709 13:22:39.367480 25626 raft_consensus.cc:3060] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.369151 25348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae34bd035b404eba838fadb2f2d50934" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:39.354362 25639 tablet_bootstrap.cc:492] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767: No bootstrap required, opened a new log
I20260709 13:22:39.371294 25639 ts_tablet_manager.cc:1403] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767: Time spent bootstrapping tablet: real 0.074s	user 0.040s	sys 0.016s
I20260709 13:22:39.371369 25499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae34bd035b404eba838fadb2f2d50934" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:39.358875 25351 raft_consensus.cc:2468] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 0.
I20260709 13:22:39.373804 25499 raft_consensus.cc:3060] T ae34bd035b404eba838fadb2f2d50934 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.360194 25638 tablet_bootstrap.cc:492] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f: No bootstrap required, opened a new log
I20260709 13:22:39.375053 25638 ts_tablet_manager.cc:1403] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f: Time spent bootstrapping tablet: real 0.086s	user 0.013s	sys 0.005s
I20260709 13:22:39.378204 25638 raft_consensus.cc:359] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.379029 25638 raft_consensus.cc:385] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.379386 25638 raft_consensus.cc:740] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.379515 25350 raft_consensus.cc:2468] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 1.
I20260709 13:22:39.378551 25497 raft_consensus.cc:2468] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 1.
I20260709 13:22:39.374054 25639 raft_consensus.cc:359] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.380419 25639 raft_consensus.cc:385] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.380357 25638 consensus_queue.cc:260] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.380730 25391 leader_election.cc:304] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:39.380812 25639 raft_consensus.cc:740] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a724fa0654fa4809b29444f707a88767, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.381845 25647 raft_consensus.cc:2804] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:39.382298 25639 consensus_queue.cc:260] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.382397 25647 raft_consensus.cc:697] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Becoming Leader. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Running, Role: LEADER
I20260709 13:22:39.383512 25638 ts_tablet_manager.cc:1434] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:22:39.384684 25638 tablet_bootstrap.cc:492] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f: Bootstrap starting.
I20260709 13:22:39.386389 25640 tablet_bootstrap.cc:492] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6: No bootstrap required, opened a new log
I20260709 13:22:39.386930 25640 ts_tablet_manager.cc:1403] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6: Time spent bootstrapping tablet: real 0.068s	user 0.011s	sys 0.000s
I20260709 13:22:39.386749 25626 raft_consensus.cc:515] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.384047 25647 consensus_queue.cc:237] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.390206 25640 raft_consensus.cc:359] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.390960 25640 raft_consensus.cc:385] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.391271 25640 raft_consensus.cc:740] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.381323 25499 raft_consensus.cc:2468] T ae34bd035b404eba838fadb2f2d50934 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 1.
I20260709 13:22:39.392593 25638 tablet_bootstrap.cc:654] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.393175 25640 consensus_queue.cc:260] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.393725 25390 leader_election.cc:304] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1c893d888632407db50d5d53a2f4edb6, a724fa0654fa4809b29444f707a88767; no voters: 01445466116342929df00ee716f54e3f
I20260709 13:22:39.394938 25659 raft_consensus.cc:2804] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:39.395715 25659 raft_consensus.cc:697] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Becoming Leader. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Running, Role: LEADER
I20260709 13:22:39.396556 25659 consensus_queue.cc:237] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.399327 25350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9115d271796f4b18a68999f53eb80e57" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:39.399964 25350 raft_consensus.cc:3060] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.402766 25639 ts_tablet_manager.cc:1434] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.007s
I20260709 13:22:39.404006 25639 tablet_bootstrap.cc:492] T acbfd09f2a2f45c7ac9aca07ee71dc7e P a724fa0654fa4809b29444f707a88767: Bootstrap starting.
I20260709 13:22:39.407881 25350 raft_consensus.cc:2468] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 1.
I20260709 13:22:39.409266 25465 leader_election.cc:304] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:39.410686 25662 raft_consensus.cc:2804] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:39.402679 25640 ts_tablet_manager.cc:1434] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:22:39.413995 25425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9115d271796f4b18a68999f53eb80e57" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:39.417654 25425 raft_consensus.cc:3060] T 9115d271796f4b18a68999f53eb80e57 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.418437 25640 tablet_bootstrap.cc:492] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6: Bootstrap starting.
I20260709 13:22:39.415910 25662 raft_consensus.cc:697] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Becoming Leader. State: Replica: a724fa0654fa4809b29444f707a88767, State: Running, Role: LEADER
I20260709 13:22:39.423019 25662 consensus_queue.cc:237] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.424502 25245 catalog_manager.cc:5697] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "1c893d888632407db50d5d53a2f4edb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:39.425652 25640 tablet_bootstrap.cc:654] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.426355 25639 tablet_bootstrap.cc:654] T acbfd09f2a2f45c7ac9aca07ee71dc7e P a724fa0654fa4809b29444f707a88767: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.429311 25425 raft_consensus.cc:2468] T 9115d271796f4b18a68999f53eb80e57 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 1.
I20260709 13:22:39.411149 25626 leader_election.cc:290] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 election: Requested vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657)
I20260709 13:22:39.436582 25638 tablet_bootstrap.cc:492] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f: No bootstrap required, opened a new log
I20260709 13:22:39.437250 25638 ts_tablet_manager.cc:1403] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f: Time spent bootstrapping tablet: real 0.053s	user 0.017s	sys 0.010s
I20260709 13:22:39.440394 25638 raft_consensus.cc:359] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.441152 25638 raft_consensus.cc:385] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.441433 25638 raft_consensus.cc:740] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.444396 25639 tablet_bootstrap.cc:492] T acbfd09f2a2f45c7ac9aca07ee71dc7e P a724fa0654fa4809b29444f707a88767: No bootstrap required, opened a new log
I20260709 13:22:39.445190 25639 ts_tablet_manager.cc:1403] T acbfd09f2a2f45c7ac9aca07ee71dc7e P a724fa0654fa4809b29444f707a88767: Time spent bootstrapping tablet: real 0.041s	user 0.015s	sys 0.001s
I20260709 13:22:39.445199 25245 catalog_manager.cc:5697] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 reported cstate change: term changed from 0 to 1, leader changed from <none> to a724fa0654fa4809b29444f707a88767 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "a724fa0654fa4809b29444f707a88767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:39.448311 25248 catalog_manager.cc:5697] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "1c893d888632407db50d5d53a2f4edb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:39.450593 25639 raft_consensus.cc:359] T acbfd09f2a2f45c7ac9aca07ee71dc7e P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.450875 25638 consensus_queue.cc:260] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.451512 25639 raft_consensus.cc:385] T acbfd09f2a2f45c7ac9aca07ee71dc7e P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.451864 25639 raft_consensus.cc:740] T acbfd09f2a2f45c7ac9aca07ee71dc7e P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a724fa0654fa4809b29444f707a88767, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.452648 25639 consensus_queue.cc:260] T acbfd09f2a2f45c7ac9aca07ee71dc7e P a724fa0654fa4809b29444f707a88767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.453334 25638 ts_tablet_manager.cc:1434] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.006s
I20260709 13:22:39.454661 25638 tablet_bootstrap.cc:492] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f: Bootstrap starting.
I20260709 13:22:39.455582 25639 ts_tablet_manager.cc:1434] T acbfd09f2a2f45c7ac9aca07ee71dc7e P a724fa0654fa4809b29444f707a88767: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:22:39.456867 25639 tablet_bootstrap.cc:492] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767: Bootstrap starting.
I20260709 13:22:39.460631 25638 tablet_bootstrap.cc:654] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.460865 25640 tablet_bootstrap.cc:492] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6: No bootstrap required, opened a new log
I20260709 13:22:39.461443 25640 ts_tablet_manager.cc:1403] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6: Time spent bootstrapping tablet: real 0.043s	user 0.015s	sys 0.008s
I20260709 13:22:39.463455 25639 tablet_bootstrap.cc:654] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.464721 25640 raft_consensus.cc:359] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.465569 25640 raft_consensus.cc:385] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.465862 25640 raft_consensus.cc:740] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.466836 25640 consensus_queue.cc:260] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.470862 25640 ts_tablet_manager.cc:1434] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:22:39.472101 25640 tablet_bootstrap.cc:492] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6: Bootstrap starting.
I20260709 13:22:39.472708 25662 raft_consensus.cc:493] T 70e830e6def34b06a3e805e4e9bf0a44 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.473353 25662 raft_consensus.cc:515] T 70e830e6def34b06a3e805e4e9bf0a44 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.478178 25662 leader_election.cc:290] T 70e830e6def34b06a3e805e4e9bf0a44 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657)
I20260709 13:22:39.479437 25350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70e830e6def34b06a3e805e4e9bf0a44" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f" is_pre_election: true
I20260709 13:22:39.480170 25640 tablet_bootstrap.cc:654] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.482213 25425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70e830e6def34b06a3e805e4e9bf0a44" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6" is_pre_election: true
I20260709 13:22:39.482983 25425 raft_consensus.cc:2393] T 70e830e6def34b06a3e805e4e9bf0a44 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a724fa0654fa4809b29444f707a88767 in current term 1: Already voted for candidate 01445466116342929df00ee716f54e3f in this term.
I20260709 13:22:39.483379 25639 tablet_bootstrap.cc:492] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767: No bootstrap required, opened a new log
I20260709 13:22:39.485991 25463 leader_election.cc:304] T 70e830e6def34b06a3e805e4e9bf0a44 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a724fa0654fa4809b29444f707a88767; no voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6
I20260709 13:22:39.487221 25662 raft_consensus.cc:3060] T 70e830e6def34b06a3e805e4e9bf0a44 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.491592 25638 tablet_bootstrap.cc:492] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f: No bootstrap required, opened a new log
I20260709 13:22:39.492156 25638 ts_tablet_manager.cc:1403] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f: Time spent bootstrapping tablet: real 0.038s	user 0.009s	sys 0.007s
I20260709 13:22:39.487236 25639 ts_tablet_manager.cc:1403] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767: Time spent bootstrapping tablet: real 0.031s	user 0.019s	sys 0.004s
I20260709 13:22:39.494081 25662 raft_consensus.cc:2749] T 70e830e6def34b06a3e805e4e9bf0a44 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:39.494557 25638 raft_consensus.cc:359] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.495313 25638 raft_consensus.cc:385] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.495586 25638 raft_consensus.cc:740] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.496282 25638 consensus_queue.cc:260] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.497300 25639 raft_consensus.cc:359] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.498075 25639 raft_consensus.cc:385] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.498135 25640 tablet_bootstrap.cc:492] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6: No bootstrap required, opened a new log
I20260709 13:22:39.498217 25638 ts_tablet_manager.cc:1434] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 13:22:39.498421 25639 raft_consensus.cc:740] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a724fa0654fa4809b29444f707a88767, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.498842 25640 ts_tablet_manager.cc:1403] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.001s
I20260709 13:22:39.499778 25638 tablet_bootstrap.cc:492] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f: Bootstrap starting.
I20260709 13:22:39.499533 25639 consensus_queue.cc:260] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.501811 25640 raft_consensus.cc:359] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.502592 25640 raft_consensus.cc:385] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.502867 25640 raft_consensus.cc:740] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.503952 25640 consensus_queue.cc:260] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.506824 25638 tablet_bootstrap.cc:654] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.509039 25640 ts_tablet_manager.cc:1434] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:22:39.510136 25640 tablet_bootstrap.cc:492] T ddfd39813ccd423d81843e1bc36a4a76 P 1c893d888632407db50d5d53a2f4edb6: Bootstrap starting.
I20260709 13:22:39.513466 25639 ts_tablet_manager.cc:1434] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:22:39.514578 25639 tablet_bootstrap.cc:492] T d08fd06d7e974780955efa1d052b5da4 P a724fa0654fa4809b29444f707a88767: Bootstrap starting.
I20260709 13:22:39.520865 25640 tablet_bootstrap.cc:654] T ddfd39813ccd423d81843e1bc36a4a76 P 1c893d888632407db50d5d53a2f4edb6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.527489 25639 tablet_bootstrap.cc:654] T d08fd06d7e974780955efa1d052b5da4 P a724fa0654fa4809b29444f707a88767: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.527789 25638 tablet_bootstrap.cc:492] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f: No bootstrap required, opened a new log
I20260709 13:22:39.528258 25638 ts_tablet_manager.cc:1403] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.013s
I20260709 13:22:39.528324 25640 tablet_bootstrap.cc:492] T ddfd39813ccd423d81843e1bc36a4a76 P 1c893d888632407db50d5d53a2f4edb6: No bootstrap required, opened a new log
I20260709 13:22:39.528824 25640 ts_tablet_manager.cc:1403] T ddfd39813ccd423d81843e1bc36a4a76 P 1c893d888632407db50d5d53a2f4edb6: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.007s
I20260709 13:22:39.529260 25627 raft_consensus.cc:493] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.530030 25662 raft_consensus.cc:493] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.529776 25627 raft_consensus.cc:515] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.530545 25662 raft_consensus.cc:515] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.530942 25638 raft_consensus.cc:359] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.531693 25638 raft_consensus.cc:385] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.531843 25640 raft_consensus.cc:359] T ddfd39813ccd423d81843e1bc36a4a76 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.531944 25638 raft_consensus.cc:740] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.545460 25640 raft_consensus.cc:385] T ddfd39813ccd423d81843e1bc36a4a76 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.532651 25662 leader_election.cc:290] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657)
I20260709 13:22:39.545866 25640 raft_consensus.cc:740] T ddfd39813ccd423d81843e1bc36a4a76 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.546437 25638 consensus_queue.cc:260] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.534265 25425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db4c952bd42842479c6e6cc4ac7d2eac" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6" is_pre_election: true
I20260709 13:22:39.548002 25425 raft_consensus.cc:2468] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 0.
I20260709 13:22:39.546736 25640 consensus_queue.cc:260] T ddfd39813ccd423d81843e1bc36a4a76 P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.549817 25463 leader_election.cc:304] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c893d888632407db50d5d53a2f4edb6, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:39.550772 25626 raft_consensus.cc:2804] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:39.551118 25626 raft_consensus.cc:493] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:39.552676 25638 ts_tablet_manager.cc:1434] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.013s
I20260709 13:22:39.551455 25626 raft_consensus.cc:3060] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.556941 25640 ts_tablet_manager.cc:1434] T ddfd39813ccd423d81843e1bc36a4a76 P 1c893d888632407db50d5d53a2f4edb6: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.003s
I20260709 13:22:39.532119 25627 leader_election.cc:290] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:39.534405 25497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db4c952bd42842479c6e6cc4ac7d2eac" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767" is_pre_election: true
I20260709 13:22:39.536113 25350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db4c952bd42842479c6e6cc4ac7d2eac" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f" is_pre_election: true
I20260709 13:22:39.568019 25350 raft_consensus.cc:2468] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 0.
I20260709 13:22:39.537065 25424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db4c952bd42842479c6e6cc4ac7d2eac" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6" is_pre_election: true
I20260709 13:22:39.569325 25424 raft_consensus.cc:2468] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 0.
I20260709 13:22:39.570395 25314 leader_election.cc:304] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:39.571270 25627 raft_consensus.cc:2804] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:39.571697 25627 raft_consensus.cc:493] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:39.572039 25627 raft_consensus.cc:3060] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.575285 25626 raft_consensus.cc:515] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.577533 25497 raft_consensus.cc:2393] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01445466116342929df00ee716f54e3f in current term 1: Already voted for candidate a724fa0654fa4809b29444f707a88767 in this term.
I20260709 13:22:39.580037 25627 raft_consensus.cc:515] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.581377 25350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db4c952bd42842479c6e6cc4ac7d2eac" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:39.581811 25626 leader_election.cc:290] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 election: Requested vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657)
I20260709 13:22:39.583046 25424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db4c952bd42842479c6e6cc4ac7d2eac" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:39.583381 25425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db4c952bd42842479c6e6cc4ac7d2eac" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:39.583842 25424 raft_consensus.cc:3060] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.584009 25350 raft_consensus.cc:2393] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a724fa0654fa4809b29444f707a88767 in current term 1: Already voted for candidate 01445466116342929df00ee716f54e3f in this term.
I20260709 13:22:39.584466 25654 raft_consensus.cc:493] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.585023 25497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db4c952bd42842479c6e6cc4ac7d2eac" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:39.585152 25654 raft_consensus.cc:515] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.585794 25497 raft_consensus.cc:2393] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 01445466116342929df00ee716f54e3f in current term 1: Already voted for candidate a724fa0654fa4809b29444f707a88767 in this term.
I20260709 13:22:39.586120 25627 leader_election.cc:290] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 election: Requested vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:39.585878 25639 tablet_bootstrap.cc:492] T d08fd06d7e974780955efa1d052b5da4 P a724fa0654fa4809b29444f707a88767: No bootstrap required, opened a new log
I20260709 13:22:39.586874 25639 ts_tablet_manager.cc:1403] T d08fd06d7e974780955efa1d052b5da4 P a724fa0654fa4809b29444f707a88767: Time spent bootstrapping tablet: real 0.073s	user 0.025s	sys 0.024s
I20260709 13:22:39.587031 25315 leader_election.cc:304] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01445466116342929df00ee716f54e3f; no voters: 1c893d888632407db50d5d53a2f4edb6, a724fa0654fa4809b29444f707a88767
I20260709 13:22:39.588976 25425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f4e02a32cd4b5c955b4f865d872842" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6" is_pre_election: true
I20260709 13:22:39.589129 25497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f4e02a32cd4b5c955b4f865d872842" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767" is_pre_election: true
I20260709 13:22:39.589562 25425 raft_consensus.cc:2468] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 0.
I20260709 13:22:39.589749 25627 raft_consensus.cc:2749] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:22:39.587347 25654 leader_election.cc:290] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:39.590545 25497 raft_consensus.cc:2468] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 0.
I20260709 13:22:39.589648 25639 raft_consensus.cc:359] T d08fd06d7e974780955efa1d052b5da4 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.591209 25314 leader_election.cc:304] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:39.591607 25639 raft_consensus.cc:385] T d08fd06d7e974780955efa1d052b5da4 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.591899 25639 raft_consensus.cc:740] T d08fd06d7e974780955efa1d052b5da4 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a724fa0654fa4809b29444f707a88767, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.592090 25627 raft_consensus.cc:2804] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:39.592593 25627 raft_consensus.cc:493] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:39.592615 25639 consensus_queue.cc:260] T d08fd06d7e974780955efa1d052b5da4 P a724fa0654fa4809b29444f707a88767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.593122 25627 raft_consensus.cc:3060] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.594262 25424 raft_consensus.cc:2468] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 1.
I20260709 13:22:39.594965 25639 ts_tablet_manager.cc:1434] T d08fd06d7e974780955efa1d052b5da4 P a724fa0654fa4809b29444f707a88767: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:22:39.595628 25463 leader_election.cc:304] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1c893d888632407db50d5d53a2f4edb6, a724fa0654fa4809b29444f707a88767; no voters: 01445466116342929df00ee716f54e3f
I20260709 13:22:39.596486 25626 raft_consensus.cc:2804] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:39.596961 25626 raft_consensus.cc:697] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Becoming Leader. State: Replica: a724fa0654fa4809b29444f707a88767, State: Running, Role: LEADER
I20260709 13:22:39.597370 25639 tablet_bootstrap.cc:492] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767: Bootstrap starting.
I20260709 13:22:39.597652 25626 consensus_queue.cc:237] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.602054 25627 raft_consensus.cc:515] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.604034 25639 tablet_bootstrap.cc:654] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.604553 25627 leader_election.cc:290] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 election: Requested vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:39.605759 25497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f4e02a32cd4b5c955b4f865d872842" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:39.606411 25497 raft_consensus.cc:3060] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.606441 25424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f4e02a32cd4b5c955b4f865d872842" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:39.607045 25424 raft_consensus.cc:3060] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.609959 25246 catalog_manager.cc:5697] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 reported cstate change: term changed from 0 to 1, leader changed from <none> to a724fa0654fa4809b29444f707a88767 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "a724fa0654fa4809b29444f707a88767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:39.612556 25424 raft_consensus.cc:2468] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 1.
I20260709 13:22:39.613879 25314 leader_election.cc:304] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:39.614056 25497 raft_consensus.cc:2468] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 1.
I20260709 13:22:39.614892 25627 raft_consensus.cc:2804] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:39.615919 25627 raft_consensus.cc:697] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Becoming Leader. State: Replica: 01445466116342929df00ee716f54e3f, State: Running, Role: LEADER
I20260709 13:22:39.616755 25627 consensus_queue.cc:237] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.618958 25639 tablet_bootstrap.cc:492] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767: No bootstrap required, opened a new log
I20260709 13:22:39.619534 25639 ts_tablet_manager.cc:1403] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.000s
I20260709 13:22:39.622393 25639 raft_consensus.cc:359] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.623108 25639 raft_consensus.cc:385] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.623412 25639 raft_consensus.cc:740] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a724fa0654fa4809b29444f707a88767, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.624070 25639 consensus_queue.cc:260] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.625582 25639 ts_tablet_manager.cc:1434] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:22:39.626864 25246 catalog_manager.cc:5697] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f reported cstate change: term changed from 0 to 1, leader changed from <none> to 01445466116342929df00ee716f54e3f (127.14.23.65). New cstate: current_term: 1 leader_uuid: "01445466116342929df00ee716f54e3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:39.629572 25627 raft_consensus.cc:493] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.630167 25627 raft_consensus.cc:515] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.633239 25424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfd39813ccd423d81843e1bc36a4a76" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6" is_pre_election: true
I20260709 13:22:39.633319 25627 leader_election.cc:290] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:39.633503 25497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfd39813ccd423d81843e1bc36a4a76" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767" is_pre_election: true
I20260709 13:22:39.633970 25424 raft_consensus.cc:2468] T ddfd39813ccd423d81843e1bc36a4a76 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 0.
I20260709 13:22:39.634192 25497 raft_consensus.cc:2468] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 0.
I20260709 13:22:39.635282 25314 leader_election.cc:304] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:39.635892 25627 raft_consensus.cc:2804] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:39.636262 25627 raft_consensus.cc:493] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:39.636585 25627 raft_consensus.cc:3060] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.638907 25654 raft_consensus.cc:493] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.639456 25654 raft_consensus.cc:515] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.641575 25654 leader_election.cc:290] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:39.642891 25497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d08fd06d7e974780955efa1d052b5da4" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767" is_pre_election: true
I20260709 13:22:39.643708 25497 raft_consensus.cc:2468] T d08fd06d7e974780955efa1d052b5da4 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 0.
I20260709 13:22:39.643877 25627 raft_consensus.cc:515] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.645560 25654 raft_consensus.cc:493] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.646153 25654 raft_consensus.cc:515] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.645264 25315 leader_election.cc:304] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:39.647665 25425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfd39813ccd423d81843e1bc36a4a76" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:39.648147 25425 raft_consensus.cc:3060] T ddfd39813ccd423d81843e1bc36a4a76 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.650076 25654 leader_election.cc:290] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:39.646997 25627 leader_election.cc:290] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 election: Requested vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:39.650372 25423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acbfd09f2a2f45c7ac9aca07ee71dc7e" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6" is_pre_election: true
I20260709 13:22:39.650812 25627 raft_consensus.cc:2804] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:39.651069 25423 raft_consensus.cc:2468] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 0.
I20260709 13:22:39.651335 25627 raft_consensus.cc:493] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:39.651693 25627 raft_consensus.cc:3060] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.652587 25314 leader_election.cc:304] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:39.653649 25654 raft_consensus.cc:2804] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:39.654091 25654 raft_consensus.cc:493] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:39.645330 25424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d08fd06d7e974780955efa1d052b5da4" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6" is_pre_election: true
I20260709 13:22:39.654582 25654 raft_consensus.cc:3060] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.654623 25499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acbfd09f2a2f45c7ac9aca07ee71dc7e" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767" is_pre_election: true
I20260709 13:22:39.655473 25497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfd39813ccd423d81843e1bc36a4a76" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:39.655532 25499 raft_consensus.cc:2468] T acbfd09f2a2f45c7ac9aca07ee71dc7e P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 0.
I20260709 13:22:39.656116 25497 raft_consensus.cc:3060] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.657403 25424 raft_consensus.cc:2468] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 0.
I20260709 13:22:39.660434 25425 raft_consensus.cc:2468] T ddfd39813ccd423d81843e1bc36a4a76 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 1.
I20260709 13:22:39.661805 25314 leader_election.cc:304] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:39.663949 25627 raft_consensus.cc:515] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.666357 25627 leader_election.cc:290] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 election: Requested vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:39.667116 25679 raft_consensus.cc:2804] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:39.667403 25499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d08fd06d7e974780955efa1d052b5da4" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:39.667791 25679 raft_consensus.cc:697] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Becoming Leader. State: Replica: 01445466116342929df00ee716f54e3f, State: Running, Role: LEADER
I20260709 13:22:39.668077 25499 raft_consensus.cc:3060] T d08fd06d7e974780955efa1d052b5da4 P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.668715 25497 raft_consensus.cc:2468] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 1.
I20260709 13:22:39.668826 25679 consensus_queue.cc:237] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.670238 25654 raft_consensus.cc:515] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.672361 25654 leader_election.cc:290] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 election: Requested vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:39.673846 25497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acbfd09f2a2f45c7ac9aca07ee71dc7e" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:39.674661 25497 raft_consensus.cc:3060] T acbfd09f2a2f45c7ac9aca07ee71dc7e P a724fa0654fa4809b29444f707a88767 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.675631 25499 raft_consensus.cc:2468] T d08fd06d7e974780955efa1d052b5da4 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 1.
I20260709 13:22:39.676985 25315 leader_election.cc:304] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:39.677682 25679 raft_consensus.cc:2804] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:39.678188 25679 raft_consensus.cc:697] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Becoming Leader. State: Replica: 01445466116342929df00ee716f54e3f, State: Running, Role: LEADER
I20260709 13:22:39.679044 25679 consensus_queue.cc:237] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.680189 25425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d08fd06d7e974780955efa1d052b5da4" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:39.680800 25425 raft_consensus.cc:3060] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.681103 25246 catalog_manager.cc:5697] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f reported cstate change: term changed from 0 to 1, leader changed from <none> to 01445466116342929df00ee716f54e3f (127.14.23.65). New cstate: current_term: 1 leader_uuid: "01445466116342929df00ee716f54e3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:39.686383 25424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acbfd09f2a2f45c7ac9aca07ee71dc7e" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:39.687179 25424 raft_consensus.cc:3060] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.696282 25425 raft_consensus.cc:2468] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 1.
I20260709 13:22:39.698730 25424 raft_consensus.cc:2468] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 1.
I20260709 13:22:39.700465 25314 leader_election.cc:304] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:39.701495 25680 raft_consensus.cc:2804] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:39.702003 25680 raft_consensus.cc:697] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Becoming Leader. State: Replica: 01445466116342929df00ee716f54e3f, State: Running, Role: LEADER
I20260709 13:22:39.702065 25497 raft_consensus.cc:2468] T acbfd09f2a2f45c7ac9aca07ee71dc7e P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 1.
I20260709 13:22:39.702883 25680 consensus_queue.cc:237] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.709327 25627 consensus_queue.cc:1048] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:39.710876 25246 catalog_manager.cc:5697] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f reported cstate change: term changed from 0 to 1, leader changed from <none> to 01445466116342929df00ee716f54e3f (127.14.23.65). New cstate: current_term: 1 leader_uuid: "01445466116342929df00ee716f54e3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:39.721633 25680 consensus_queue.cc:1048] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:39.735513 25249 catalog_manager.cc:5697] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f reported cstate change: term changed from 0 to 1, leader changed from <none> to 01445466116342929df00ee716f54e3f (127.14.23.65). New cstate: current_term: 1 leader_uuid: "01445466116342929df00ee716f54e3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:39.807504 25424 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ae34bd035b404eba838fadb2f2d50934"
dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
mode: GRACEFUL
new_leader_uuid: "01445466116342929df00ee716f54e3f"
 from {username='slave'} at 127.0.0.1:35940
I20260709 13:22:39.808032 25424 raft_consensus.cc:606] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Received request to transfer leadership to TS 01445466116342929df00ee716f54e3f
I20260709 13:22:39.813134 25424 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0526fea4b4214a128677c8f013ea8cc8"
dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
mode: GRACEFUL
new_leader_uuid: "01445466116342929df00ee716f54e3f"
 from {username='slave'} at 127.0.0.1:35940
I20260709 13:22:39.813685 25424 raft_consensus.cc:606] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Received request to transfer leadership to TS 01445466116342929df00ee716f54e3f
I20260709 13:22:39.818614 25497 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9115d271796f4b18a68999f53eb80e57"
dest_uuid: "a724fa0654fa4809b29444f707a88767"
mode: GRACEFUL
new_leader_uuid: "01445466116342929df00ee716f54e3f"
 from {username='slave'} at 127.0.0.1:36232
I20260709 13:22:39.819092 25497 raft_consensus.cc:606] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Received request to transfer leadership to TS 01445466116342929df00ee716f54e3f
I20260709 13:22:39.821201 25626 consensus_queue.cc:1048] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:39.827358 25497 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db4c952bd42842479c6e6cc4ac7d2eac"
dest_uuid: "a724fa0654fa4809b29444f707a88767"
mode: GRACEFUL
new_leader_uuid: "1c893d888632407db50d5d53a2f4edb6"
 from {username='slave'} at 127.0.0.1:36232
I20260709 13:22:39.827955 25497 raft_consensus.cc:606] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Received request to transfer leadership to TS 1c893d888632407db50d5d53a2f4edb6
I20260709 13:22:39.832682 25626 consensus_queue.cc:1048] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:39.834115 25647 consensus_queue.cc:1048] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:39.850265 25626 raft_consensus.cc:993] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767: : Instructing follower 01445466116342929df00ee716f54e3f to start an election
I20260709 13:22:39.850719 25626 raft_consensus.cc:1081] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Signalling peer 01445466116342929df00ee716f54e3f to start an election
I20260709 13:22:39.852561 25350 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9115d271796f4b18a68999f53eb80e57"
dest_uuid: "01445466116342929df00ee716f54e3f"
 from {username='slave'} at 127.0.0.1:35482
I20260709 13:22:39.853225 25350 raft_consensus.cc:493] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:39.853588 25350 raft_consensus.cc:3060] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:39.856956 25351 raft_consensus.cc:3060] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.868006 25647 consensus_queue.cc:1048] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:39.870028 25350 raft_consensus.cc:515] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.872116 25350 leader_election.cc:290] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 2 election: Requested vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:39.873385 25425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9115d271796f4b18a68999f53eb80e57" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:39.873626 25497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9115d271796f4b18a68999f53eb80e57" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:39.874044 25425 raft_consensus.cc:3060] T 9115d271796f4b18a68999f53eb80e57 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:39.874179 25497 raft_consensus.cc:3055] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:39.874558 25497 raft_consensus.cc:740] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a724fa0654fa4809b29444f707a88767, State: Running, Role: LEADER
I20260709 13:22:39.875299 25497 consensus_queue.cc:260] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.876350 25351 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "acbfd09f2a2f45c7ac9aca07ee71dc7e"
dest_uuid: "01445466116342929df00ee716f54e3f"
mode: GRACEFUL
new_leader_uuid: "1c893d888632407db50d5d53a2f4edb6"
 from {username='slave'} at 127.0.0.1:35498
I20260709 13:22:39.876510 25497 raft_consensus.cc:3060] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:39.881243 25425 raft_consensus.cc:2468] T 9115d271796f4b18a68999f53eb80e57 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 2.
I20260709 13:22:39.882537 25314 leader_election.cc:304] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:39.882983 25497 raft_consensus.cc:2468] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 2.
I20260709 13:22:39.883332 25680 raft_consensus.cc:2804] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:39.883818 25680 raft_consensus.cc:697] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f [term 2 LEADER]: Becoming Leader. State: Replica: 01445466116342929df00ee716f54e3f, State: Running, Role: LEADER
I20260709 13:22:39.884568 25680 consensus_queue.cc:237] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.886178 25351 raft_consensus.cc:606] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Received request to transfer leadership to TS 1c893d888632407db50d5d53a2f4edb6
I20260709 13:22:39.890216 25647 raft_consensus.cc:993] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6: : Instructing follower 01445466116342929df00ee716f54e3f to start an election
I20260709 13:22:39.890661 25659 raft_consensus.cc:1081] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Signalling peer 01445466116342929df00ee716f54e3f to start an election
I20260709 13:22:39.892012 25351 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b9f4e02a32cd4b5c955b4f865d872842"
dest_uuid: "01445466116342929df00ee716f54e3f"
mode: GRACEFUL
new_leader_uuid: "1c893d888632407db50d5d53a2f4edb6"
 from {username='slave'} at 127.0.0.1:35498
I20260709 13:22:39.892253 25348 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ae34bd035b404eba838fadb2f2d50934"
dest_uuid: "01445466116342929df00ee716f54e3f"
 from {username='slave'} at 127.0.0.1:35476
I20260709 13:22:39.892601 25351 raft_consensus.cc:606] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Received request to transfer leadership to TS 1c893d888632407db50d5d53a2f4edb6
I20260709 13:22:39.892961 25348 raft_consensus.cc:493] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:39.893360 25348 raft_consensus.cc:3060] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:39.898351 25351 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d08fd06d7e974780955efa1d052b5da4"
dest_uuid: "01445466116342929df00ee716f54e3f"
mode: GRACEFUL
new_leader_uuid: "1c893d888632407db50d5d53a2f4edb6"
 from {username='slave'} at 127.0.0.1:35498
I20260709 13:22:39.898924 25351 raft_consensus.cc:606] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Received request to transfer leadership to TS 1c893d888632407db50d5d53a2f4edb6
I20260709 13:22:39.899938 25348 raft_consensus.cc:515] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.901804 25659 consensus_queue.cc:1048] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:39.902776 25425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae34bd035b404eba838fadb2f2d50934" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:39.903362 25425 raft_consensus.cc:3055] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:39.903638 25425 raft_consensus.cc:740] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Running, Role: LEADER
I20260709 13:22:39.904484 25425 consensus_queue.cc:260] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.905551 25425 raft_consensus.cc:3060] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:39.907528 25246 catalog_manager.cc:5697] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f reported cstate change: term changed from 1 to 2, leader changed from a724fa0654fa4809b29444f707a88767 (127.14.23.67) to 01445466116342929df00ee716f54e3f (127.14.23.65). New cstate: current_term: 2 leader_uuid: "01445466116342929df00ee716f54e3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:39.910880 25350 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddfd39813ccd423d81843e1bc36a4a76"
dest_uuid: "01445466116342929df00ee716f54e3f"
mode: GRACEFUL
new_leader_uuid: "a724fa0654fa4809b29444f707a88767"
 from {username='slave'} at 127.0.0.1:35498
I20260709 13:22:39.911561 25350 raft_consensus.cc:606] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Received request to transfer leadership to TS a724fa0654fa4809b29444f707a88767
I20260709 13:22:39.914501 25425 raft_consensus.cc:2468] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 2.
I20260709 13:22:39.911332 25348 leader_election.cc:290] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 2 election: Requested vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:39.914919 25647 raft_consensus.cc:993] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6: : Instructing follower 01445466116342929df00ee716f54e3f to start an election
I20260709 13:22:39.915536 25647 raft_consensus.cc:1081] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Signalling peer 01445466116342929df00ee716f54e3f to start an election
I20260709 13:22:39.917395 25348 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0526fea4b4214a128677c8f013ea8cc8"
dest_uuid: "01445466116342929df00ee716f54e3f"
 from {username='slave'} at 127.0.0.1:35476
I20260709 13:22:39.918269 25348 raft_consensus.cc:493] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:39.919008 25348 raft_consensus.cc:3060] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:39.919792 25314 leader_election.cc:304] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:39.920662 25680 raft_consensus.cc:2804] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:39.921113 25680 raft_consensus.cc:697] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f [term 2 LEADER]: Becoming Leader. State: Replica: 01445466116342929df00ee716f54e3f, State: Running, Role: LEADER
I20260709 13:22:39.914678 25497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae34bd035b404eba838fadb2f2d50934" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:39.922017 25680 consensus_queue.cc:237] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.922776 25497 raft_consensus.cc:3060] T ae34bd035b404eba838fadb2f2d50934 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:39.924240 25659 consensus_queue.cc:1048] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:39.929762 25497 raft_consensus.cc:2468] T ae34bd035b404eba838fadb2f2d50934 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 2.
I20260709 13:22:39.935020 25348 raft_consensus.cc:515] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.937549 25348 leader_election.cc:290] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 2 election: Requested vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:39.938190 25499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0526fea4b4214a128677c8f013ea8cc8" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:39.939291 25499 raft_consensus.cc:3060] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:39.939541 25425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0526fea4b4214a128677c8f013ea8cc8" candidate_uuid: "01445466116342929df00ee716f54e3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:39.940148 25425 raft_consensus.cc:3055] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:39.940443 25425 raft_consensus.cc:740] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Running, Role: LEADER
I20260709 13:22:39.941175 25425 consensus_queue.cc:260] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.942835 25425 raft_consensus.cc:3060] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:39.944581 25246 catalog_manager.cc:5697] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f reported cstate change: term changed from 1 to 2, leader changed from 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66) to 01445466116342929df00ee716f54e3f (127.14.23.65). New cstate: current_term: 2 leader_uuid: "01445466116342929df00ee716f54e3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:39.946684 25499 raft_consensus.cc:2468] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 2.
I20260709 13:22:39.948115 25315 leader_election.cc:304] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:39.948942 25680 raft_consensus.cc:2804] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:39.948954 25425 raft_consensus.cc:2468] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01445466116342929df00ee716f54e3f in term 2.
I20260709 13:22:39.949524 25680 raft_consensus.cc:697] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [term 2 LEADER]: Becoming Leader. State: Replica: 01445466116342929df00ee716f54e3f, State: Running, Role: LEADER
I20260709 13:22:39.950403 25680 consensus_queue.cc:237] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:39.960546 25246 catalog_manager.cc:5697] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f reported cstate change: term changed from 1 to 2, leader changed from 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66) to 01445466116342929df00ee716f54e3f (127.14.23.65). New cstate: current_term: 2 leader_uuid: "01445466116342929df00ee716f54e3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:39.988842 25662 consensus_queue.cc:1048] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:39.994861 25666 raft_consensus.cc:993] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767: : Instructing follower 1c893d888632407db50d5d53a2f4edb6 to start an election
I20260709 13:22:39.995369 25666 raft_consensus.cc:1081] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Signalling peer 1c893d888632407db50d5d53a2f4edb6 to start an election
I20260709 13:22:40.001643 25424 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "db4c952bd42842479c6e6cc4ac7d2eac"
dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
 from {username='slave'} at 127.0.0.1:35936
I20260709 13:22:40.002283 25424 raft_consensus.cc:493] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:40.002681 25424 raft_consensus.cc:3060] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.004297 25666 consensus_queue.cc:1048] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:40.010608 25424 raft_consensus.cc:515] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:40.012566 25627 consensus_queue.cc:1048] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:40.022397 25424 leader_election.cc:290] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 2 election: Requested vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:40.023051 25348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db4c952bd42842479c6e6cc4ac7d2eac" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:40.023746 25348 raft_consensus.cc:3060] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.024861 25497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db4c952bd42842479c6e6cc4ac7d2eac" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:40.025475 25497 raft_consensus.cc:3055] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:40.025769 25497 raft_consensus.cc:740] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a724fa0654fa4809b29444f707a88767, State: Running, Role: LEADER
I20260709 13:22:40.026503 25497 consensus_queue.cc:260] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:40.027720 25497 raft_consensus.cc:3060] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.023260 25627 raft_consensus.cc:993] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f: : Instructing follower 1c893d888632407db50d5d53a2f4edb6 to start an election
I20260709 13:22:40.033792 25627 raft_consensus.cc:1081] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Signalling peer 1c893d888632407db50d5d53a2f4edb6 to start an election
I20260709 13:22:40.038303 25424 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b9f4e02a32cd4b5c955b4f865d872842"
dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
 from {username='slave'} at 127.0.0.1:35938
I20260709 13:22:40.038954 25424 raft_consensus.cc:493] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:40.039353 25424 raft_consensus.cc:3060] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.039575 25627 consensus_queue.cc:1048] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:40.041385 25348 raft_consensus.cc:2468] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 2.
I20260709 13:22:40.045066 25391 leader_election.cc:304] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:40.044561 25497 raft_consensus.cc:2468] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 2.
I20260709 13:22:40.046176 25647 raft_consensus.cc:2804] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:40.046751 25647 raft_consensus.cc:697] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6 [term 2 LEADER]: Becoming Leader. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Running, Role: LEADER
I20260709 13:22:40.047622 25647 consensus_queue.cc:237] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:40.048156 25424 raft_consensus.cc:515] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:40.050500 25424 leader_election.cc:290] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 2 election: Requested vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:40.053120 25348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f4e02a32cd4b5c955b4f865d872842" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:40.054659 25348 raft_consensus.cc:3055] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:40.055002 25348 raft_consensus.cc:740] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Running, Role: LEADER
I20260709 13:22:40.053498 25497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f4e02a32cd4b5c955b4f865d872842" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:40.055974 25348 consensus_queue.cc:260] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:40.057132 25348 raft_consensus.cc:3060] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.056546 25497 raft_consensus.cc:3060] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.081669 25627 consensus_queue.cc:1048] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:40.086390 25497 raft_consensus.cc:2468] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 2.
I20260709 13:22:40.086382 25348 raft_consensus.cc:2468] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 2.
I20260709 13:22:40.088265 25390 leader_election.cc:304] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c893d888632407db50d5d53a2f4edb6, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:40.089521 25647 raft_consensus.cc:2804] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:40.090063 25647 raft_consensus.cc:697] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [term 2 LEADER]: Becoming Leader. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Running, Role: LEADER
I20260709 13:22:40.089512 25245 catalog_manager.cc:5697] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6 reported cstate change: term changed from 1 to 2, leader changed from a724fa0654fa4809b29444f707a88767 (127.14.23.67) to 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "1c893d888632407db50d5d53a2f4edb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:40.091037 25647 consensus_queue.cc:237] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:40.092089 25654 raft_consensus.cc:993] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f: : Instructing follower 1c893d888632407db50d5d53a2f4edb6 to start an election
I20260709 13:22:40.092625 25654 raft_consensus.cc:1081] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Signalling peer 1c893d888632407db50d5d53a2f4edb6 to start an election
I20260709 13:22:40.098948 25680 consensus_queue.cc:1048] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:40.099829 25424 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d08fd06d7e974780955efa1d052b5da4"
dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
 from {username='slave'} at 127.0.0.1:35938
I20260709 13:22:40.102169 25424 raft_consensus.cc:493] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:40.102598 25424 raft_consensus.cc:3060] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.103096 25680 consensus_queue.cc:1048] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:40.108306 25245 catalog_manager.cc:5697] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 reported cstate change: term changed from 1 to 2, leader changed from 01445466116342929df00ee716f54e3f (127.14.23.65) to 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "1c893d888632407db50d5d53a2f4edb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:40.116524 25627 consensus_queue.cc:1048] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:40.114584 25424 raft_consensus.cc:515] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:40.135144 25680 raft_consensus.cc:993] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f: : Instructing follower a724fa0654fa4809b29444f707a88767 to start an election
I20260709 13:22:40.135655 25679 raft_consensus.cc:1081] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Signalling peer a724fa0654fa4809b29444f707a88767 to start an election
I20260709 13:22:40.136286 25424 leader_election.cc:290] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 2 election: Requested vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:40.136279 25348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d08fd06d7e974780955efa1d052b5da4" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:40.136710 25497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d08fd06d7e974780955efa1d052b5da4" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:40.137244 25348 raft_consensus.cc:3055] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:40.137270 25497 raft_consensus.cc:3060] T d08fd06d7e974780955efa1d052b5da4 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.137717 25348 raft_consensus.cc:740] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Running, Role: LEADER
I20260709 13:22:40.138566 25348 consensus_queue.cc:260] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:40.139864 25348 raft_consensus.cc:3060] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.145058 25497 raft_consensus.cc:2468] T d08fd06d7e974780955efa1d052b5da4 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 2.
I20260709 13:22:40.146811 25499 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ddfd39813ccd423d81843e1bc36a4a76"
dest_uuid: "a724fa0654fa4809b29444f707a88767"
 from {username='slave'} at 127.0.0.1:36224
I20260709 13:22:40.147555 25499 raft_consensus.cc:493] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:40.147957 25499 raft_consensus.cc:3060] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.148131 25348 raft_consensus.cc:2468] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 2.
I20260709 13:22:40.149387 25391 leader_election.cc:304] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:40.150511 25680 consensus_queue.cc:1048] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:40.151088 25659 raft_consensus.cc:2804] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:40.151639 25659 raft_consensus.cc:697] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6 [term 2 LEADER]: Becoming Leader. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Running, Role: LEADER
I20260709 13:22:40.152455 25659 consensus_queue.cc:237] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:40.157972 25680 consensus_queue.cc:1048] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:40.169456 25245 catalog_manager.cc:5697] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6 reported cstate change: term changed from 1 to 2, leader changed from 01445466116342929df00ee716f54e3f (127.14.23.65) to 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "1c893d888632407db50d5d53a2f4edb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:40.171814 25499 raft_consensus.cc:515] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:40.173637 25498 raft_consensus.cc:1240] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Rejecting Update request from peer 01445466116342929df00ee716f54e3f for earlier term 1. Current term is 2. Ops: []
I20260709 13:22:40.176853 25627 consensus_queue.cc:1059] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:40.179443 25680 raft_consensus.cc:3055] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:40.179858 25680 raft_consensus.cc:740] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Running, Role: LEADER
I20260709 13:22:40.179539 25654 raft_consensus.cc:993] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f: : Instructing follower 1c893d888632407db50d5d53a2f4edb6 to start an election
I20260709 13:22:40.180770 25680 consensus_queue.cc:260] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:40.181644 25654 raft_consensus.cc:1081] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Signalling peer 1c893d888632407db50d5d53a2f4edb6 to start an election
I20260709 13:22:40.181797 25680 raft_consensus.cc:3060] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.183557 25425 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "acbfd09f2a2f45c7ac9aca07ee71dc7e"
dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
 from {username='slave'} at 127.0.0.1:35938
I20260709 13:22:40.184087 25425 raft_consensus.cc:493] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:40.184450 25425 raft_consensus.cc:3060] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.184746 25348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfd39813ccd423d81843e1bc36a4a76" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:40.186941 25499 leader_election.cc:290] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 2 election: Requested vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657)
I20260709 13:22:40.188330 25423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddfd39813ccd423d81843e1bc36a4a76" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:40.188994 25423 raft_consensus.cc:3060] T ddfd39813ccd423d81843e1bc36a4a76 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.191088 25425 raft_consensus.cc:515] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:40.194000 25351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acbfd09f2a2f45c7ac9aca07ee71dc7e" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:40.194643 25351 raft_consensus.cc:3055] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:40.194989 25351 raft_consensus.cc:740] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Running, Role: LEADER
I20260709 13:22:40.195658 25423 raft_consensus.cc:2468] T ddfd39813ccd423d81843e1bc36a4a76 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 2.
I20260709 13:22:40.195776 25351 consensus_queue.cc:260] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:40.197016 25351 raft_consensus.cc:3060] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.198026 25348 raft_consensus.cc:2468] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 2.
I20260709 13:22:40.198983 25463 leader_election.cc:304] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c893d888632407db50d5d53a2f4edb6, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:40.199890 25626 raft_consensus.cc:2804] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:40.200373 25626 raft_consensus.cc:697] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767 [term 2 LEADER]: Becoming Leader. State: Replica: a724fa0654fa4809b29444f707a88767, State: Running, Role: LEADER
I20260709 13:22:40.201150 25626 consensus_queue.cc:237] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:40.202382 25499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acbfd09f2a2f45c7ac9aca07ee71dc7e" candidate_uuid: "1c893d888632407db50d5d53a2f4edb6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a724fa0654fa4809b29444f707a88767"
I20260709 13:22:40.205261 25499 raft_consensus.cc:3060] T acbfd09f2a2f45c7ac9aca07ee71dc7e P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.208669 25351 raft_consensus.cc:2468] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 2.
I20260709 13:22:40.209985 25391 leader_election.cc:304] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, 1c893d888632407db50d5d53a2f4edb6; no voters: 
I20260709 13:22:40.210743 25659 raft_consensus.cc:2804] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:40.211216 25659 raft_consensus.cc:697] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6 [term 2 LEADER]: Becoming Leader. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Running, Role: LEADER
I20260709 13:22:40.212026 25659 consensus_queue.cc:237] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:40.200770 25425 leader_election.cc:290] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6 [CANDIDATE]: Term 2 election: Requested vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), a724fa0654fa4809b29444f707a88767 (127.14.23.67:42881)
I20260709 13:22:40.213069 25499 raft_consensus.cc:2468] T acbfd09f2a2f45c7ac9aca07ee71dc7e P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c893d888632407db50d5d53a2f4edb6 in term 2.
I20260709 13:22:40.217310 25245 catalog_manager.cc:5697] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767 reported cstate change: term changed from 1 to 2, leader changed from 01445466116342929df00ee716f54e3f (127.14.23.65) to a724fa0654fa4809b29444f707a88767 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "a724fa0654fa4809b29444f707a88767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:40.223630 25249 catalog_manager.cc:5697] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6 reported cstate change: term changed from 1 to 2, leader changed from 01445466116342929df00ee716f54e3f (127.14.23.65) to 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "1c893d888632407db50d5d53a2f4edb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:40.290357 25425 raft_consensus.cc:1275] T 9115d271796f4b18a68999f53eb80e57 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Refusing update from remote peer 01445466116342929df00ee716f54e3f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.291517 25627 consensus_queue.cc:1048] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:40.299450 25499 raft_consensus.cc:1275] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Refusing update from remote peer 01445466116342929df00ee716f54e3f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.302560 25680 consensus_queue.cc:1048] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:40.423163 25499 raft_consensus.cc:1275] T ae34bd035b404eba838fadb2f2d50934 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Refusing update from remote peer 01445466116342929df00ee716f54e3f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.424474 25680 consensus_queue.cc:1048] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:40.432426 25425 raft_consensus.cc:1275] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Refusing update from remote peer 01445466116342929df00ee716f54e3f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.434746 25679 consensus_queue.cc:1048] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:40.486495 25351 raft_consensus.cc:1275] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Refusing update from remote peer 1c893d888632407db50d5d53a2f4edb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.487702 25659 consensus_queue.cc:1048] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:40.490038 25499 raft_consensus.cc:1275] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Refusing update from remote peer 01445466116342929df00ee716f54e3f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.491648 25679 consensus_queue.cc:1048] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:40.492399 25425 raft_consensus.cc:1275] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Refusing update from remote peer 01445466116342929df00ee716f54e3f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.495018 25679 consensus_queue.cc:1048] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:40.496117 25498 raft_consensus.cc:1275] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Refusing update from remote peer 1c893d888632407db50d5d53a2f4edb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.497855 25647 consensus_queue.cc:1048] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:40.582629 25351 raft_consensus.cc:1275] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Refusing update from remote peer 1c893d888632407db50d5d53a2f4edb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.583952 25647 consensus_queue.cc:1048] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:40.591168 25499 raft_consensus.cc:1275] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Refusing update from remote peer 1c893d888632407db50d5d53a2f4edb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.592903 25659 consensus_queue.cc:1048] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:40.623396 25499 raft_consensus.cc:1275] T d08fd06d7e974780955efa1d052b5da4 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Refusing update from remote peer 1c893d888632407db50d5d53a2f4edb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.624722 25659 consensus_queue.cc:1048] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:40.631942 25351 raft_consensus.cc:1275] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Refusing update from remote peer 1c893d888632407db50d5d53a2f4edb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.633930 25558 consensus_queue.cc:1048] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:40.673334 25499 raft_consensus.cc:1275] T acbfd09f2a2f45c7ac9aca07ee71dc7e P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Refusing update from remote peer 1c893d888632407db50d5d53a2f4edb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.674583 25647 consensus_queue.cc:1048] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:40.681805 25351 raft_consensus.cc:1275] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Refusing update from remote peer 1c893d888632407db50d5d53a2f4edb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.683326 25708 consensus_queue.cc:1048] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:40.768783 25425 raft_consensus.cc:1275] T ddfd39813ccd423d81843e1bc36a4a76 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Refusing update from remote peer a724fa0654fa4809b29444f707a88767: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.770012 25662 consensus_queue.cc:1048] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:40.780061 25351 raft_consensus.cc:1275] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Refusing update from remote peer a724fa0654fa4809b29444f707a88767: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.781720 25626 consensus_queue.cc:1048] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:42.915509 25245 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33602:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:22:42.934542 25245 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33602:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:22:42.995486 14429 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 13:22:43.146368 25315 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:42881, user_credentials={real_user=slave}} blocked reactor thread for 60189.1us
W20260709 13:22:43.211458 25391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.65:43051, user_credentials={real_user=slave}} blocked reactor thread for 66929.8us
I20260709 13:22:43.308954 25766 tablet.cc:1724] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.330476 25349 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0526fea4b4214a128677c8f013ea8cc8"
dest_uuid: "01445466116342929df00ee716f54e3f"
mode: GRACEFUL
new_leader_uuid: "a724fa0654fa4809b29444f707a88767"
 from {username='slave'} at 127.0.0.1:58212
I20260709 13:22:43.331364 25349 raft_consensus.cc:606] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [term 2 LEADER]: Received request to transfer leadership to TS a724fa0654fa4809b29444f707a88767
I20260709 13:22:43.353061 25776 tablet.cc:1724] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:43.370548 25452 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:d08fd06d7e974780955efa1d052b5da4)
I20260709 13:22:43.370751 25778 tablet.cc:1724] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.371796 25779 tablet.cc:1724] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:43.377517 25753 log.cc:927] Time spent T 70e830e6def34b06a3e805e4e9bf0a44 P a724fa0654fa4809b29444f707a88767: Append to log took a long time: real 0.164s	user 0.003s	sys 0.000s
I20260709 13:22:43.315943 25773 tablet.cc:1724] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.389775 25774 tablet.cc:1724] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.390962 25767 tablet.cc:1724] T 70e830e6def34b06a3e805e4e9bf0a44 P 1c893d888632407db50d5d53a2f4edb6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.416205 25777 tablet.cc:1724] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.419229 25754 raft_consensus.cc:993] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f: : Instructing follower a724fa0654fa4809b29444f707a88767 to start an election
I20260709 13:22:43.429029 25754 raft_consensus.cc:1081] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [term 2 LEADER]: Signalling peer a724fa0654fa4809b29444f707a88767 to start an election
W20260709 13:22:43.433100 25377 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:0526fea4b4214a128677c8f013ea8cc8)
I20260709 13:22:43.440212 25779 tablet.cc:1724] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.443698 25766 tablet.cc:1724] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.453116 25497 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0526fea4b4214a128677c8f013ea8cc8"
dest_uuid: "a724fa0654fa4809b29444f707a88767"
 from {username='slave'} at 127.0.0.1:36224
I20260709 13:22:43.454104 25497 raft_consensus.cc:493] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:43.454784 25497 raft_consensus.cc:3060] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:43.454128 25767 tablet.cc:1724] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:43.456740 25377 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:0526fea4b4214a128677c8f013ea8cc8)
I20260709 13:22:43.467700 25422 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b9f4e02a32cd4b5c955b4f865d872842"
dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
mode: GRACEFUL
new_leader_uuid: "a724fa0654fa4809b29444f707a88767"
 from {username='slave'} at 127.0.0.1:44594
I20260709 13:22:43.468412 25422 raft_consensus.cc:606] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [term 2 LEADER]: Received request to transfer leadership to TS a724fa0654fa4809b29444f707a88767
I20260709 13:22:43.470515 14429 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:22:43.471161 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:43.467308 25767 tablet.cc:1724] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.481985 25351 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "01c94152f0c44421b2c2e3560785b3e6"
dest_uuid: "01445466116342929df00ee716f54e3f"
mode: GRACEFUL
new_leader_uuid: "a724fa0654fa4809b29444f707a88767"
 from {username='slave'} at 127.0.0.1:58212
I20260709 13:22:43.481624 25783 tablet.cc:1724] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.482724 25351 raft_consensus.cc:606] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [term 2 LEADER]: Received request to transfer leadership to TS a724fa0654fa4809b29444f707a88767
I20260709 13:22:43.484652 25776 tablet.cc:1724] T ddfd39813ccd423d81843e1bc36a4a76 P 1c893d888632407db50d5d53a2f4edb6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.493800 25766 tablet.cc:1724] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.494052 25780 tablet.cc:1724] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.495352 25424 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "efc2e95552a142ea8f26983d2b49e0c5"
dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
mode: GRACEFUL
new_leader_uuid: "a724fa0654fa4809b29444f707a88767"
 from {username='slave'} at 127.0.0.1:44594
I20260709 13:22:43.496027 25424 raft_consensus.cc:606] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Received request to transfer leadership to TS a724fa0654fa4809b29444f707a88767
I20260709 13:22:43.496220 25776 tablet.cc:1724] T 9115d271796f4b18a68999f53eb80e57 P 1c893d888632407db50d5d53a2f4edb6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.498090 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:22:43.498792 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:43.440279 25784 tablet.cc:1724] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.497148 25774 tablet.cc:1724] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.499464 25785 tablet.cc:1724] T ae34bd035b404eba838fadb2f2d50934 P a724fa0654fa4809b29444f707a88767: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.522195 25497 raft_consensus.cc:515] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:43.529130 25346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0526fea4b4214a128677c8f013ea8cc8" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:43.529808 25346 raft_consensus.cc:3055] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:43.530210 25346 raft_consensus.cc:740] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Running, Role: LEADER
I20260709 13:22:43.531096 25346 consensus_queue.cc:260] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:43.532230 25346 raft_consensus.cc:3060] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:43.515257 25766 tablet.cc:1724] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.540355 25783 tablet.cc:1724] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.544768 25346 raft_consensus.cc:2468] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 3.
I20260709 13:22:43.546435 25465 leader_election.cc:304] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:43.547529 25758 raft_consensus.cc:2804] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:43.548061 25758 raft_consensus.cc:697] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 [term 3 LEADER]: Becoming Leader. State: Replica: a724fa0654fa4809b29444f707a88767, State: Running, Role: LEADER
I20260709 13:22:43.548853 25758 consensus_queue.cc:237] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:43.549315 25778 tablet.cc:1724] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.550534 25424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0526fea4b4214a128677c8f013ea8cc8" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:43.551179 25424 raft_consensus.cc:3060] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:43.547641 25497 leader_election.cc:290] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 3 election: Requested vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657)
I20260709 13:22:43.562431 25424 raft_consensus.cc:2468] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 3.
I20260709 13:22:43.551177 25784 tablet.cc:1724] T 70e830e6def34b06a3e805e4e9bf0a44 P a724fa0654fa4809b29444f707a88767: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.565101 25785 tablet.cc:1724] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.565339 25780 tablet.cc:1724] T d08fd06d7e974780955efa1d052b5da4 P a724fa0654fa4809b29444f707a88767: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.581792 25780 tablet.cc:1724] T acbfd09f2a2f45c7ac9aca07ee71dc7e P a724fa0654fa4809b29444f707a88767: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.593066 25249 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f: soft_deleted_table [id=59210192c00845278c200bb77ef719a3] alter complete (version 1)
I20260709 13:22:43.604265 25246 catalog_manager.cc:5697] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 reported cstate change: term changed from 2 to 3, leader changed from 01445466116342929df00ee716f54e3f (127.14.23.65) to a724fa0654fa4809b29444f707a88767 (127.14.23.67). New cstate: current_term: 3 leader_uuid: "a724fa0654fa4809b29444f707a88767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:43.803689 25769 raft_consensus.cc:993] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f: : Instructing follower a724fa0654fa4809b29444f707a88767 to start an election
I20260709 13:22:43.804296 25760 raft_consensus.cc:1081] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [term 2 LEADER]: Signalling peer a724fa0654fa4809b29444f707a88767 to start an election
I20260709 13:22:43.805853 25497 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "01c94152f0c44421b2c2e3560785b3e6"
dest_uuid: "a724fa0654fa4809b29444f707a88767"
 from {username='slave'} at 127.0.0.1:36224
I20260709 13:22:43.806344 25497 raft_consensus.cc:493] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:43.806598 25497 raft_consensus.cc:3060] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:43.811124 25497 raft_consensus.cc:515] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } } peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } }
I20260709 13:22:43.812533 25497 leader_election.cc:290] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 3 election: Requested vote from peers 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657), 01445466116342929df00ee716f54e3f (127.14.23.65:43051)
I20260709 13:22:43.813405 25424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c94152f0c44421b2c2e3560785b3e6" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:43.813601 25346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c94152f0c44421b2c2e3560785b3e6" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:43.813989 25424 raft_consensus.cc:3060] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:43.814203 25346 raft_consensus.cc:3055] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:43.814522 25346 raft_consensus.cc:740] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 01445466116342929df00ee716f54e3f, State: Running, Role: LEADER
I20260709 13:22:43.815174 25346 consensus_queue.cc:260] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } } peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } }
I20260709 13:22:43.816123 25346 raft_consensus.cc:3060] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:43.819710 25424 raft_consensus.cc:2468] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 3.
I20260709 13:22:43.820981 25463 leader_election.cc:304] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c893d888632407db50d5d53a2f4edb6, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:43.821280 25346 raft_consensus.cc:2468] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 3.
I20260709 13:22:43.821710 25758 raft_consensus.cc:2804] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:43.822280 25758 raft_consensus.cc:697] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 [term 3 LEADER]: Becoming Leader. State: Replica: a724fa0654fa4809b29444f707a88767, State: Running, Role: LEADER
I20260709 13:22:43.823060 25758 consensus_queue.cc:237] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } } peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } }
I20260709 13:22:43.830422 25246 catalog_manager.cc:5697] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 reported cstate change: term changed from 2 to 3, leader changed from 01445466116342929df00ee716f54e3f (127.14.23.65) to a724fa0654fa4809b29444f707a88767 (127.14.23.67). New cstate: current_term: 3 leader_uuid: "a724fa0654fa4809b29444f707a88767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:43.872728 25755 raft_consensus.cc:993] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6: : Instructing follower a724fa0654fa4809b29444f707a88767 to start an election
I20260709 13:22:43.873322 25756 raft_consensus.cc:1081] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [term 2 LEADER]: Signalling peer a724fa0654fa4809b29444f707a88767 to start an election
I20260709 13:22:43.874986 25495 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b9f4e02a32cd4b5c955b4f865d872842"
dest_uuid: "a724fa0654fa4809b29444f707a88767"
 from {username='slave'} at 127.0.0.1:36214
I20260709 13:22:43.875545 25495 raft_consensus.cc:493] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:43.875728 25770 raft_consensus.cc:993] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6: : Instructing follower a724fa0654fa4809b29444f707a88767 to start an election
I20260709 13:22:43.875873 25495 raft_consensus.cc:3060] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:43.876237 25756 raft_consensus.cc:1081] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Signalling peer a724fa0654fa4809b29444f707a88767 to start an election
I20260709 13:22:43.877657 25497 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "efc2e95552a142ea8f26983d2b49e0c5"
dest_uuid: "a724fa0654fa4809b29444f707a88767"
 from {username='slave'} at 127.0.0.1:36214
I20260709 13:22:43.878104 25497 raft_consensus.cc:493] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:43.878356 25497 raft_consensus.cc:3060] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:43.881590 25495 raft_consensus.cc:515] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:43.882834 25497 raft_consensus.cc:515] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:43.883888 25495 leader_election.cc:290] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 3 election: Requested vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657)
I20260709 13:22:43.884665 25424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f4e02a32cd4b5c955b4f865d872842" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:43.884815 25497 leader_election.cc:290] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 2 election: Requested vote from peers 01445466116342929df00ee716f54e3f (127.14.23.65:43051), 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657)
I20260709 13:22:43.884881 25346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f4e02a32cd4b5c955b4f865d872842" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:43.885602 25346 raft_consensus.cc:3060] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:43.885807 25424 raft_consensus.cc:3055] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:43.886278 25424 raft_consensus.cc:740] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Running, Role: LEADER
I20260709 13:22:43.886334 25351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc2e95552a142ea8f26983d2b49e0c5" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01445466116342929df00ee716f54e3f"
I20260709 13:22:43.886971 25351 raft_consensus.cc:3060] T efc2e95552a142ea8f26983d2b49e0c5 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:43.887766 25424 consensus_queue.cc:260] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:43.888916 25424 raft_consensus.cc:3060] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:43.893724 25422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc2e95552a142ea8f26983d2b49e0c5" candidate_uuid: "a724fa0654fa4809b29444f707a88767" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c893d888632407db50d5d53a2f4edb6"
I20260709 13:22:43.894367 25422 raft_consensus.cc:3055] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:43.894621 25351 raft_consensus.cc:2468] T efc2e95552a142ea8f26983d2b49e0c5 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 2.
I20260709 13:22:43.894625 25422 raft_consensus.cc:740] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1c893d888632407db50d5d53a2f4edb6, State: Running, Role: LEADER
I20260709 13:22:43.895872 25465 leader_election.cc:304] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:43.895773 25422 consensus_queue.cc:260] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:43.896575 25758 raft_consensus.cc:2804] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:43.896822 25346 raft_consensus.cc:2468] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 3.
I20260709 13:22:43.897135 25758 raft_consensus.cc:697] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767 [term 2 LEADER]: Becoming Leader. State: Replica: a724fa0654fa4809b29444f707a88767, State: Running, Role: LEADER
I20260709 13:22:43.897055 25422 raft_consensus.cc:3060] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:43.898018 25465 leader_election.cc:304] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01445466116342929df00ee716f54e3f, a724fa0654fa4809b29444f707a88767; no voters: 
I20260709 13:22:43.898152 25758 consensus_queue.cc:237] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:43.898859 25662 raft_consensus.cc:2804] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:43.900170 25662 raft_consensus.cc:697] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 [term 3 LEADER]: Becoming Leader. State: Replica: a724fa0654fa4809b29444f707a88767, State: Running, Role: LEADER
I20260709 13:22:43.901016 25662 consensus_queue.cc:237] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } }
I20260709 13:22:43.903766 25424 raft_consensus.cc:2468] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 3.
I20260709 13:22:43.907413 25422 raft_consensus.cc:2468] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a724fa0654fa4809b29444f707a88767 in term 2.
I20260709 13:22:43.910584 25246 catalog_manager.cc:5697] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767 reported cstate change: term changed from 1 to 2, leader changed from 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66) to a724fa0654fa4809b29444f707a88767 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "a724fa0654fa4809b29444f707a88767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:43.922309 25246 catalog_manager.cc:5697] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 reported cstate change: term changed from 2 to 3, leader changed from 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66) to a724fa0654fa4809b29444f707a88767 (127.14.23.67). New cstate: current_term: 3 leader_uuid: "a724fa0654fa4809b29444f707a88767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a724fa0654fa4809b29444f707a88767" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 42881 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:44.052775 25346 raft_consensus.cc:1275] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [term 3 FOLLOWER]: Refusing update from remote peer a724fa0654fa4809b29444f707a88767: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:22:44.054098 25758 consensus_queue.cc:1048] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:44.065966 25424 raft_consensus.cc:1275] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [term 3 FOLLOWER]: Refusing update from remote peer a724fa0654fa4809b29444f707a88767: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:22:44.067453 25662 consensus_queue.cc:1048] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:44.301569 25346 raft_consensus.cc:1275] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [term 3 FOLLOWER]: Refusing update from remote peer a724fa0654fa4809b29444f707a88767: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:22:44.302784 25793 consensus_queue.cc:1048] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:44.310637 25424 raft_consensus.cc:1275] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [term 3 FOLLOWER]: Refusing update from remote peer a724fa0654fa4809b29444f707a88767: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:22:44.312230 25758 consensus_queue.cc:1048] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:22:44.384918 25424 raft_consensus.cc:1275] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [term 3 FOLLOWER]: Refusing update from remote peer a724fa0654fa4809b29444f707a88767: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:44.386041 25758 consensus_queue.cc:1048] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:44.394490 25346 raft_consensus.cc:1275] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [term 3 FOLLOWER]: Refusing update from remote peer a724fa0654fa4809b29444f707a88767: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:44.396134 25796 consensus_queue.cc:1048] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:44.435949 25346 raft_consensus.cc:1275] T efc2e95552a142ea8f26983d2b49e0c5 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Refusing update from remote peer a724fa0654fa4809b29444f707a88767: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:44.437158 25796 consensus_queue.cc:1048] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01445466116342929df00ee716f54e3f" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:44.445679 25424 raft_consensus.cc:1275] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Refusing update from remote peer a724fa0654fa4809b29444f707a88767: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:44.447027 25662 consensus_queue.cc:1048] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c893d888632407db50d5d53a2f4edb6" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:44.556632 14429 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:22:44.561954 14429 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:22:44.562390 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:44.568686 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:22:44.569216 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:45.606688 14429 tablet_server.cc:179] TabletServer@127.14.23.65:0 shutting down...
W20260709 13:22:45.614459 25391 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 5 similar messages]
W20260709 13:22:45.618700 25391 consensus_peers.cc:597] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6 -> Peer 01445466116342929df00ee716f54e3f (127.14.23.65:43051): Couldn't send request to peer 01445466116342929df00ee716f54e3f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:45.638067 25391 consensus_peers.cc:597] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6 -> Peer 01445466116342929df00ee716f54e3f (127.14.23.65:43051): Couldn't send request to peer 01445466116342929df00ee716f54e3f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:45.640972 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:45.641825 14429 tablet_replica.cc:333] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f: stopping tablet replica
I20260709 13:22:45.642649 14429 raft_consensus.cc:2243] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.643391 14429 raft_consensus.cc:2272] T b9f4e02a32cd4b5c955b4f865d872842 P 01445466116342929df00ee716f54e3f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.646148 14429 tablet_replica.cc:333] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f: stopping tablet replica
I20260709 13:22:45.646770 14429 raft_consensus.cc:2243] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.647426 14429 raft_consensus.cc:2272] T d08fd06d7e974780955efa1d052b5da4 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.649887 14429 tablet_replica.cc:333] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f: stopping tablet replica
I20260709 13:22:45.650632 14429 raft_consensus.cc:2243] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.651320 14429 raft_consensus.cc:2272] T 0526fea4b4214a128677c8f013ea8cc8 P 01445466116342929df00ee716f54e3f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.654098 14429 tablet_replica.cc:333] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f: stopping tablet replica
I20260709 13:22:45.654747 14429 raft_consensus.cc:2243] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.655459 14429 raft_consensus.cc:2272] T db4c952bd42842479c6e6cc4ac7d2eac P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.658219 14429 tablet_replica.cc:333] T 6729a5f0b78e4d148483f7b5d427304a P 01445466116342929df00ee716f54e3f: stopping tablet replica
I20260709 13:22:45.659121 14429 raft_consensus.cc:2243] T 6729a5f0b78e4d148483f7b5d427304a P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.659801 14429 raft_consensus.cc:2272] T 6729a5f0b78e4d148483f7b5d427304a P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.662588 14429 tablet_replica.cc:333] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f: stopping tablet replica
I20260709 13:22:45.663163 14429 raft_consensus.cc:2243] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.663647 14429 raft_consensus.cc:2272] T 01c94152f0c44421b2c2e3560785b3e6 P 01445466116342929df00ee716f54e3f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.667193 14429 tablet_replica.cc:333] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f: stopping tablet replica
I20260709 13:22:45.668118 14429 raft_consensus.cc:2243] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:45.669312 14429 raft_consensus.cc:2272] T 8b34ba27e9a24cab9530218fa592c6ae P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.672472 14429 tablet_replica.cc:333] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f: stopping tablet replica
I20260709 13:22:45.673070 14429 raft_consensus.cc:2243] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:45.674019 14429 raft_consensus.cc:2272] T 70e830e6def34b06a3e805e4e9bf0a44 P 01445466116342929df00ee716f54e3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.676505 14429 tablet_replica.cc:333] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f: stopping tablet replica
I20260709 13:22:45.677100 14429 raft_consensus.cc:2243] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:45.678028 14429 raft_consensus.cc:2272] T 9115d271796f4b18a68999f53eb80e57 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.680217 14429 tablet_replica.cc:333] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f: stopping tablet replica
I20260709 13:22:45.680819 14429 raft_consensus.cc:2243] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:45.681715 14429 raft_consensus.cc:2272] T 5fead9ecfeb04d0d94509484c7234118 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.684130 14429 tablet_replica.cc:333] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f: stopping tablet replica
I20260709 13:22:45.684697 14429 raft_consensus.cc:2243] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.685341 14429 raft_consensus.cc:2272] T ddfd39813ccd423d81843e1bc36a4a76 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.687780 14429 tablet_replica.cc:333] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f: stopping tablet replica
I20260709 13:22:45.688300 14429 raft_consensus.cc:2243] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:45.689234 14429 raft_consensus.cc:2272] T ae34bd035b404eba838fadb2f2d50934 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.691804 14429 tablet_replica.cc:333] T 1cefd5d18235480d9e47c7bf0377e4f0 P 01445466116342929df00ee716f54e3f: stopping tablet replica
I20260709 13:22:45.692490 14429 raft_consensus.cc:2243] T 1cefd5d18235480d9e47c7bf0377e4f0 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.693092 14429 raft_consensus.cc:2272] T 1cefd5d18235480d9e47c7bf0377e4f0 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.696269 14429 tablet_replica.cc:333] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f: stopping tablet replica
I20260709 13:22:45.697072 14429 raft_consensus.cc:2243] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.697903 14429 raft_consensus.cc:2272] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.701122 14429 tablet_replica.cc:333] T 28a64081a0c94a92a11015ec646adf0c P 01445466116342929df00ee716f54e3f: stopping tablet replica
I20260709 13:22:45.701764 14429 raft_consensus.cc:2243] T 28a64081a0c94a92a11015ec646adf0c P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.702329 14429 raft_consensus.cc:2272] T 28a64081a0c94a92a11015ec646adf0c P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.704660 14429 tablet_replica.cc:333] T efc2e95552a142ea8f26983d2b49e0c5 P 01445466116342929df00ee716f54e3f: stopping tablet replica
I20260709 13:22:45.705263 14429 raft_consensus.cc:2243] T efc2e95552a142ea8f26983d2b49e0c5 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.705792 14429 raft_consensus.cc:2272] T efc2e95552a142ea8f26983d2b49e0c5 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.708819 14429 tablet_replica.cc:333] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f: stopping tablet replica
I20260709 13:22:45.709489 14429 raft_consensus.cc:2243] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:45.710481 14429 raft_consensus.cc:2272] T 30912c2e960f4e5c85dfe927e98a1e9f P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.712733 14429 tablet_replica.cc:333] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f: stopping tablet replica
I20260709 13:22:45.713162 14429 raft_consensus.cc:2243] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.713673 14429 raft_consensus.cc:2272] T 0768383f820c4e4cb8d8595901456ae1 P 01445466116342929df00ee716f54e3f [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:45.728241 25465 consensus_peers.cc:597] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767 -> Peer 01445466116342929df00ee716f54e3f (127.14.23.65:43051): Couldn't send request to peer 01445466116342929df00ee716f54e3f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:45.762213 14429 tablet_server.cc:196] TabletServer@127.14.23.65:0 shutdown complete.
W20260709 13:22:45.770596 25391 consensus_peers.cc:597] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6 -> Peer 01445466116342929df00ee716f54e3f (127.14.23.65:43051): Couldn't send request to peer 01445466116342929df00ee716f54e3f. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43051: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:45.793622 25391 consensus_peers.cc:597] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6 -> Peer 01445466116342929df00ee716f54e3f (127.14.23.65:43051): Couldn't send request to peer 01445466116342929df00ee716f54e3f. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43051: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:45.812125 14429 tablet_server.cc:179] TabletServer@127.14.23.66:0 shutting down...
I20260709 13:22:45.840782 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:45.841645 14429 tablet_replica.cc:333] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6: stopping tablet replica
I20260709 13:22:45.842819 14429 raft_consensus.cc:2243] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.843400 14429 raft_consensus.cc:2272] T b9f4e02a32cd4b5c955b4f865d872842 P 1c893d888632407db50d5d53a2f4edb6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.846200 14429 tablet_replica.cc:333] T ddfd39813ccd423d81843e1bc36a4a76 P 1c893d888632407db50d5d53a2f4edb6: stopping tablet replica
I20260709 13:22:45.846830 14429 raft_consensus.cc:2243] T ddfd39813ccd423d81843e1bc36a4a76 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.847398 14429 raft_consensus.cc:2272] T ddfd39813ccd423d81843e1bc36a4a76 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.849812 14429 tablet_replica.cc:333] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6: stopping tablet replica
I20260709 13:22:45.850497 14429 raft_consensus.cc:2243] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.851082 14429 raft_consensus.cc:2272] T ae34bd035b404eba838fadb2f2d50934 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.853813 14429 tablet_replica.cc:333] T 70e830e6def34b06a3e805e4e9bf0a44 P 1c893d888632407db50d5d53a2f4edb6: stopping tablet replica
I20260709 13:22:45.854583 14429 raft_consensus.cc:2243] T 70e830e6def34b06a3e805e4e9bf0a44 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.855345 14429 raft_consensus.cc:2272] T 70e830e6def34b06a3e805e4e9bf0a44 P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.858062 14429 tablet_replica.cc:333] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6: stopping tablet replica
I20260709 13:22:45.858686 14429 raft_consensus.cc:2243] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:22:45.858989 25465 consensus_peers.cc:597] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 -> Peer 01445466116342929df00ee716f54e3f (127.14.23.65:43051): Couldn't send request to peer 01445466116342929df00ee716f54e3f. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43051: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:45.859730 14429 raft_consensus.cc:2272] T d08fd06d7e974780955efa1d052b5da4 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.862217 14429 tablet_replica.cc:333] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6: stopping tablet replica
I20260709 13:22:45.862846 14429 raft_consensus.cc:2243] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.863447 14429 raft_consensus.cc:2272] T 0526fea4b4214a128677c8f013ea8cc8 P 1c893d888632407db50d5d53a2f4edb6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.865887 14429 tablet_replica.cc:333] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6: stopping tablet replica
I20260709 13:22:45.866463 14429 raft_consensus.cc:2243] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.866986 14429 raft_consensus.cc:2272] T 5fead9ecfeb04d0d94509484c7234118 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.869531 14429 tablet_replica.cc:333] T 9115d271796f4b18a68999f53eb80e57 P 1c893d888632407db50d5d53a2f4edb6: stopping tablet replica
I20260709 13:22:45.870271 14429 raft_consensus.cc:2243] T 9115d271796f4b18a68999f53eb80e57 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.870826 14429 raft_consensus.cc:2272] T 9115d271796f4b18a68999f53eb80e57 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:45.873164 25463 consensus_peers.cc:597] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 -> Peer 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657): Couldn't send request to peer 1c893d888632407db50d5d53a2f4edb6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:45.873778 14429 tablet_replica.cc:333] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6: stopping tablet replica
I20260709 13:22:45.874472 14429 raft_consensus.cc:2243] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:45.875404 14429 raft_consensus.cc:2272] T db4c952bd42842479c6e6cc4ac7d2eac P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.877722 14429 tablet_replica.cc:333] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6: stopping tablet replica
I20260709 13:22:45.878320 14429 raft_consensus.cc:2243] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:45.879240 14429 raft_consensus.cc:2272] T 6729a5f0b78e4d148483f7b5d427304a P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.882014 14429 tablet_replica.cc:333] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6: stopping tablet replica
I20260709 13:22:45.882699 14429 raft_consensus.cc:2243] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.883344 14429 raft_consensus.cc:2272] T 01c94152f0c44421b2c2e3560785b3e6 P 1c893d888632407db50d5d53a2f4edb6 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:45.885388 25465 consensus_peers.cc:597] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 -> Peer 01445466116342929df00ee716f54e3f (127.14.23.65:43051): Couldn't send request to peer 01445466116342929df00ee716f54e3f. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43051: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:45.886190 14429 tablet_replica.cc:333] T 8b34ba27e9a24cab9530218fa592c6ae P 1c893d888632407db50d5d53a2f4edb6: stopping tablet replica
I20260709 13:22:45.886816 14429 raft_consensus.cc:2243] T 8b34ba27e9a24cab9530218fa592c6ae P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.887332 14429 raft_consensus.cc:2272] T 8b34ba27e9a24cab9530218fa592c6ae P 1c893d888632407db50d5d53a2f4edb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.889547 14429 tablet_replica.cc:333] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6: stopping tablet replica
I20260709 13:22:45.890121 14429 raft_consensus.cc:2243] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:45.890995 14429 raft_consensus.cc:2272] T 0768383f820c4e4cb8d8595901456ae1 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.893273 14429 tablet_replica.cc:333] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6: stopping tablet replica
I20260709 13:22:45.893814 14429 raft_consensus.cc:2243] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.894363 14429 raft_consensus.cc:2272] T 30912c2e960f4e5c85dfe927e98a1e9f P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.896620 14429 tablet_replica.cc:333] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6: stopping tablet replica
I20260709 13:22:45.897135 14429 raft_consensus.cc:2243] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.897578 14429 raft_consensus.cc:2272] T efc2e95552a142ea8f26983d2b49e0c5 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.899840 14429 tablet_replica.cc:333] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6: stopping tablet replica
I20260709 13:22:45.900362 14429 raft_consensus.cc:2243] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:45.901381 14429 raft_consensus.cc:2272] T acbfd09f2a2f45c7ac9aca07ee71dc7e P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.903714 14429 tablet_replica.cc:333] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6: stopping tablet replica
I20260709 13:22:45.904327 14429 raft_consensus.cc:2243] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.905049 14429 raft_consensus.cc:2272] T 28a64081a0c94a92a11015ec646adf0c P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.908207 14429 tablet_replica.cc:333] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6: stopping tablet replica
I20260709 13:22:45.908912 14429 raft_consensus.cc:2243] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:45.910017 14429 raft_consensus.cc:2272] T 1cefd5d18235480d9e47c7bf0377e4f0 P 1c893d888632407db50d5d53a2f4edb6 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:45.947639 25465 consensus_peers.cc:597] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 -> Peer 01445466116342929df00ee716f54e3f (127.14.23.65:43051): Couldn't send request to peer 01445466116342929df00ee716f54e3f. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43051: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:45.952862 14429 tablet_server.cc:196] TabletServer@127.14.23.66:0 shutdown complete.
W20260709 13:22:45.970824 25463 consensus_peers.cc:597] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767 -> Peer 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657): Couldn't send request to peer 1c893d888632407db50d5d53a2f4edb6. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.66:46657: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:46.058446 25465 consensus_peers.cc:597] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 -> Peer 01445466116342929df00ee716f54e3f (127.14.23.65:43051): Couldn't send request to peer 01445466116342929df00ee716f54e3f. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43051: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:46.065413 14429 tablet_server.cc:179] TabletServer@127.14.23.67:0 shutting down...
W20260709 13:22:46.197396 25465 consensus_peers.cc:597] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767 -> Peer 01445466116342929df00ee716f54e3f (127.14.23.65:43051): Couldn't send request to peer 01445466116342929df00ee716f54e3f. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:43051: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:46.207830 25463 consensus_peers.cc:597] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 -> Peer 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657): Couldn't send request to peer 1c893d888632407db50d5d53a2f4edb6. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.66:46657: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:46.208662 25463 consensus_peers.cc:597] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 -> Peer 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657): Couldn't send request to peer 1c893d888632407db50d5d53a2f4edb6. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.66:46657: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:46.211717 25463 consensus_peers.cc:597] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767 -> Peer 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657): Couldn't send request to peer 1c893d888632407db50d5d53a2f4edb6. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.66:46657: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:46.264464 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:46.265295 14429 tablet_replica.cc:333] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767: stopping tablet replica
I20260709 13:22:46.266256 14429 raft_consensus.cc:2243] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:46.267174 14429 raft_consensus.cc:2272] T b9f4e02a32cd4b5c955b4f865d872842 P a724fa0654fa4809b29444f707a88767 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.269519 14429 tablet_replica.cc:333] T 70e830e6def34b06a3e805e4e9bf0a44 P a724fa0654fa4809b29444f707a88767: stopping tablet replica
I20260709 13:22:46.270193 14429 raft_consensus.cc:2243] T 70e830e6def34b06a3e805e4e9bf0a44 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.270711 14429 raft_consensus.cc:2272] T 70e830e6def34b06a3e805e4e9bf0a44 P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.272943 14429 tablet_replica.cc:333] T d08fd06d7e974780955efa1d052b5da4 P a724fa0654fa4809b29444f707a88767: stopping tablet replica
I20260709 13:22:46.273517 14429 raft_consensus.cc:2243] T d08fd06d7e974780955efa1d052b5da4 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.274184 14429 raft_consensus.cc:2272] T d08fd06d7e974780955efa1d052b5da4 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.277611 14429 tablet_replica.cc:333] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767: stopping tablet replica
I20260709 13:22:46.278568 14429 raft_consensus.cc:2243] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 [term 3 LEADER]: Raft consensus shutting down.
W20260709 13:22:46.278792 25463 consensus_peers.cc:597] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 -> Peer 1c893d888632407db50d5d53a2f4edb6 (127.14.23.66:46657): Couldn't send request to peer 1c893d888632407db50d5d53a2f4edb6. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.66:46657: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:46.279991 14429 raft_consensus.cc:2272] T 0526fea4b4214a128677c8f013ea8cc8 P a724fa0654fa4809b29444f707a88767 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.282756 14429 tablet_replica.cc:333] T acbfd09f2a2f45c7ac9aca07ee71dc7e P a724fa0654fa4809b29444f707a88767: stopping tablet replica
I20260709 13:22:46.283352 14429 raft_consensus.cc:2243] T acbfd09f2a2f45c7ac9aca07ee71dc7e P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.283838 14429 raft_consensus.cc:2272] T acbfd09f2a2f45c7ac9aca07ee71dc7e P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.285980 14429 tablet_replica.cc:333] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767: stopping tablet replica
I20260709 13:22:46.286533 14429 raft_consensus.cc:2243] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:46.287294 14429 raft_consensus.cc:2272] T 28a64081a0c94a92a11015ec646adf0c P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.289419 14429 tablet_replica.cc:333] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767: stopping tablet replica
I20260709 13:22:46.290009 14429 raft_consensus.cc:2243] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:46.290854 14429 raft_consensus.cc:2272] T efc2e95552a142ea8f26983d2b49e0c5 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.292936 14429 tablet_replica.cc:333] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767: stopping tablet replica
I20260709 13:22:46.293429 14429 raft_consensus.cc:2243] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.293902 14429 raft_consensus.cc:2272] T 9115d271796f4b18a68999f53eb80e57 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.296033 14429 tablet_replica.cc:333] T 5fead9ecfeb04d0d94509484c7234118 P a724fa0654fa4809b29444f707a88767: stopping tablet replica
I20260709 13:22:46.296525 14429 raft_consensus.cc:2243] T 5fead9ecfeb04d0d94509484c7234118 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.296962 14429 raft_consensus.cc:2272] T 5fead9ecfeb04d0d94509484c7234118 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.299145 14429 tablet_replica.cc:333] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767: stopping tablet replica
I20260709 13:22:46.299630 14429 raft_consensus.cc:2243] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.300068 14429 raft_consensus.cc:2272] T 1cefd5d18235480d9e47c7bf0377e4f0 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.302254 14429 tablet_replica.cc:333] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767: stopping tablet replica
I20260709 13:22:46.302757 14429 raft_consensus.cc:2243] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:46.303459 14429 raft_consensus.cc:2272] T ddfd39813ccd423d81843e1bc36a4a76 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.305433 14429 tablet_replica.cc:333] T ae34bd035b404eba838fadb2f2d50934 P a724fa0654fa4809b29444f707a88767: stopping tablet replica
I20260709 13:22:46.305958 14429 raft_consensus.cc:2243] T ae34bd035b404eba838fadb2f2d50934 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.306397 14429 raft_consensus.cc:2272] T ae34bd035b404eba838fadb2f2d50934 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.308393 14429 tablet_replica.cc:333] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767: stopping tablet replica
I20260709 13:22:46.308907 14429 raft_consensus.cc:2243] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:46.309818 14429 raft_consensus.cc:2272] T 01c94152f0c44421b2c2e3560785b3e6 P a724fa0654fa4809b29444f707a88767 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.312041 14429 tablet_replica.cc:333] T 8b34ba27e9a24cab9530218fa592c6ae P a724fa0654fa4809b29444f707a88767: stopping tablet replica
I20260709 13:22:46.312546 14429 raft_consensus.cc:2243] T 8b34ba27e9a24cab9530218fa592c6ae P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.312997 14429 raft_consensus.cc:2272] T 8b34ba27e9a24cab9530218fa592c6ae P a724fa0654fa4809b29444f707a88767 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.315055 14429 tablet_replica.cc:333] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767: stopping tablet replica
I20260709 13:22:46.315550 14429 raft_consensus.cc:2243] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.316002 14429 raft_consensus.cc:2272] T db4c952bd42842479c6e6cc4ac7d2eac P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.318095 14429 tablet_replica.cc:333] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767: stopping tablet replica
I20260709 13:22:46.318591 14429 raft_consensus.cc:2243] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.319013 14429 raft_consensus.cc:2272] T 6729a5f0b78e4d148483f7b5d427304a P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.321079 14429 tablet_replica.cc:333] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767: stopping tablet replica
I20260709 13:22:46.321524 14429 raft_consensus.cc:2243] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.321977 14429 raft_consensus.cc:2272] T 0768383f820c4e4cb8d8595901456ae1 P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.324092 14429 tablet_replica.cc:333] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767: stopping tablet replica
I20260709 13:22:46.324535 14429 raft_consensus.cc:2243] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.324981 14429 raft_consensus.cc:2272] T 30912c2e960f4e5c85dfe927e98a1e9f P a724fa0654fa4809b29444f707a88767 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:47.213585 25521 debug-util.cc:398] Leaking SignalData structure 0x7b0800388760 after lost signal to thread 14432
W20260709 13:22:47.214649 25521 debug-util.cc:398] Leaking SignalData structure 0x7b080029f7c0 after lost signal to thread 25283
W20260709 13:22:47.366158 14429 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 25521)
W20260709 13:22:48.366668 14429 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 25521)
W20260709 13:22:49.367084 14429 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 25521)
I20260709 13:22:49.670816 14429 tablet_server.cc:196] TabletServer@127.14.23.67:0 shutdown complete.
I20260709 13:22:49.710584 14429 master.cc:562] Master@127.14.23.126:39227 shutting down...
I20260709 13:22:49.731339 14429 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:49.731976 14429 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:49.732318 14429 tablet_replica.cc:333] T 00000000000000000000000000000000 P 64cc9b71002c4a708e70fdba2f0dfb4f: stopping tablet replica
I20260709 13:22:49.752928 14429 master.cc:584] Master@127.14.23.126:39227 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (15978 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:22:49.803086 14429 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.23.126:41765
I20260709 13:22:49.804183 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:49.809798 25816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:49.809981 25815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:49.810367 25818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:49.812121 14429 server_base.cc:1061] running on GCE node
I20260709 13:22:49.813117 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:49.813308 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:49.813463 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603369813446 us; error 0 us; skew 500 ppm
I20260709 13:22:49.813998 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:49.816294 14429 webserver.cc:533] Webserver started at http://127.14.23.126:44435/ using document root <none> and password file <none>
I20260709 13:22:49.816762 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:49.816931 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:49.817203 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:49.818832 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603055563165-14429-0/minicluster-data/master-0-root/instance:
uuid: "5c581d2486bd457b9c3575d973c5bee9"
format_stamp: "Formatted at 2026-07-09 13:22:49 on dist-test-slave-twc1"
I20260709 13:22:49.823151 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:22:49.826665 25823 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:49.827401 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:22:49.827697 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603055563165-14429-0/minicluster-data/master-0-root
uuid: "5c581d2486bd457b9c3575d973c5bee9"
format_stamp: "Formatted at 2026-07-09 13:22:49 on dist-test-slave-twc1"
I20260709 13:22:49.827958 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603055563165-14429-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603055563165-14429-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603055563165-14429-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:49.850591 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:49.851851 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:49.890249 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.126:41765
I20260709 13:22:49.890368 25874 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.126:41765 every 8 connection(s)
I20260709 13:22:49.894156 25875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:49.906252 25875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9: Bootstrap starting.
I20260709 13:22:49.911319 25875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:49.915804 25875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9: No bootstrap required, opened a new log
I20260709 13:22:49.918186 25875 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c581d2486bd457b9c3575d973c5bee9" member_type: VOTER }
I20260709 13:22:49.918648 25875 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:49.918835 25875 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c581d2486bd457b9c3575d973c5bee9, State: Initialized, Role: FOLLOWER
I20260709 13:22:49.919332 25875 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c581d2486bd457b9c3575d973c5bee9" member_type: VOTER }
I20260709 13:22:49.919790 25875 raft_consensus.cc:399] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:22:49.919979 25875 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:22:49.920207 25875 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:49.925323 25875 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c581d2486bd457b9c3575d973c5bee9" member_type: VOTER }
I20260709 13:22:49.925854 25875 leader_election.cc:304] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c581d2486bd457b9c3575d973c5bee9; no voters: 
I20260709 13:22:49.927019 25875 leader_election.cc:290] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:22:49.927356 25878 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:49.928700 25878 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9 [term 1 LEADER]: Becoming Leader. State: Replica: 5c581d2486bd457b9c3575d973c5bee9, State: Running, Role: LEADER
I20260709 13:22:49.929437 25878 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c581d2486bd457b9c3575d973c5bee9" member_type: VOTER }
I20260709 13:22:49.930068 25875 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:22:49.934697 25879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c581d2486bd457b9c3575d973c5bee9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c581d2486bd457b9c3575d973c5bee9" member_type: VOTER } }
I20260709 13:22:49.935469 25879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9 [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:49.936022 25880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c581d2486bd457b9c3575d973c5bee9. Latest consensus state: current_term: 1 leader_uuid: "5c581d2486bd457b9c3575d973c5bee9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c581d2486bd457b9c3575d973c5bee9" member_type: VOTER } }
I20260709 13:22:49.936678 25880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9 [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:49.938084 25886 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:22:49.942831 25886 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:22:49.944970 14429 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:22:49.951779 25886 catalog_manager.cc:1383] Generated new cluster ID: 6b7c368d4b324640b8392ced35a2383c
I20260709 13:22:49.952057 25886 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:22:49.969028 25886 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:22:49.970435 25886 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:22:49.989645 25886 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9: Generated new TSK 0
I20260709 13:22:49.990278 25886 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:22:50.011648 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:50.017627 25896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:50.018481 25897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:50.019924 25899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:50.021101 14429 server_base.cc:1061] running on GCE node
I20260709 13:22:50.022116 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:50.022328 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:50.022476 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603370022460 us; error 0 us; skew 500 ppm
I20260709 13:22:50.023193 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:50.025525 14429 webserver.cc:533] Webserver started at http://127.14.23.65:45191/ using document root <none> and password file <none>
I20260709 13:22:50.026053 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:50.026240 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:50.026489 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:50.027539 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603055563165-14429-0/minicluster-data/ts-0-root/instance:
uuid: "f5efb07a61464511be72667a6cab7599"
format_stamp: "Formatted at 2026-07-09 13:22:50 on dist-test-slave-twc1"
I20260709 13:22:50.032198 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 13:22:50.035672 25904 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:50.036449 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:22:50.036736 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603055563165-14429-0/minicluster-data/ts-0-root
uuid: "f5efb07a61464511be72667a6cab7599"
format_stamp: "Formatted at 2026-07-09 13:22:50 on dist-test-slave-twc1"
I20260709 13:22:50.036994 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603055563165-14429-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603055563165-14429-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603055563165-14429-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:50.066583 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:50.067816 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:50.069290 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:50.071563 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:50.071767 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:50.071980 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:50.072140 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:50.111778 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.65:45725
I20260709 13:22:50.111877 25966 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.65:45725 every 8 connection(s)
I20260709 13:22:50.116683 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:50.123924 25971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:50.125963 25972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:50.126987 25974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:50.127938 14429 server_base.cc:1061] running on GCE node
I20260709 13:22:50.128916 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:50.129129 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:50.129302 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603370129284 us; error 0 us; skew 500 ppm
I20260709 13:22:50.129999 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:50.132629 14429 webserver.cc:533] Webserver started at http://127.14.23.66:36401/ using document root <none> and password file <none>
I20260709 13:22:50.133126 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:50.133308 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:50.133567 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:50.134792 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603055563165-14429-0/minicluster-data/ts-1-root/instance:
uuid: "6b39791309b04490b6c527fdb0df2046"
format_stamp: "Formatted at 2026-07-09 13:22:50 on dist-test-slave-twc1"
I20260709 13:22:50.139829 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.001s
I20260709 13:22:50.142069 25967 heartbeater.cc:344] Connected to a master server at 127.14.23.126:41765
I20260709 13:22:50.142633 25967 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:50.143524 25967 heartbeater.cc:507] Master 127.14.23.126:41765 requested a full tablet report, sending...
I20260709 13:22:50.143873 25979 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:50.144987 14429 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 13:22:50.145337 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603055563165-14429-0/minicluster-data/ts-1-root
uuid: "6b39791309b04490b6c527fdb0df2046"
format_stamp: "Formatted at 2026-07-09 13:22:50 on dist-test-slave-twc1"
I20260709 13:22:50.145612 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603055563165-14429-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603055563165-14429-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603055563165-14429-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:50.146157 25840 ts_manager.cc:194] Registered new tserver with Master: f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725)
I20260709 13:22:50.148227 25840 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52868
I20260709 13:22:50.162791 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:50.164152 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:50.165742 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:50.168088 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:50.168292 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:50.168524 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:50.168682 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:50.209458 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.66:40197
I20260709 13:22:50.209580 26041 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.66:40197 every 8 connection(s)
I20260709 13:22:50.214337 14429 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:50.222150 26045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:50.223248 26046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:50.226958 14429 server_base.cc:1061] running on GCE node
I20260709 13:22:50.227682 26042 heartbeater.cc:344] Connected to a master server at 127.14.23.126:41765
W20260709 13:22:50.227792 26048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:50.228125 26042 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:50.228909 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:22:50.229103 26042 heartbeater.cc:507] Master 127.14.23.126:41765 requested a full tablet report, sending...
W20260709 13:22:50.229280 14429 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:50.229538 14429 hybrid_clock.cc:648] HybridClock initialized: now 1783603370229518 us; error 0 us; skew 500 ppm
I20260709 13:22:50.230324 14429 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:50.231560 25840 ts_manager.cc:194] Registered new tserver with Master: 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:50.233022 14429 webserver.cc:533] Webserver started at http://127.14.23.67:46325/ using document root <none> and password file <none>
I20260709 13:22:50.233207 25840 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52878
I20260709 13:22:50.233598 14429 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:50.233853 14429 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:50.234193 14429 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:50.235738 14429 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603055563165-14429-0/minicluster-data/ts-2-root/instance:
uuid: "86ab90de6ed14e13b5ee21a100ee8151"
format_stamp: "Formatted at 2026-07-09 13:22:50 on dist-test-slave-twc1"
I20260709 13:22:50.240461 14429 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:22:50.244021 26053 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:50.244897 14429 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 13:22:50.245206 14429 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603055563165-14429-0/minicluster-data/ts-2-root
uuid: "86ab90de6ed14e13b5ee21a100ee8151"
format_stamp: "Formatted at 2026-07-09 13:22:50 on dist-test-slave-twc1"
I20260709 13:22:50.245458 14429 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603055563165-14429-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603055563165-14429-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskS1TJee/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603055563165-14429-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:50.259354 14429 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:50.260618 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:50.262171 14429 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:50.264534 14429 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:50.264740 14429 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:50.265021 14429 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:50.265179 14429 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:50.307054 14429 rpc_server.cc:307] RPC server started. Bound to: 127.14.23.67:38207
I20260709 13:22:50.307163 26115 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.23.67:38207 every 8 connection(s)
I20260709 13:22:50.321165 26116 heartbeater.cc:344] Connected to a master server at 127.14.23.126:41765
I20260709 13:22:50.321650 26116 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:50.322402 26116 heartbeater.cc:507] Master 127.14.23.126:41765 requested a full tablet report, sending...
I20260709 13:22:50.324561 25840 ts_manager.cc:194] Registered new tserver with Master: 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:50.325147 14429 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014805087s
I20260709 13:22:50.326171 25840 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52880
I20260709 13:22:50.350848 25840 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52882:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:22:50.353668 25840 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:22:50.434448 26081 tablet_service.cc:1511] Processing CreateTablet for tablet 64e40e1547c14815b57cd70e9bc884b2 (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:50.435118 26079 tablet_service.cc:1511] Processing CreateTablet for tablet f6e06dcb93a347bfb244d15cc49cc104 (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:50.436010 26081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64e40e1547c14815b57cd70e9bc884b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.437435 26080 tablet_service.cc:1511] Processing CreateTablet for tablet 4a610fca006245cbac0d5f66538993e4 (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:50.438978 26080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a610fca006245cbac0d5f66538993e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.438727 26007 tablet_service.cc:1511] Processing CreateTablet for tablet 64e40e1547c14815b57cd70e9bc884b2 (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:50.440150 26007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64e40e1547c14815b57cd70e9bc884b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.442448 25930 tablet_service.cc:1511] Processing CreateTablet for tablet f6e06dcb93a347bfb244d15cc49cc104 (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:50.443127 26006 tablet_service.cc:1511] Processing CreateTablet for tablet 4a610fca006245cbac0d5f66538993e4 (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:50.443915 25930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6e06dcb93a347bfb244d15cc49cc104. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.444480 26006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a610fca006245cbac0d5f66538993e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.444308 25932 tablet_service.cc:1511] Processing CreateTablet for tablet 64e40e1547c14815b57cd70e9bc884b2 (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:50.443136 25931 tablet_service.cc:1511] Processing CreateTablet for tablet 4a610fca006245cbac0d5f66538993e4 (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:50.450428 25931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a610fca006245cbac0d5f66538993e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.450575 26003 tablet_service.cc:1511] Processing CreateTablet for tablet 2a1e227945bc461f8c4226b73d895b63 (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:50.451077 25929 tablet_service.cc:1511] Processing CreateTablet for tablet db9a2bfd8849495da7c36e5712750012 (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:50.451974 26003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a1e227945bc461f8c4226b73d895b63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.452503 25929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db9a2bfd8849495da7c36e5712750012. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.455071 25928 tablet_service.cc:1511] Processing CreateTablet for tablet 2a1e227945bc461f8c4226b73d895b63 (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:50.460016 25928 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a1e227945bc461f8c4226b73d895b63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.456280 26005 tablet_service.cc:1511] Processing CreateTablet for tablet f6e06dcb93a347bfb244d15cc49cc104 (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:50.462239 26079 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6e06dcb93a347bfb244d15cc49cc104. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.462783 26005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6e06dcb93a347bfb244d15cc49cc104. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.457216 26004 tablet_service.cc:1511] Processing CreateTablet for tablet db9a2bfd8849495da7c36e5712750012 (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:50.465883 26004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db9a2bfd8849495da7c36e5712750012. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.466289 25932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64e40e1547c14815b57cd70e9bc884b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.471382 26002 tablet_service.cc:1511] Processing CreateTablet for tablet edbb1098626c47439e5e3b7d2edca307 (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:50.472954 26002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edbb1098626c47439e5e3b7d2edca307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.472805 26078 tablet_service.cc:1511] Processing CreateTablet for tablet db9a2bfd8849495da7c36e5712750012 (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:50.474745 26078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db9a2bfd8849495da7c36e5712750012. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.471328 25927 tablet_service.cc:1511] Processing CreateTablet for tablet edbb1098626c47439e5e3b7d2edca307 (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:50.476301 25927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edbb1098626c47439e5e3b7d2edca307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.479312 26077 tablet_service.cc:1511] Processing CreateTablet for tablet 2a1e227945bc461f8c4226b73d895b63 (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:50.481559 26077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a1e227945bc461f8c4226b73d895b63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.484238 26076 tablet_service.cc:1511] Processing CreateTablet for tablet edbb1098626c47439e5e3b7d2edca307 (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:50.485745 26076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edbb1098626c47439e5e3b7d2edca307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.491832 26075 tablet_service.cc:1511] Processing CreateTablet for tablet 6c5cf4443b384aee96a6383ab797603c (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:50.493233 26075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c5cf4443b384aee96a6383ab797603c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.487356 26001 tablet_service.cc:1511] Processing CreateTablet for tablet 6c5cf4443b384aee96a6383ab797603c (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:50.489248 25926 tablet_service.cc:1511] Processing CreateTablet for tablet 6c5cf4443b384aee96a6383ab797603c (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:50.497546 25926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c5cf4443b384aee96a6383ab797603c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.495052 26000 tablet_service.cc:1511] Processing CreateTablet for tablet cd842edf9c67436fb5d5e3172a02908f (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:50.501281 26000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd842edf9c67436fb5d5e3172a02908f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.504954 25925 tablet_service.cc:1511] Processing CreateTablet for tablet cd842edf9c67436fb5d5e3172a02908f (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:50.506435 25925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd842edf9c67436fb5d5e3172a02908f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.511077 26001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c5cf4443b384aee96a6383ab797603c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.542907 26074 tablet_service.cc:1511] Processing CreateTablet for tablet cd842edf9c67436fb5d5e3172a02908f (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:50.544396 26074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd842edf9c67436fb5d5e3172a02908f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.544175 25924 tablet_service.cc:1511] Processing CreateTablet for tablet 7be8d8fb42cd4874a094e408852cfa43 (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:50.545590 25924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7be8d8fb42cd4874a094e408852cfa43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.545339 26005 tablet_service.cc:1511] Processing CreateTablet for tablet 7be8d8fb42cd4874a094e408852cfa43 (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:50.546640 26005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7be8d8fb42cd4874a094e408852cfa43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.576597 26132 tablet_bootstrap.cc:492] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599: Bootstrap starting.
I20260709 13:22:50.590354 26073 tablet_service.cc:1511] Processing CreateTablet for tablet 7be8d8fb42cd4874a094e408852cfa43 (DEFAULT_TABLE table=test-workload [id=731b99551b1141dfb31f84feeb771d67]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:50.602183 26073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7be8d8fb42cd4874a094e408852cfa43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:50.621887 26132 tablet_bootstrap.cc:654] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.622836 26131 tablet_bootstrap.cc:492] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046: Bootstrap starting.
I20260709 13:22:50.627835 26133 tablet_bootstrap.cc:492] T 64e40e1547c14815b57cd70e9bc884b2 P 86ab90de6ed14e13b5ee21a100ee8151: Bootstrap starting.
I20260709 13:22:50.634181 26131 tablet_bootstrap.cc:654] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.635329 26133 tablet_bootstrap.cc:654] T 64e40e1547c14815b57cd70e9bc884b2 P 86ab90de6ed14e13b5ee21a100ee8151: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.638857 26131 tablet_bootstrap.cc:492] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046: No bootstrap required, opened a new log
I20260709 13:22:50.638710 26132 tablet_bootstrap.cc:492] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599: No bootstrap required, opened a new log
I20260709 13:22:50.639616 26131 ts_tablet_manager.cc:1403] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.000s
I20260709 13:22:50.639714 26132 ts_tablet_manager.cc:1403] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599: Time spent bootstrapping tablet: real 0.063s	user 0.011s	sys 0.008s
I20260709 13:22:50.641891 26133 tablet_bootstrap.cc:492] T 64e40e1547c14815b57cd70e9bc884b2 P 86ab90de6ed14e13b5ee21a100ee8151: No bootstrap required, opened a new log
I20260709 13:22:50.642524 26133 ts_tablet_manager.cc:1403] T 64e40e1547c14815b57cd70e9bc884b2 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent bootstrapping tablet: real 0.015s	user 0.013s	sys 0.000s
I20260709 13:22:50.643373 26131 raft_consensus.cc:359] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.643971 26131 raft_consensus.cc:385] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:50.643705 26132 raft_consensus.cc:359] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.644297 26131 raft_consensus.cc:740] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Initialized, Role: FOLLOWER
I20260709 13:22:50.644564 26132 raft_consensus.cc:385] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:50.644938 26132 raft_consensus.cc:740] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5efb07a61464511be72667a6cab7599, State: Initialized, Role: FOLLOWER
I20260709 13:22:50.645077 26131 consensus_queue.cc:260] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.645686 26133 raft_consensus.cc:359] T 64e40e1547c14815b57cd70e9bc884b2 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } }
I20260709 13:22:50.645674 26132 consensus_queue.cc:260] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.646651 26133 raft_consensus.cc:385] T 64e40e1547c14815b57cd70e9bc884b2 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:50.647148 26133 raft_consensus.cc:740] T 64e40e1547c14815b57cd70e9bc884b2 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Initialized, Role: FOLLOWER
I20260709 13:22:50.647254 26042 heartbeater.cc:499] Master 127.14.23.126:41765 was elected leader, sending a full tablet report...
I20260709 13:22:50.647985 26131 ts_tablet_manager.cc:1434] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:22:50.648406 26133 consensus_queue.cc:260] T 64e40e1547c14815b57cd70e9bc884b2 P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } }
I20260709 13:22:50.649040 26131 tablet_bootstrap.cc:492] T f6e06dcb93a347bfb244d15cc49cc104 P 6b39791309b04490b6c527fdb0df2046: Bootstrap starting.
I20260709 13:22:50.650317 25967 heartbeater.cc:499] Master 127.14.23.126:41765 was elected leader, sending a full tablet report...
I20260709 13:22:50.650864 26132 ts_tablet_manager.cc:1434] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.001s
I20260709 13:22:50.651953 26132 tablet_bootstrap.cc:492] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599: Bootstrap starting.
I20260709 13:22:50.656128 26131 tablet_bootstrap.cc:654] T f6e06dcb93a347bfb244d15cc49cc104 P 6b39791309b04490b6c527fdb0df2046: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.658660 26132 tablet_bootstrap.cc:654] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.661640 26116 heartbeater.cc:499] Master 127.14.23.126:41765 was elected leader, sending a full tablet report...
I20260709 13:22:50.662339 26131 tablet_bootstrap.cc:492] T f6e06dcb93a347bfb244d15cc49cc104 P 6b39791309b04490b6c527fdb0df2046: No bootstrap required, opened a new log
I20260709 13:22:50.662979 26131 ts_tablet_manager.cc:1403] T f6e06dcb93a347bfb244d15cc49cc104 P 6b39791309b04490b6c527fdb0df2046: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.004s
I20260709 13:22:50.667279 26133 ts_tablet_manager.cc:1434] T 64e40e1547c14815b57cd70e9bc884b2 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:22:50.668644 26133 tablet_bootstrap.cc:492] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151: Bootstrap starting.
I20260709 13:22:50.673115 26131 raft_consensus.cc:359] T f6e06dcb93a347bfb244d15cc49cc104 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.674830 26131 raft_consensus.cc:385] T f6e06dcb93a347bfb244d15cc49cc104 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:50.675191 26131 raft_consensus.cc:740] T f6e06dcb93a347bfb244d15cc49cc104 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Initialized, Role: FOLLOWER
I20260709 13:22:50.676108 26132 tablet_bootstrap.cc:492] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599: No bootstrap required, opened a new log
I20260709 13:22:50.676237 26133 tablet_bootstrap.cc:654] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.676779 26132 ts_tablet_manager.cc:1403] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599: Time spent bootstrapping tablet: real 0.025s	user 0.019s	sys 0.005s
I20260709 13:22:50.676131 26131 consensus_queue.cc:260] T f6e06dcb93a347bfb244d15cc49cc104 P 6b39791309b04490b6c527fdb0df2046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.679011 26131 ts_tablet_manager.cc:1434] T f6e06dcb93a347bfb244d15cc49cc104 P 6b39791309b04490b6c527fdb0df2046: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:22:50.680110 26131 tablet_bootstrap.cc:492] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046: Bootstrap starting.
I20260709 13:22:50.680027 26132 raft_consensus.cc:359] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.680976 26132 raft_consensus.cc:385] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:50.681317 26132 raft_consensus.cc:740] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5efb07a61464511be72667a6cab7599, State: Initialized, Role: FOLLOWER
I20260709 13:22:50.682252 26132 consensus_queue.cc:260] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.684332 26132 ts_tablet_manager.cc:1434] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:22:50.685400 26132 tablet_bootstrap.cc:492] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599: Bootstrap starting.
I20260709 13:22:50.688524 26133 tablet_bootstrap.cc:492] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151: No bootstrap required, opened a new log
I20260709 13:22:50.689184 26133 ts_tablet_manager.cc:1403] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent bootstrapping tablet: real 0.021s	user 0.005s	sys 0.014s
I20260709 13:22:50.691828 26132 tablet_bootstrap.cc:654] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.692430 26133 raft_consensus.cc:359] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.693449 26133 raft_consensus.cc:385] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:50.693614 26131 tablet_bootstrap.cc:654] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.693854 26133 raft_consensus.cc:740] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Initialized, Role: FOLLOWER
I20260709 13:22:50.694907 26133 consensus_queue.cc:260] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.697185 26133 ts_tablet_manager.cc:1434] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 13:22:50.698639 26133 tablet_bootstrap.cc:492] T cd842edf9c67436fb5d5e3172a02908f P 86ab90de6ed14e13b5ee21a100ee8151: Bootstrap starting.
I20260709 13:22:50.702646 26140 raft_consensus.cc:493] T 64e40e1547c14815b57cd70e9bc884b2 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:50.703393 26140 raft_consensus.cc:515] T 64e40e1547c14815b57cd70e9bc884b2 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } }
I20260709 13:22:50.704015 26132 tablet_bootstrap.cc:492] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599: No bootstrap required, opened a new log
I20260709 13:22:50.704600 26138 raft_consensus.cc:493] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:50.705174 26138 raft_consensus.cc:515] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.710330 26138 leader_election.cc:290] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
W20260709 13:22:50.713647 26043 tablet.cc:2406] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:22:50.704757 26132 ts_tablet_manager.cc:1403] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.009s
I20260709 13:22:50.721387 25942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64e40e1547c14815b57cd70e9bc884b2" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599" is_pre_election: true
I20260709 13:22:50.717554 26132 raft_consensus.cc:359] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.722762 26132 raft_consensus.cc:385] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:50.723094 26132 raft_consensus.cc:740] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5efb07a61464511be72667a6cab7599, State: Initialized, Role: FOLLOWER
W20260709 13:22:50.724013 26057 leader_election.cc:343] T 64e40e1547c14815b57cd70e9bc884b2 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:50.725315 26132 consensus_queue.cc:260] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.725730 26140 leader_election.cc:290] T 64e40e1547c14815b57cd70e9bc884b2 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:50.730202 26133 tablet_bootstrap.cc:654] T cd842edf9c67436fb5d5e3172a02908f P 86ab90de6ed14e13b5ee21a100ee8151: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.730794 26132 ts_tablet_manager.cc:1434] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:22:50.735365 26132 tablet_bootstrap.cc:492] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599: Bootstrap starting.
I20260709 13:22:50.740700 26131 tablet_bootstrap.cc:492] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046: No bootstrap required, opened a new log
I20260709 13:22:50.741333 26131 ts_tablet_manager.cc:1403] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046: Time spent bootstrapping tablet: real 0.061s	user 0.021s	sys 0.022s
I20260709 13:22:50.746310 26132 tablet_bootstrap.cc:654] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.746009 26131 raft_consensus.cc:359] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.748973 26131 raft_consensus.cc:385] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:50.751974 26017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64e40e1547c14815b57cd70e9bc884b2" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046" is_pre_election: true
I20260709 13:22:50.752436 26131 raft_consensus.cc:740] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Initialized, Role: FOLLOWER
W20260709 13:22:50.753649 26056 leader_election.cc:343] T 64e40e1547c14815b57cd70e9bc884b2 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:50.753618 26131 consensus_queue.cc:260] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.754563 26056 leader_election.cc:304] T 64e40e1547c14815b57cd70e9bc884b2 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151; no voters: 6b39791309b04490b6c527fdb0df2046, f5efb07a61464511be72667a6cab7599
I20260709 13:22:50.756208 26140 raft_consensus.cc:2749] T 64e40e1547c14815b57cd70e9bc884b2 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:50.756377 26131 ts_tablet_manager.cc:1434] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046: Time spent starting tablet: real 0.015s	user 0.001s	sys 0.008s
I20260709 13:22:50.751348 26091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9a2bfd8849495da7c36e5712750012" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151" is_pre_election: true
I20260709 13:22:50.757561 26131 tablet_bootstrap.cc:492] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046: Bootstrap starting.
I20260709 13:22:50.751977 26133 tablet_bootstrap.cc:492] T cd842edf9c67436fb5d5e3172a02908f P 86ab90de6ed14e13b5ee21a100ee8151: No bootstrap required, opened a new log
I20260709 13:22:50.759021 26133 ts_tablet_manager.cc:1403] T cd842edf9c67436fb5d5e3172a02908f P 86ab90de6ed14e13b5ee21a100ee8151: Time spent bootstrapping tablet: real 0.061s	user 0.002s	sys 0.018s
I20260709 13:22:50.762347 26133 raft_consensus.cc:359] T cd842edf9c67436fb5d5e3172a02908f P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.763260 26133 raft_consensus.cc:385] T cd842edf9c67436fb5d5e3172a02908f P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:50.763646 26133 raft_consensus.cc:740] T cd842edf9c67436fb5d5e3172a02908f P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Initialized, Role: FOLLOWER
I20260709 13:22:50.764469 26131 tablet_bootstrap.cc:654] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.764568 26133 consensus_queue.cc:260] T cd842edf9c67436fb5d5e3172a02908f P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
W20260709 13:22:50.765837 25906 leader_election.cc:343] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:50.767367 26133 ts_tablet_manager.cc:1434] T cd842edf9c67436fb5d5e3172a02908f P 86ab90de6ed14e13b5ee21a100ee8151: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:22:50.768433 26133 tablet_bootstrap.cc:492] T 4a610fca006245cbac0d5f66538993e4 P 86ab90de6ed14e13b5ee21a100ee8151: Bootstrap starting.
I20260709 13:22:50.775580 26133 tablet_bootstrap.cc:654] T 4a610fca006245cbac0d5f66538993e4 P 86ab90de6ed14e13b5ee21a100ee8151: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.779513 26131 tablet_bootstrap.cc:492] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046: No bootstrap required, opened a new log
I20260709 13:22:50.780210 26131 ts_tablet_manager.cc:1403] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.008s
I20260709 13:22:50.785202 26017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9a2bfd8849495da7c36e5712750012" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046" is_pre_election: true
I20260709 13:22:50.783906 26131 raft_consensus.cc:359] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.786546 26131 raft_consensus.cc:385] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:50.786923 26131 raft_consensus.cc:740] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Initialized, Role: FOLLOWER
W20260709 13:22:50.787034 25907 leader_election.cc:343] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:50.787308 26132 tablet_bootstrap.cc:492] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599: No bootstrap required, opened a new log
I20260709 13:22:50.787593 25907 leader_election.cc:304] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5efb07a61464511be72667a6cab7599; no voters: 6b39791309b04490b6c527fdb0df2046, 86ab90de6ed14e13b5ee21a100ee8151
I20260709 13:22:50.787731 26133 tablet_bootstrap.cc:492] T 4a610fca006245cbac0d5f66538993e4 P 86ab90de6ed14e13b5ee21a100ee8151: No bootstrap required, opened a new log
I20260709 13:22:50.788020 26132 ts_tablet_manager.cc:1403] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599: Time spent bootstrapping tablet: real 0.053s	user 0.012s	sys 0.031s
I20260709 13:22:50.787817 26131 consensus_queue.cc:260] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.788576 26133 ts_tablet_manager.cc:1403] T 4a610fca006245cbac0d5f66538993e4 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.005s
I20260709 13:22:50.788982 26138 raft_consensus.cc:2749] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:50.791579 26132 raft_consensus.cc:359] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.791640 26133 raft_consensus.cc:359] T 4a610fca006245cbac0d5f66538993e4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.792522 26132 raft_consensus.cc:385] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:50.792632 26133 raft_consensus.cc:385] T 4a610fca006245cbac0d5f66538993e4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:50.793000 26132 raft_consensus.cc:740] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5efb07a61464511be72667a6cab7599, State: Initialized, Role: FOLLOWER
I20260709 13:22:50.793138 26133 raft_consensus.cc:740] T 4a610fca006245cbac0d5f66538993e4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Initialized, Role: FOLLOWER
I20260709 13:22:50.793236 26131 ts_tablet_manager.cc:1434] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:22:50.794557 26131 tablet_bootstrap.cc:492] T 64e40e1547c14815b57cd70e9bc884b2 P 6b39791309b04490b6c527fdb0df2046: Bootstrap starting.
I20260709 13:22:50.794109 26132 consensus_queue.cc:260] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.794209 26133 consensus_queue.cc:260] T 4a610fca006245cbac0d5f66538993e4 P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.796555 26132 ts_tablet_manager.cc:1434] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:22:50.799317 26133 ts_tablet_manager.cc:1434] T 4a610fca006245cbac0d5f66538993e4 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:22:50.800621 26133 tablet_bootstrap.cc:492] T 2a1e227945bc461f8c4226b73d895b63 P 86ab90de6ed14e13b5ee21a100ee8151: Bootstrap starting.
I20260709 13:22:50.801503 26132 tablet_bootstrap.cc:492] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599: Bootstrap starting.
I20260709 13:22:50.801909 26131 tablet_bootstrap.cc:654] T 64e40e1547c14815b57cd70e9bc884b2 P 6b39791309b04490b6c527fdb0df2046: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.807586 26133 tablet_bootstrap.cc:654] T 2a1e227945bc461f8c4226b73d895b63 P 86ab90de6ed14e13b5ee21a100ee8151: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.809324 26132 tablet_bootstrap.cc:654] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.820441 26131 tablet_bootstrap.cc:492] T 64e40e1547c14815b57cd70e9bc884b2 P 6b39791309b04490b6c527fdb0df2046: No bootstrap required, opened a new log
I20260709 13:22:50.821038 26131 ts_tablet_manager.cc:1403] T 64e40e1547c14815b57cd70e9bc884b2 P 6b39791309b04490b6c527fdb0df2046: Time spent bootstrapping tablet: real 0.027s	user 0.005s	sys 0.015s
I20260709 13:22:50.821638 26132 tablet_bootstrap.cc:492] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599: No bootstrap required, opened a new log
I20260709 13:22:50.822229 26132 ts_tablet_manager.cc:1403] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.004s
I20260709 13:22:50.824251 26131 raft_consensus.cc:359] T 64e40e1547c14815b57cd70e9bc884b2 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } }
I20260709 13:22:50.825107 26131 raft_consensus.cc:385] T 64e40e1547c14815b57cd70e9bc884b2 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:50.825428 26133 tablet_bootstrap.cc:492] T 2a1e227945bc461f8c4226b73d895b63 P 86ab90de6ed14e13b5ee21a100ee8151: No bootstrap required, opened a new log
I20260709 13:22:50.825430 26131 raft_consensus.cc:740] T 64e40e1547c14815b57cd70e9bc884b2 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Initialized, Role: FOLLOWER
I20260709 13:22:50.825316 26132 raft_consensus.cc:359] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.826143 26132 raft_consensus.cc:385] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:50.826164 26133 ts_tablet_manager.cc:1403] T 2a1e227945bc461f8c4226b73d895b63 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.002s
I20260709 13:22:50.826546 26132 raft_consensus.cc:740] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5efb07a61464511be72667a6cab7599, State: Initialized, Role: FOLLOWER
I20260709 13:22:50.826474 26131 consensus_queue.cc:260] T 64e40e1547c14815b57cd70e9bc884b2 P 6b39791309b04490b6c527fdb0df2046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } }
I20260709 13:22:50.827534 26132 consensus_queue.cc:260] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.829584 26131 ts_tablet_manager.cc:1434] T 64e40e1547c14815b57cd70e9bc884b2 P 6b39791309b04490b6c527fdb0df2046: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:22:50.829464 26133 raft_consensus.cc:359] T 2a1e227945bc461f8c4226b73d895b63 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.830453 26133 raft_consensus.cc:385] T 2a1e227945bc461f8c4226b73d895b63 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:50.830986 26131 tablet_bootstrap.cc:492] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046: Bootstrap starting.
I20260709 13:22:50.833542 26133 raft_consensus.cc:740] T 2a1e227945bc461f8c4226b73d895b63 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Initialized, Role: FOLLOWER
I20260709 13:22:50.834576 26133 consensus_queue.cc:260] T 2a1e227945bc461f8c4226b73d895b63 P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.832268 26132 ts_tablet_manager.cc:1434] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 13:22:50.838913 26132 tablet_bootstrap.cc:492] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599: Bootstrap starting.
I20260709 13:22:50.839751 26133 ts_tablet_manager.cc:1434] T 2a1e227945bc461f8c4226b73d895b63 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:22:50.840969 26133 tablet_bootstrap.cc:492] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151: Bootstrap starting.
I20260709 13:22:50.846782 26132 tablet_bootstrap.cc:654] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.847848 26133 tablet_bootstrap.cc:654] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.847896 26131 tablet_bootstrap.cc:654] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.858296 26133 tablet_bootstrap.cc:492] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151: No bootstrap required, opened a new log
I20260709 13:22:50.858753 26133 ts_tablet_manager.cc:1403] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.008s
I20260709 13:22:50.859464 26131 tablet_bootstrap.cc:492] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046: No bootstrap required, opened a new log
I20260709 13:22:50.859463 26132 tablet_bootstrap.cc:492] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599: No bootstrap required, opened a new log
I20260709 13:22:50.860086 26131 ts_tablet_manager.cc:1403] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.007s
I20260709 13:22:50.860234 26132 ts_tablet_manager.cc:1403] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.005s
I20260709 13:22:50.861573 26133 raft_consensus.cc:359] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.862385 26133 raft_consensus.cc:385] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:50.862727 26133 raft_consensus.cc:740] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Initialized, Role: FOLLOWER
I20260709 13:22:50.862515 26131 raft_consensus.cc:359] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.863281 26131 raft_consensus.cc:385] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:50.863385 26132 raft_consensus.cc:359] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } }
I20260709 13:22:50.863622 26131 raft_consensus.cc:740] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Initialized, Role: FOLLOWER
I20260709 13:22:50.864197 26132 raft_consensus.cc:385] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:50.863605 26133 consensus_queue.cc:260] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.864579 26132 raft_consensus.cc:740] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5efb07a61464511be72667a6cab7599, State: Initialized, Role: FOLLOWER
I20260709 13:22:50.864629 26131 consensus_queue.cc:260] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.865321 26138 raft_consensus.cc:493] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:50.865579 26132 consensus_queue.cc:260] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } }
I20260709 13:22:50.865967 26138 raft_consensus.cc:515] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.868266 26138 leader_election.cc:290] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:50.869807 26131 ts_tablet_manager.cc:1434] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.002s
I20260709 13:22:50.871126 26017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5cf4443b384aee96a6383ab797603c" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046" is_pre_election: true
I20260709 13:22:50.871758 26017 raft_consensus.cc:2468] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 0.
I20260709 13:22:50.873145 25907 leader_election.cc:304] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:50.873509 26131 tablet_bootstrap.cc:492] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046: Bootstrap starting.
I20260709 13:22:50.875027 26091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5cf4443b384aee96a6383ab797603c" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151" is_pre_election: true
W20260709 13:22:50.876847 25906 leader_election.cc:343] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:50.880714 26131 tablet_bootstrap.cc:654] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.883104 26138 raft_consensus.cc:2804] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:50.886303 26133 ts_tablet_manager.cc:1434] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 13:22:50.886437 26138 raft_consensus.cc:493] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:50.888003 26133 tablet_bootstrap.cc:492] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151: Bootstrap starting.
I20260709 13:22:50.893023 26131 tablet_bootstrap.cc:492] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046: No bootstrap required, opened a new log
I20260709 13:22:50.893703 26131 ts_tablet_manager.cc:1403] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.007s
I20260709 13:22:50.895033 26138 raft_consensus.cc:3060] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:50.895637 26133 tablet_bootstrap.cc:654] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.897169 26131 raft_consensus.cc:359] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.898036 26131 raft_consensus.cc:385] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:50.898371 26131 raft_consensus.cc:740] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Initialized, Role: FOLLOWER
I20260709 13:22:50.899384 26131 consensus_queue.cc:260] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.904341 26131 ts_tablet_manager.cc:1434] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:22:50.905616 26131 tablet_bootstrap.cc:492] T db9a2bfd8849495da7c36e5712750012 P 6b39791309b04490b6c527fdb0df2046: Bootstrap starting.
I20260709 13:22:50.912142 26131 tablet_bootstrap.cc:654] T db9a2bfd8849495da7c36e5712750012 P 6b39791309b04490b6c527fdb0df2046: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.917153 26138 raft_consensus.cc:515] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.920396 26017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5cf4443b384aee96a6383ab797603c" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:50.921051 26017 raft_consensus.cc:3060] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:50.927573 26131 tablet_bootstrap.cc:492] T db9a2bfd8849495da7c36e5712750012 P 6b39791309b04490b6c527fdb0df2046: No bootstrap required, opened a new log
I20260709 13:22:50.928234 26131 ts_tablet_manager.cc:1403] T db9a2bfd8849495da7c36e5712750012 P 6b39791309b04490b6c527fdb0df2046: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.010s
I20260709 13:22:50.928979 26017 raft_consensus.cc:2468] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 1.
I20260709 13:22:50.931007 25907 leader_election.cc:304] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:50.931806 26131 raft_consensus.cc:359] T db9a2bfd8849495da7c36e5712750012 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.932701 26131 raft_consensus.cc:385] T db9a2bfd8849495da7c36e5712750012 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:50.933040 26131 raft_consensus.cc:740] T db9a2bfd8849495da7c36e5712750012 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Initialized, Role: FOLLOWER
I20260709 13:22:50.933990 26131 consensus_queue.cc:260] T db9a2bfd8849495da7c36e5712750012 P 6b39791309b04490b6c527fdb0df2046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.935117 26137 raft_consensus.cc:493] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:50.936857 26137 raft_consensus.cc:515] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.937567 26091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5cf4443b384aee96a6383ab797603c" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
W20260709 13:22:50.939254 25906 leader_election.cc:343] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:50.939953 26138 leader_election.cc:290] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 election: Requested vote from peers 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
W20260709 13:22:50.940362 25888 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:50.940943 26138 raft_consensus.cc:2804] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:50.944443 26137 leader_election.cc:290] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:50.942257 26138 raft_consensus.cc:697] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Becoming Leader. State: Replica: f5efb07a61464511be72667a6cab7599, State: Running, Role: LEADER
I20260709 13:22:50.946477 26132 ts_tablet_manager.cc:1434] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599: Time spent starting tablet: real 0.086s	user 0.038s	sys 0.046s
I20260709 13:22:50.946274 26138 consensus_queue.cc:237] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.947825 26132 tablet_bootstrap.cc:492] T 2a1e227945bc461f8c4226b73d895b63 P f5efb07a61464511be72667a6cab7599: Bootstrap starting.
I20260709 13:22:50.944360 26133 tablet_bootstrap.cc:492] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151: No bootstrap required, opened a new log
I20260709 13:22:50.950369 26133 ts_tablet_manager.cc:1403] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151: Time spent bootstrapping tablet: real 0.063s	user 0.023s	sys 0.020s
I20260709 13:22:50.953603 26133 raft_consensus.cc:359] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.954404 26133 raft_consensus.cc:385] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:50.954704 26133 raft_consensus.cc:740] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Initialized, Role: FOLLOWER
I20260709 13:22:50.955117 26131 ts_tablet_manager.cc:1434] T db9a2bfd8849495da7c36e5712750012 P 6b39791309b04490b6c527fdb0df2046: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.001s
I20260709 13:22:50.955677 26132 tablet_bootstrap.cc:654] T 2a1e227945bc461f8c4226b73d895b63 P f5efb07a61464511be72667a6cab7599: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.957841 26131 tablet_bootstrap.cc:492] T 7be8d8fb42cd4874a094e408852cfa43 P 6b39791309b04490b6c527fdb0df2046: Bootstrap starting.
I20260709 13:22:50.936268 26162 raft_consensus.cc:493] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:50.955559 26133 consensus_queue.cc:260] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.961037 26162 raft_consensus.cc:515] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.962900 26133 ts_tablet_manager.cc:1434] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:22:50.964107 26133 tablet_bootstrap.cc:492] T edbb1098626c47439e5e3b7d2edca307 P 86ab90de6ed14e13b5ee21a100ee8151: Bootstrap starting.
I20260709 13:22:50.964430 26162 leader_election.cc:290] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:50.965656 26091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e06dcb93a347bfb244d15cc49cc104" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151" is_pre_election: true
I20260709 13:22:50.966291 26131 tablet_bootstrap.cc:654] T 7be8d8fb42cd4874a094e408852cfa43 P 6b39791309b04490b6c527fdb0df2046: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.966398 26091 raft_consensus.cc:2468] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 0.
I20260709 13:22:50.967949 25906 leader_election.cc:304] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:50.968784 26162 raft_consensus.cc:2804] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:50.969228 26162 raft_consensus.cc:493] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:50.969635 26162 raft_consensus.cc:3060] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:50.971277 26133 tablet_bootstrap.cc:654] T edbb1098626c47439e5e3b7d2edca307 P 86ab90de6ed14e13b5ee21a100ee8151: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:50.964241 26017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e06dcb93a347bfb244d15cc49cc104" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046" is_pre_election: true
I20260709 13:22:50.979797 25942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a1e227945bc461f8c4226b73d895b63" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599" is_pre_election: true
I20260709 13:22:50.979911 26017 raft_consensus.cc:2468] T f6e06dcb93a347bfb244d15cc49cc104 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 0.
I20260709 13:22:50.981009 26132 tablet_bootstrap.cc:492] T 2a1e227945bc461f8c4226b73d895b63 P f5efb07a61464511be72667a6cab7599: No bootstrap required, opened a new log
I20260709 13:22:50.981743 26132 ts_tablet_manager.cc:1403] T 2a1e227945bc461f8c4226b73d895b63 P f5efb07a61464511be72667a6cab7599: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.000s
W20260709 13:22:50.982502 25983 leader_election.cc:343] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:50.985105 26132 raft_consensus.cc:359] T 2a1e227945bc461f8c4226b73d895b63 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.986200 26132 raft_consensus.cc:385] T 2a1e227945bc461f8c4226b73d895b63 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:50.986577 26132 raft_consensus.cc:740] T 2a1e227945bc461f8c4226b73d895b63 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5efb07a61464511be72667a6cab7599, State: Initialized, Role: FOLLOWER
I20260709 13:22:50.987563 26132 consensus_queue.cc:260] T 2a1e227945bc461f8c4226b73d895b63 P f5efb07a61464511be72667a6cab7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.989890 26091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a1e227945bc461f8c4226b73d895b63" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151" is_pre_election: true
I20260709 13:22:50.990693 26091 raft_consensus.cc:2468] T 2a1e227945bc461f8c4226b73d895b63 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 0.
I20260709 13:22:50.990566 26162 raft_consensus.cc:515] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.992274 25981 leader_election.cc:304] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, 86ab90de6ed14e13b5ee21a100ee8151; no voters: f5efb07a61464511be72667a6cab7599
I20260709 13:22:50.990147 26132 ts_tablet_manager.cc:1434] T 2a1e227945bc461f8c4226b73d895b63 P f5efb07a61464511be72667a6cab7599: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:22:50.993458 26137 raft_consensus.cc:2804] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:50.993860 26137 raft_consensus.cc:493] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:50.994210 26137 raft_consensus.cc:3060] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:50.994549 26162 leader_election.cc:290] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 election: Requested vote from peers 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:50.994459 26132 tablet_bootstrap.cc:492] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599: Bootstrap starting.
I20260709 13:22:50.995261 26133 tablet_bootstrap.cc:492] T edbb1098626c47439e5e3b7d2edca307 P 86ab90de6ed14e13b5ee21a100ee8151: No bootstrap required, opened a new log
I20260709 13:22:50.995699 26133 ts_tablet_manager.cc:1403] T edbb1098626c47439e5e3b7d2edca307 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent bootstrapping tablet: real 0.032s	user 0.013s	sys 0.000s
I20260709 13:22:50.997354 25840 catalog_manager.cc:5697] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5efb07a61464511be72667a6cab7599 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "f5efb07a61464511be72667a6cab7599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:50.998435 26091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e06dcb93a347bfb244d15cc49cc104" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
I20260709 13:22:50.998565 26133 raft_consensus.cc:359] T edbb1098626c47439e5e3b7d2edca307 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:50.999046 26091 raft_consensus.cc:3060] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:50.999339 26133 raft_consensus.cc:385] T edbb1098626c47439e5e3b7d2edca307 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:50.999645 26133 raft_consensus.cc:740] T edbb1098626c47439e5e3b7d2edca307 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Initialized, Role: FOLLOWER
I20260709 13:22:50.994401 26017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e06dcb93a347bfb244d15cc49cc104" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:51.000494 26133 consensus_queue.cc:260] T edbb1098626c47439e5e3b7d2edca307 P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.001302 26017 raft_consensus.cc:3060] T f6e06dcb93a347bfb244d15cc49cc104 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:51.002166 26132 tablet_bootstrap.cc:654] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:51.005606 26133 ts_tablet_manager.cc:1434] T edbb1098626c47439e5e3b7d2edca307 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent starting tablet: real 0.010s	user 0.000s	sys 0.007s
I20260709 13:22:51.006498 26137 raft_consensus.cc:515] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.006913 26133 tablet_bootstrap.cc:492] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151: Bootstrap starting.
I20260709 13:22:51.013864 26132 tablet_bootstrap.cc:492] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599: No bootstrap required, opened a new log
I20260709 13:22:51.014541 26132 ts_tablet_manager.cc:1403] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.002s
I20260709 13:22:51.015549 25942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a1e227945bc461f8c4226b73d895b63" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599"
I20260709 13:22:51.016220 25942 raft_consensus.cc:3060] T 2a1e227945bc461f8c4226b73d895b63 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:51.017354 26132 raft_consensus.cc:359] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.018105 26132 raft_consensus.cc:385] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:51.018366 26132 raft_consensus.cc:740] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5efb07a61464511be72667a6cab7599, State: Initialized, Role: FOLLOWER
I20260709 13:22:51.018770 26133 tablet_bootstrap.cc:654] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:51.019050 26132 consensus_queue.cc:260] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.012509 26091 raft_consensus.cc:2468] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 1.
I20260709 13:22:51.020331 26017 raft_consensus.cc:2468] T f6e06dcb93a347bfb244d15cc49cc104 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 1.
I20260709 13:22:51.021174 26132 ts_tablet_manager.cc:1434] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:22:51.021196 25906 leader_election.cc:304] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:51.022423 26132 tablet_bootstrap.cc:492] T cd842edf9c67436fb5d5e3172a02908f P f5efb07a61464511be72667a6cab7599: Bootstrap starting.
I20260709 13:22:51.022558 26162 raft_consensus.cc:2804] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:51.023075 26162 raft_consensus.cc:697] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Becoming Leader. State: Replica: f5efb07a61464511be72667a6cab7599, State: Running, Role: LEADER
I20260709 13:22:51.023898 26162 consensus_queue.cc:237] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.025981 25942 raft_consensus.cc:2468] T 2a1e227945bc461f8c4226b73d895b63 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 1.
I20260709 13:22:51.027751 25983 leader_election.cc:304] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:51.029345 26132 tablet_bootstrap.cc:654] T cd842edf9c67436fb5d5e3172a02908f P f5efb07a61464511be72667a6cab7599: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:51.034763 26131 tablet_bootstrap.cc:492] T 7be8d8fb42cd4874a094e408852cfa43 P 6b39791309b04490b6c527fdb0df2046: No bootstrap required, opened a new log
I20260709 13:22:51.039399 26091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a1e227945bc461f8c4226b73d895b63" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
I20260709 13:22:51.040517 26091 raft_consensus.cc:3060] T 2a1e227945bc461f8c4226b73d895b63 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:51.042193 26132 tablet_bootstrap.cc:492] T cd842edf9c67436fb5d5e3172a02908f P f5efb07a61464511be72667a6cab7599: No bootstrap required, opened a new log
I20260709 13:22:51.042915 26132 ts_tablet_manager.cc:1403] T cd842edf9c67436fb5d5e3172a02908f P f5efb07a61464511be72667a6cab7599: Time spent bootstrapping tablet: real 0.021s	user 0.006s	sys 0.013s
I20260709 13:22:51.043227 26140 raft_consensus.cc:493] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:51.043846 26140 raft_consensus.cc:515] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.045934 26132 raft_consensus.cc:359] T cd842edf9c67436fb5d5e3172a02908f P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.046809 26132 raft_consensus.cc:385] T cd842edf9c67436fb5d5e3172a02908f P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:51.046916 26133 tablet_bootstrap.cc:492] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151: No bootstrap required, opened a new log
I20260709 13:22:51.047156 26132 raft_consensus.cc:740] T cd842edf9c67436fb5d5e3172a02908f P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5efb07a61464511be72667a6cab7599, State: Initialized, Role: FOLLOWER
I20260709 13:22:51.047698 26133 ts_tablet_manager.cc:1403] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent bootstrapping tablet: real 0.041s	user 0.008s	sys 0.009s
I20260709 13:22:51.048097 26132 consensus_queue.cc:260] T cd842edf9c67436fb5d5e3172a02908f P f5efb07a61464511be72667a6cab7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.048774 26140 leader_election.cc:290] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:51.050354 26132 ts_tablet_manager.cc:1434] T cd842edf9c67436fb5d5e3172a02908f P f5efb07a61464511be72667a6cab7599: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:22:51.051188 26133 raft_consensus.cc:359] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.051733 26133 raft_consensus.cc:385] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:51.051935 26133 raft_consensus.cc:740] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Initialized, Role: FOLLOWER
I20260709 13:22:51.052562 26133 consensus_queue.cc:260] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.054356 26133 ts_tablet_manager.cc:1434] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:22:51.062137 26091 raft_consensus.cc:2468] T 2a1e227945bc461f8c4226b73d895b63 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 1.
I20260709 13:22:51.063944 26162 raft_consensus.cc:493] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:51.064527 26162 raft_consensus.cc:515] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } }
I20260709 13:22:51.064941 26175 raft_consensus.cc:2804] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:51.066479 26131 ts_tablet_manager.cc:1403] T 7be8d8fb42cd4874a094e408852cfa43 P 6b39791309b04490b6c527fdb0df2046: Time spent bootstrapping tablet: real 0.106s	user 0.014s	sys 0.003s
I20260709 13:22:51.066773 26162 leader_election.cc:290] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:51.068885 26091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64e40e1547c14815b57cd70e9bc884b2" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151" is_pre_election: true
I20260709 13:22:51.069715 26091 raft_consensus.cc:2468] T 64e40e1547c14815b57cd70e9bc884b2 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 0.
I20260709 13:22:51.071074 25906 leader_election.cc:304] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:51.071218 26172 raft_consensus.cc:493] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:51.072001 26162 raft_consensus.cc:2804] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:51.071856 26172 raft_consensus.cc:515] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
W20260709 13:22:51.072111 25983 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.23.65:45725, user_credentials={real_user=slave}} blocked reactor thread for 44501us
I20260709 13:22:51.072521 26162 raft_consensus.cc:493] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:51.072678 26137 leader_election.cc:290] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 election: Requested vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:51.073060 26162 raft_consensus.cc:3060] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:51.062602 25942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9a2bfd8849495da7c36e5712750012" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599" is_pre_election: true
I20260709 13:22:51.074241 26137 raft_consensus.cc:493] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:51.074684 25942 raft_consensus.cc:2468] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 0.
I20260709 13:22:51.074786 26137 raft_consensus.cc:515] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.076074 26057 leader_election.cc:304] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:51.076949 26140 raft_consensus.cc:2804] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:51.077354 26140 raft_consensus.cc:493] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:51.077677 26140 raft_consensus.cc:3060] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:51.078079 26172 leader_election.cc:290] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:51.078765 26172 raft_consensus.cc:493] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:51.079576 26176 raft_consensus.cc:493] T db9a2bfd8849495da7c36e5712750012 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:51.080065 26176 raft_consensus.cc:515] T db9a2bfd8849495da7c36e5712750012 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.080817 26089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbb1098626c47439e5e3b7d2edca307" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151" is_pre_election: true
I20260709 13:22:51.069979 26131 raft_consensus.cc:359] T 7be8d8fb42cd4874a094e408852cfa43 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.081532 26089 raft_consensus.cc:2468] T edbb1098626c47439e5e3b7d2edca307 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 0.
I20260709 13:22:51.081626 26131 raft_consensus.cc:385] T 7be8d8fb42cd4874a094e408852cfa43 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:51.082355 26175 raft_consensus.cc:697] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 [term 1 LEADER]: Becoming Leader. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Running, Role: LEADER
I20260709 13:22:51.079386 26090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a610fca006245cbac0d5f66538993e4" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151" is_pre_election: true
I20260709 13:22:51.082026 26172 raft_consensus.cc:515] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.083106 26090 raft_consensus.cc:2468] T 4a610fca006245cbac0d5f66538993e4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 0.
I20260709 13:22:51.083246 26175 consensus_queue.cc:237] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.084513 26172 leader_election.cc:290] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:51.082147 26137 leader_election.cc:290] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:51.082057 26131 raft_consensus.cc:740] T 7be8d8fb42cd4874a094e408852cfa43 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Initialized, Role: FOLLOWER
I20260709 13:22:51.089092 26140 raft_consensus.cc:515] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.089847 26090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd842edf9c67436fb5d5e3172a02908f" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151" is_pre_election: true
I20260709 13:22:51.090215 26176 leader_election.cc:290] T db9a2bfd8849495da7c36e5712750012 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:51.089082 26131 consensus_queue.cc:260] T 7be8d8fb42cd4874a094e408852cfa43 P 6b39791309b04490b6c527fdb0df2046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.091348 26140 leader_election.cc:290] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 election: Requested vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:51.093117 26131 ts_tablet_manager.cc:1434] T 7be8d8fb42cd4874a094e408852cfa43 P 6b39791309b04490b6c527fdb0df2046: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 13:22:51.093509 26090 raft_consensus.cc:2468] T cd842edf9c67436fb5d5e3172a02908f P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 0.
I20260709 13:22:51.095461 25942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9a2bfd8849495da7c36e5712750012" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599"
I20260709 13:22:51.096328 25942 raft_consensus.cc:3060] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:51.102447 25981 leader_election.cc:304] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, 86ab90de6ed14e13b5ee21a100ee8151; no voters: 
I20260709 13:22:51.106174 26138 raft_consensus.cc:493] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:51.106930 26138 raft_consensus.cc:515] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.109230 26138 leader_election.cc:290] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:51.110926 26176 raft_consensus.cc:2804] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:51.111363 26176 raft_consensus.cc:493] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:51.111754 26176 raft_consensus.cc:3060] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:51.111492 26090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbb1098626c47439e5e3b7d2edca307" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151" is_pre_election: true
I20260709 13:22:51.112576 26090 raft_consensus.cc:2468] T edbb1098626c47439e5e3b7d2edca307 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 0.
I20260709 13:22:51.113196 26162 raft_consensus.cc:515] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } }
I20260709 13:22:51.114142 25906 leader_election.cc:304] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:51.115554 26138 raft_consensus.cc:2804] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:51.116048 26138 raft_consensus.cc:493] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:51.116456 26138 raft_consensus.cc:3060] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:51.111094 25981 leader_election.cc:304] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, 86ab90de6ed14e13b5ee21a100ee8151; no voters: 
I20260709 13:22:51.117740 26137 raft_consensus.cc:2804] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:51.117936 25981 leader_election.cc:304] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, 86ab90de6ed14e13b5ee21a100ee8151; no voters: 
I20260709 13:22:51.118146 26137 raft_consensus.cc:493] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:51.118587 26137 raft_consensus.cc:3060] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:51.118826 26172 raft_consensus.cc:2804] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:51.119283 26172 raft_consensus.cc:493] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:51.119611 26172 raft_consensus.cc:3060] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:51.122905 26162 leader_election.cc:290] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 election: Requested vote from peers 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:51.125957 26137 raft_consensus.cc:515] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.126853 25942 raft_consensus.cc:2468] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 1.
I20260709 13:22:51.126003 26176 raft_consensus.cc:515] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.130997 26158 raft_consensus.cc:493] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:51.131574 26158 raft_consensus.cc:515] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.128460 26057 leader_election.cc:304] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:51.132083 26015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64e40e1547c14815b57cd70e9bc884b2" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:51.128880 26017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64e40e1547c14815b57cd70e9bc884b2" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046" is_pre_election: true
I20260709 13:22:51.130402 26091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9a2bfd8849495da7c36e5712750012" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151" is_pre_election: true
I20260709 13:22:51.133481 26137 leader_election.cc:290] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 election: Requested vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:51.132788 26140 raft_consensus.cc:2804] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:51.132767 26015 raft_consensus.cc:3060] T 64e40e1547c14815b57cd70e9bc884b2 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:51.135288 26087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbb1098626c47439e5e3b7d2edca307" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
I20260709 13:22:51.136062 26087 raft_consensus.cc:3060] T edbb1098626c47439e5e3b7d2edca307 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:51.136894 26086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a610fca006245cbac0d5f66538993e4" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151" is_pre_election: true
I20260709 13:22:51.135035 26158 leader_election.cc:290] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:51.130517 26016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbb1098626c47439e5e3b7d2edca307" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046" is_pre_election: true
I20260709 13:22:51.138981 26016 raft_consensus.cc:2393] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5efb07a61464511be72667a6cab7599 in current term 1: Already voted for candidate 6b39791309b04490b6c527fdb0df2046 in this term.
I20260709 13:22:51.130600 26088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a610fca006245cbac0d5f66538993e4" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
I20260709 13:22:51.127332 26172 raft_consensus.cc:515] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.129320 26089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64e40e1547c14815b57cd70e9bc884b2" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
I20260709 13:22:51.141628 26089 raft_consensus.cc:3060] T 64e40e1547c14815b57cd70e9bc884b2 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:51.135035 25840 catalog_manager.cc:5697] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5efb07a61464511be72667a6cab7599 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "f5efb07a61464511be72667a6cab7599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:51.145018 26138 raft_consensus.cc:515] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.147979 26138 leader_election.cc:290] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 election: Requested vote from peers 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:51.149209 26088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbb1098626c47439e5e3b7d2edca307" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
I20260709 13:22:51.149726 26087 raft_consensus.cc:2468] T edbb1098626c47439e5e3b7d2edca307 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 1.
I20260709 13:22:51.150749 26176 leader_election.cc:290] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 election: Requested vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:51.151152 25981 leader_election.cc:304] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, 86ab90de6ed14e13b5ee21a100ee8151; no voters: 
I20260709 13:22:51.137601 26086 raft_consensus.cc:2468] T 4a610fca006245cbac0d5f66538993e4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 0.
I20260709 13:22:51.153298 26017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9a2bfd8849495da7c36e5712750012" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:51.137388 26162 raft_consensus.cc:493] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:51.152057 26016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9a2bfd8849495da7c36e5712750012" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046" is_pre_election: true
I20260709 13:22:51.153937 26017 raft_consensus.cc:3060] T db9a2bfd8849495da7c36e5712750012 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:51.155651 25906 leader_election.cc:304] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:51.156121 26162 raft_consensus.cc:515] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.157061 26138 raft_consensus.cc:2804] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:51.157579 26138 raft_consensus.cc:493] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:51.152642 26172 leader_election.cc:290] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 election: Requested vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:51.159459 26086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd842edf9c67436fb5d5e3172a02908f" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
I20260709 13:22:51.159669 26087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be8d8fb42cd4874a094e408852cfa43" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151" is_pre_election: true
I20260709 13:22:51.160296 26162 leader_election.cc:290] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:51.160084 26176 raft_consensus.cc:2804] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:51.162206 26087 raft_consensus.cc:2468] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 0.
I20260709 13:22:51.162640 26176 raft_consensus.cc:697] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 [term 1 LEADER]: Becoming Leader. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Running, Role: LEADER
I20260709 13:22:51.163138 26086 raft_consensus.cc:3060] T cd842edf9c67436fb5d5e3172a02908f P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:51.164443 25906 leader_election.cc:304] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:51.164038 25838 catalog_manager.cc:5697] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b39791309b04490b6c527fdb0df2046 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "6b39791309b04490b6c527fdb0df2046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:51.165501 26158 raft_consensus.cc:2804] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:51.165968 26158 raft_consensus.cc:493] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:51.166320 26158 raft_consensus.cc:3060] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:51.161080 26138 raft_consensus.cc:3060] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:51.164096 26176 consensus_queue.cc:237] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.169809 25942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a610fca006245cbac0d5f66538993e4" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599" is_pre_election: true
I20260709 13:22:51.172225 25941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbb1098626c47439e5e3b7d2edca307" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599" is_pre_election: true
I20260709 13:22:51.173283 25941 raft_consensus.cc:2393] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b39791309b04490b6c527fdb0df2046 in current term 1: Already voted for candidate f5efb07a61464511be72667a6cab7599 in this term.
I20260709 13:22:51.174150 26015 raft_consensus.cc:2468] T 64e40e1547c14815b57cd70e9bc884b2 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 1.
I20260709 13:22:51.155797 26089 raft_consensus.cc:2468] T 64e40e1547c14815b57cd70e9bc884b2 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 1.
I20260709 13:22:51.161041 26140 raft_consensus.cc:697] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Becoming Leader. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Running, Role: LEADER
I20260709 13:22:51.177276 25906 leader_election.cc:304] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:51.177500 25941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9a2bfd8849495da7c36e5712750012" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599" is_pre_election: true
I20260709 13:22:51.178520 25941 raft_consensus.cc:2393] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b39791309b04490b6c527fdb0df2046 in current term 1: Already voted for candidate 86ab90de6ed14e13b5ee21a100ee8151 in this term.
I20260709 13:22:51.178889 26138 raft_consensus.cc:515] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.180143 25942 raft_consensus.cc:2393] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b39791309b04490b6c527fdb0df2046 in current term 1: Already voted for candidate f5efb07a61464511be72667a6cab7599 in this term.
I20260709 13:22:51.179960 26158 raft_consensus.cc:515] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.181180 26138 leader_election.cc:290] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 election: Requested vote from peers 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:51.182605 26089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a610fca006245cbac0d5f66538993e4" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
I20260709 13:22:51.183223 26089 raft_consensus.cc:3060] T 4a610fca006245cbac0d5f66538993e4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:51.184896 26016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a610fca006245cbac0d5f66538993e4" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046" is_pre_election: true
I20260709 13:22:51.185355 26014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbb1098626c47439e5e3b7d2edca307" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:51.185748 26016 raft_consensus.cc:2393] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5efb07a61464511be72667a6cab7599 in current term 1: Already voted for candidate 6b39791309b04490b6c527fdb0df2046 in this term.
I20260709 13:22:51.184239 26181 raft_consensus.cc:493] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:51.186421 26012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a610fca006245cbac0d5f66538993e4" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:51.187247 26012 raft_consensus.cc:2393] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f5efb07a61464511be72667a6cab7599 in current term 1: Already voted for candidate 6b39791309b04490b6c527fdb0df2046 in this term.
I20260709 13:22:51.188278 26011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be8d8fb42cd4874a094e408852cfa43" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:51.188953 26011 raft_consensus.cc:3060] T 7be8d8fb42cd4874a094e408852cfa43 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:51.188602 26140 consensus_queue.cc:237] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.189213 26087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be8d8fb42cd4874a094e408852cfa43" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
I20260709 13:22:51.181280 26162 raft_consensus.cc:2804] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:51.189940 26158 leader_election.cc:290] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 election: Requested vote from peers 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:51.190418 26013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be8d8fb42cd4874a094e408852cfa43" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046" is_pre_election: true
I20260709 13:22:51.190110 26181 raft_consensus.cc:515] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.191977 26087 raft_consensus.cc:3060] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:51.177546 26017 raft_consensus.cc:2468] T db9a2bfd8849495da7c36e5712750012 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 1.
I20260709 13:22:51.194998 25907 leader_election.cc:304] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5efb07a61464511be72667a6cab7599; no voters: 6b39791309b04490b6c527fdb0df2046, 86ab90de6ed14e13b5ee21a100ee8151
I20260709 13:22:51.197654 25940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edbb1098626c47439e5e3b7d2edca307" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599"
I20260709 13:22:51.197652 26138 raft_consensus.cc:2749] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:22:51.198176 25939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd842edf9c67436fb5d5e3172a02908f" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599"
I20260709 13:22:51.199503 25939 raft_consensus.cc:3060] T cd842edf9c67436fb5d5e3172a02908f P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:51.199703 26181 leader_election.cc:290] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:51.194860 26089 raft_consensus.cc:2468] T 4a610fca006245cbac0d5f66538993e4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 1.
I20260709 13:22:51.190548 26162 raft_consensus.cc:697] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Becoming Leader. State: Replica: f5efb07a61464511be72667a6cab7599, State: Running, Role: LEADER
I20260709 13:22:51.201231 25906 leader_election.cc:304] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599; no voters: 6b39791309b04490b6c527fdb0df2046
I20260709 13:22:51.201485 26162 consensus_queue.cc:237] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } }
I20260709 13:22:51.199497 25940 raft_consensus.cc:2393] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b39791309b04490b6c527fdb0df2046 in current term 1: Already voted for candidate f5efb07a61464511be72667a6cab7599 in this term.
I20260709 13:22:51.202478 26138 raft_consensus.cc:2804] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:51.203004 26138 raft_consensus.cc:697] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Becoming Leader. State: Replica: f5efb07a61464511be72667a6cab7599, State: Running, Role: LEADER
I20260709 13:22:51.204006 26138 consensus_queue.cc:237] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.206229 25940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be8d8fb42cd4874a094e408852cfa43" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599" is_pre_election: true
I20260709 13:22:51.207094 25940 raft_consensus.cc:2393] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 86ab90de6ed14e13b5ee21a100ee8151 in current term 1: Already voted for candidate f5efb07a61464511be72667a6cab7599 in this term.
I20260709 13:22:51.197880 25942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd842edf9c67436fb5d5e3172a02908f" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599" is_pre_election: true
I20260709 13:22:51.221144 26011 raft_consensus.cc:2468] T 7be8d8fb42cd4874a094e408852cfa43 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 1.
I20260709 13:22:51.214227 26181 raft_consensus.cc:493] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:51.222889 26181 raft_consensus.cc:515] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.223129 25981 leader_election.cc:304] T db9a2bfd8849495da7c36e5712750012 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046; no voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599
I20260709 13:22:51.224376 26176 raft_consensus.cc:2749] T db9a2bfd8849495da7c36e5712750012 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:51.225179 26181 leader_election.cc:290] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:51.227238 25942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5cf4443b384aee96a6383ab797603c" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599" is_pre_election: true
I20260709 13:22:51.229100 26086 raft_consensus.cc:2468] T cd842edf9c67436fb5d5e3172a02908f P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 1.
I20260709 13:22:51.228276 25840 catalog_manager.cc:5697] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "86ab90de6ed14e13b5ee21a100ee8151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:51.215214 25941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a610fca006245cbac0d5f66538993e4" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599"
I20260709 13:22:51.230939 25981 leader_election.cc:304] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, 86ab90de6ed14e13b5ee21a100ee8151; no voters: 
I20260709 13:22:51.232090 26176 raft_consensus.cc:2804] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:51.232553 26176 raft_consensus.cc:697] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046 [term 1 LEADER]: Becoming Leader. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Running, Role: LEADER
I20260709 13:22:51.233346 26176 consensus_queue.cc:237] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.233909 25836 catalog_manager.cc:5697] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b39791309b04490b6c527fdb0df2046 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "6b39791309b04490b6c527fdb0df2046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:51.235586 25838 catalog_manager.cc:5697] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5efb07a61464511be72667a6cab7599 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "f5efb07a61464511be72667a6cab7599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:51.239182 25983 leader_election.cc:304] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046; no voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599
I20260709 13:22:51.240131 26172 raft_consensus.cc:2749] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:22:51.236990 26087 raft_consensus.cc:2468] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 1.
I20260709 13:22:51.242537 25906 leader_election.cc:304] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:51.243290 26162 raft_consensus.cc:2804] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:51.243775 26162 raft_consensus.cc:697] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Becoming Leader. State: Replica: f5efb07a61464511be72667a6cab7599, State: Running, Role: LEADER
I20260709 13:22:51.244550 26162 consensus_queue.cc:237] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.247807 26011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be8d8fb42cd4874a094e408852cfa43" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046" is_pre_election: true
I20260709 13:22:51.248661 26011 raft_consensus.cc:2393] T 7be8d8fb42cd4874a094e408852cfa43 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 86ab90de6ed14e13b5ee21a100ee8151 in current term 1: Already voted for candidate f5efb07a61464511be72667a6cab7599 in this term.
I20260709 13:22:51.249866 26011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5cf4443b384aee96a6383ab797603c" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046" is_pre_election: true
I20260709 13:22:51.250622 26011 raft_consensus.cc:2393] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 86ab90de6ed14e13b5ee21a100ee8151 in current term 1: Already voted for candidate f5efb07a61464511be72667a6cab7599 in this term.
I20260709 13:22:51.250916 26056 leader_election.cc:304] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151; no voters: 6b39791309b04490b6c527fdb0df2046, f5efb07a61464511be72667a6cab7599
I20260709 13:22:51.251775 26181 raft_consensus.cc:2749] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:51.252688 26056 leader_election.cc:304] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151; no voters: 6b39791309b04490b6c527fdb0df2046, f5efb07a61464511be72667a6cab7599
I20260709 13:22:51.253381 26181 raft_consensus.cc:3060] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:51.256271 25939 raft_consensus.cc:2468] T cd842edf9c67436fb5d5e3172a02908f P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 1.
I20260709 13:22:51.260608 26181 raft_consensus.cc:2749] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:51.275986 25836 catalog_manager.cc:5697] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b39791309b04490b6c527fdb0df2046 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "6b39791309b04490b6c527fdb0df2046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:51.277406 25838 catalog_manager.cc:5697] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5efb07a61464511be72667a6cab7599 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "f5efb07a61464511be72667a6cab7599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:51.279230 25838 catalog_manager.cc:5697] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5efb07a61464511be72667a6cab7599 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "f5efb07a61464511be72667a6cab7599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:51.354422 26087 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db9a2bfd8849495da7c36e5712750012"
dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
mode: GRACEFUL
new_leader_uuid: "f5efb07a61464511be72667a6cab7599"
 from {username='slave'} at 127.0.0.1:51018
I20260709 13:22:51.354991 26087 raft_consensus.cc:606] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Received request to transfer leadership to TS f5efb07a61464511be72667a6cab7599
I20260709 13:22:51.374097 25939 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c5cf4443b384aee96a6383ab797603c"
dest_uuid: "f5efb07a61464511be72667a6cab7599"
mode: GRACEFUL
new_leader_uuid: "6b39791309b04490b6c527fdb0df2046"
 from {username='slave'} at 127.0.0.1:33226
I20260709 13:22:51.374676 25939 raft_consensus.cc:606] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Received request to transfer leadership to TS 6b39791309b04490b6c527fdb0df2046
I20260709 13:22:51.375905 26162 consensus_queue.cc:1048] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:51.383423 26138 raft_consensus.cc:993] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599: : Instructing follower 6b39791309b04490b6c527fdb0df2046 to start an election
I20260709 13:22:51.383911 26138 raft_consensus.cc:1081] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Signalling peer 6b39791309b04490b6c527fdb0df2046 to start an election
I20260709 13:22:51.386498 26017 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c5cf4443b384aee96a6383ab797603c"
dest_uuid: "6b39791309b04490b6c527fdb0df2046"
 from {username='slave'} at 127.0.0.1:48444
I20260709 13:22:51.386590 25939 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7be8d8fb42cd4874a094e408852cfa43"
dest_uuid: "f5efb07a61464511be72667a6cab7599"
mode: GRACEFUL
new_leader_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
 from {username='slave'} at 127.0.0.1:33226
I20260709 13:22:51.387202 25939 raft_consensus.cc:606] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Received request to transfer leadership to TS 86ab90de6ed14e13b5ee21a100ee8151
I20260709 13:22:51.391896 26017 raft_consensus.cc:493] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:51.392445 26017 raft_consensus.cc:3060] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:51.394929 26158 consensus_queue.cc:1048] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:51.399816 26017 raft_consensus.cc:515] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.405433 26017 leader_election.cc:290] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 2 election: Requested vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:51.406327 25939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5cf4443b384aee96a6383ab797603c" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f5efb07a61464511be72667a6cab7599"
I20260709 13:22:51.406993 26087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5cf4443b384aee96a6383ab797603c" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
I20260709 13:22:51.407016 25939 raft_consensus.cc:3055] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:51.407757 25939 raft_consensus.cc:740] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5efb07a61464511be72667a6cab7599, State: Running, Role: LEADER
I20260709 13:22:51.407778 26087 raft_consensus.cc:3060] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:51.408676 25939 consensus_queue.cc:260] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.409967 25939 raft_consensus.cc:3060] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:51.413801 26087 raft_consensus.cc:2468] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 2.
I20260709 13:22:51.415467 25939 raft_consensus.cc:2468] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 2.
I20260709 13:22:51.416514 25981 leader_election.cc:304] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, 86ab90de6ed14e13b5ee21a100ee8151; no voters: 
I20260709 13:22:51.417253 26176 raft_consensus.cc:2804] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:51.417706 26176 raft_consensus.cc:697] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [term 2 LEADER]: Becoming Leader. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Running, Role: LEADER
I20260709 13:22:51.418437 26176 consensus_queue.cc:237] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.424525 25838 catalog_manager.cc:5697] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 reported cstate change: term changed from 1 to 2, leader changed from f5efb07a61464511be72667a6cab7599 (127.14.23.65) to 6b39791309b04490b6c527fdb0df2046 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "6b39791309b04490b6c527fdb0df2046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:51.470510 26176 consensus_queue.cc:1048] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:51.481688 26176 consensus_queue.cc:1048] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:51.501458 26158 consensus_queue.cc:1048] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:51.511642 26138 consensus_queue.cc:1048] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:51.552560 26172 consensus_queue.cc:1048] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:51.563158 26137 consensus_queue.cc:1048] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:51.625801 26138 consensus_queue.cc:1048] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:51.636797 26162 consensus_queue.cc:1048] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:51.708545 26162 consensus_queue.cc:1048] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:51.719985 26181 consensus_queue.cc:1048] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:51.720518 26158 consensus_queue.cc:1048] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:51.722941 26137 consensus_queue.cc:1048] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:51.738724 26137 consensus_queue.cc:1048] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:51.744156 26158 raft_consensus.cc:993] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599: : Instructing follower 86ab90de6ed14e13b5ee21a100ee8151 to start an election
I20260709 13:22:51.744704 26158 raft_consensus.cc:1081] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Signalling peer 86ab90de6ed14e13b5ee21a100ee8151 to start an election
I20260709 13:22:51.754170 26087 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7be8d8fb42cd4874a094e408852cfa43"
dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
 from {username='slave'} at 127.0.0.1:51008
I20260709 13:22:51.754885 26087 raft_consensus.cc:493] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:51.755350 26087 raft_consensus.cc:3060] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:51.755846 26181 consensus_queue.cc:1048] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:51.763152 26087 raft_consensus.cc:515] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.765552 26087 leader_election.cc:290] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 2 election: Requested vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:51.766646 25941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be8d8fb42cd4874a094e408852cfa43" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f5efb07a61464511be72667a6cab7599"
I20260709 13:22:51.766719 26011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be8d8fb42cd4874a094e408852cfa43" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:51.767516 26011 raft_consensus.cc:3060] T 7be8d8fb42cd4874a094e408852cfa43 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:51.767483 25941 raft_consensus.cc:3055] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:51.768024 25941 raft_consensus.cc:740] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5efb07a61464511be72667a6cab7599, State: Running, Role: LEADER
I20260709 13:22:51.768956 25941 consensus_queue.cc:260] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.770713 26162 consensus_queue.cc:1048] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:51.772578 25941 raft_consensus.cc:3060] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:51.774751 26011 raft_consensus.cc:2468] T 7be8d8fb42cd4874a094e408852cfa43 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 2.
I20260709 13:22:51.776189 26056 leader_election.cc:304] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, 86ab90de6ed14e13b5ee21a100ee8151; no voters: 
I20260709 13:22:51.782100 25941 raft_consensus.cc:2468] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 2.
I20260709 13:22:51.786302 26211 raft_consensus.cc:2804] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:51.794487 26210 raft_consensus.cc:993] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151: : Instructing follower f5efb07a61464511be72667a6cab7599 to start an election
I20260709 13:22:51.794577 26138 consensus_queue.cc:1048] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:51.795339 26212 raft_consensus.cc:1081] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Signalling peer f5efb07a61464511be72667a6cab7599 to start an election
I20260709 13:22:51.798799 25939 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "db9a2bfd8849495da7c36e5712750012"
dest_uuid: "f5efb07a61464511be72667a6cab7599"
 from {username='slave'} at 127.0.0.1:33210
I20260709 13:22:51.799505 25939 raft_consensus.cc:493] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:51.799918 25939 raft_consensus.cc:3060] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:51.806216 26211 raft_consensus.cc:697] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 LEADER]: Becoming Leader. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Running, Role: LEADER
I20260709 13:22:51.807302 26211 consensus_queue.cc:237] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.816340 25939 raft_consensus.cc:515] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.817580 25941 raft_consensus.cc:1240] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Rejecting Update request from peer 86ab90de6ed14e13b5ee21a100ee8151 for earlier term 1. Current term is 2. Ops: []
I20260709 13:22:51.819509 25939 leader_election.cc:290] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 2 election: Requested vote from peers 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:51.819903 26011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9a2bfd8849495da7c36e5712750012" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:51.820133 26087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9a2bfd8849495da7c36e5712750012" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
I20260709 13:22:51.820617 26011 raft_consensus.cc:3060] T db9a2bfd8849495da7c36e5712750012 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:51.820771 26087 raft_consensus.cc:3055] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:51.821244 26087 raft_consensus.cc:740] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Running, Role: LEADER
I20260709 13:22:51.822098 26087 consensus_queue.cc:260] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.823217 26087 raft_consensus.cc:3060] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:22:51.826740 26211 consensus_queue.cc:1175] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f5efb07a61464511be72667a6cab7599" responder_term: 2 status { last_received { term: 1 index: 1 } last_committed_idx: 0 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 86ab90de6ed14e13b5ee21a100ee8151 for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:22:51.828765 26011 raft_consensus.cc:2468] T db9a2bfd8849495da7c36e5712750012 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 2.
I20260709 13:22:51.828578 25836 catalog_manager.cc:5697] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 reported cstate change: term changed from 1 to 2, leader changed from f5efb07a61464511be72667a6cab7599 (127.14.23.65) to 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "86ab90de6ed14e13b5ee21a100ee8151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:51.829990 26087 raft_consensus.cc:2468] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 2.
I20260709 13:22:51.830461 25907 leader_election.cc:304] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:51.831283 26138 raft_consensus.cc:2804] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:51.831787 26138 raft_consensus.cc:697] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [term 2 LEADER]: Becoming Leader. State: Replica: f5efb07a61464511be72667a6cab7599, State: Running, Role: LEADER
I20260709 13:22:51.832610 26138 consensus_queue.cc:237] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:51.840468 25840 catalog_manager.cc:5697] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 reported cstate change: term changed from 1 to 2, leader changed from 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67) to f5efb07a61464511be72667a6cab7599 (127.14.23.65). New cstate: current_term: 2 leader_uuid: "f5efb07a61464511be72667a6cab7599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:52.021026 25939 raft_consensus.cc:1275] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Refusing update from remote peer 6b39791309b04490b6c527fdb0df2046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:52.022323 26137 consensus_queue.cc:1048] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:52.030452 26087 raft_consensus.cc:1275] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Refusing update from remote peer 6b39791309b04490b6c527fdb0df2046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:52.032032 26176 consensus_queue.cc:1048] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:52.222344 26011 raft_consensus.cc:1275] T db9a2bfd8849495da7c36e5712750012 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Refusing update from remote peer f5efb07a61464511be72667a6cab7599: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:52.223727 26138 consensus_queue.cc:1048] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:52.239467 25939 raft_consensus.cc:1275] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Refusing update from remote peer 86ab90de6ed14e13b5ee21a100ee8151: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:52.240449 26087 raft_consensus.cc:1275] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Refusing update from remote peer f5efb07a61464511be72667a6cab7599: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:52.241394 26211 consensus_queue.cc:1048] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:52.242561 26138 consensus_queue.cc:1048] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:52.250849 26011 raft_consensus.cc:1275] T 7be8d8fb42cd4874a094e408852cfa43 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Refusing update from remote peer 86ab90de6ed14e13b5ee21a100ee8151: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:52.253139 26181 consensus_queue.cc:1048] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:54.414075 25838 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38910:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:22:54.416257 25838 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:22:54.465098 25925 tablet_service.cc:1511] Processing CreateTablet for tablet c650179e86e34300b488920b6251f0ef (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:54.465327 26000 tablet_service.cc:1511] Processing CreateTablet for tablet c650179e86e34300b488920b6251f0ef (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:54.466598 26000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c650179e86e34300b488920b6251f0ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.466923 26073 tablet_service.cc:1511] Processing CreateTablet for tablet c650179e86e34300b488920b6251f0ef (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:54.467775 25925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c650179e86e34300b488920b6251f0ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.468147 26073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c650179e86e34300b488920b6251f0ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.471104 25927 tablet_service.cc:1511] Processing CreateTablet for tablet 2629764551584920aa90bb1e5af028a4 (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:54.472463 25927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2629764551584920aa90bb1e5af028a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.471104 26081 tablet_service.cc:1511] Processing CreateTablet for tablet 2629764551584920aa90bb1e5af028a4 (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:54.472903 25928 tablet_service.cc:1511] Processing CreateTablet for tablet d9971b1d28824061bed7951411f342d1 (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:54.474189 25928 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9971b1d28824061bed7951411f342d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.473399 26081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2629764551584920aa90bb1e5af028a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.475270 26077 tablet_service.cc:1511] Processing CreateTablet for tablet d9971b1d28824061bed7951411f342d1 (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:54.476722 26077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9971b1d28824061bed7951411f342d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.475348 26005 tablet_service.cc:1511] Processing CreateTablet for tablet d9971b1d28824061bed7951411f342d1 (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:54.478241 26005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9971b1d28824061bed7951411f342d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.478940 26003 tablet_service.cc:1511] Processing CreateTablet for tablet 2629764551584920aa90bb1e5af028a4 (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:54.480261 26003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2629764551584920aa90bb1e5af028a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.490209 25932 tablet_service.cc:1511] Processing CreateTablet for tablet 4833eb35531e43efb9b9ed29308ea39c (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:54.491623 25932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4833eb35531e43efb9b9ed29308ea39c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.494951 26004 tablet_service.cc:1511] Processing CreateTablet for tablet 4833eb35531e43efb9b9ed29308ea39c (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:54.496380 26004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4833eb35531e43efb9b9ed29308ea39c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.498896 25926 tablet_service.cc:1511] Processing CreateTablet for tablet f5867e87b6214e5595f804343a724e00 (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:54.499611 26006 tablet_service.cc:1511] Processing CreateTablet for tablet f5867e87b6214e5595f804343a724e00 (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:54.500273 25926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5867e87b6214e5595f804343a724e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.500926 26006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5867e87b6214e5595f804343a724e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.503652 26076 tablet_service.cc:1511] Processing CreateTablet for tablet 4833eb35531e43efb9b9ed29308ea39c (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:54.505007 26076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4833eb35531e43efb9b9ed29308ea39c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.517604 26075 tablet_service.cc:1511] Processing CreateTablet for tablet f5867e87b6214e5595f804343a724e00 (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:54.519062 26075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5867e87b6214e5595f804343a724e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.534938 26229 tablet_bootstrap.cc:492] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599: Bootstrap starting.
I20260709 13:22:54.567883 26001 tablet_service.cc:1511] Processing CreateTablet for tablet 982a987942324746b31628f39ac33ee4 (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:54.569377 26001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 982a987942324746b31628f39ac33ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.572966 26229 tablet_bootstrap.cc:654] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:54.575963 26230 tablet_bootstrap.cc:492] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046: Bootstrap starting.
I20260709 13:22:54.588593 25926 tablet_service.cc:1511] Processing CreateTablet for tablet 982a987942324746b31628f39ac33ee4 (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:54.590061 25926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 982a987942324746b31628f39ac33ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.594745 26230 tablet_bootstrap.cc:654] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:54.604457 25925 tablet_service.cc:1511] Processing CreateTablet for tablet b0cf16ae335440edba437fd87306c982 (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:54.608639 25925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0cf16ae335440edba437fd87306c982. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.610016 26000 tablet_service.cc:1511] Processing CreateTablet for tablet b0cf16ae335440edba437fd87306c982 (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:54.611546 26000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0cf16ae335440edba437fd87306c982. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.605225 26231 tablet_bootstrap.cc:492] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151: Bootstrap starting.
I20260709 13:22:54.621546 26231 tablet_bootstrap.cc:654] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:54.614349 26081 tablet_service.cc:1511] Processing CreateTablet for tablet 982a987942324746b31628f39ac33ee4 (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:54.628995 26081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 982a987942324746b31628f39ac33ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.629961 26229 tablet_bootstrap.cc:492] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599: No bootstrap required, opened a new log
I20260709 13:22:54.629384 26076 tablet_service.cc:1511] Processing CreateTablet for tablet b0cf16ae335440edba437fd87306c982 (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:54.630626 26229 ts_tablet_manager.cc:1403] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599: Time spent bootstrapping tablet: real 0.096s	user 0.028s	sys 0.027s
I20260709 13:22:54.631130 26076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0cf16ae335440edba437fd87306c982. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.633865 26229 raft_consensus.cc:359] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.634765 26229 raft_consensus.cc:385] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:54.635057 26229 raft_consensus.cc:740] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5efb07a61464511be72667a6cab7599, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.635982 26229 consensus_queue.cc:260] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.638975 26229 ts_tablet_manager.cc:1434] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:22:54.640209 26230 tablet_bootstrap.cc:492] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046: No bootstrap required, opened a new log
I20260709 13:22:54.640844 26230 ts_tablet_manager.cc:1403] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046: Time spent bootstrapping tablet: real 0.065s	user 0.011s	sys 0.001s
I20260709 13:22:54.644155 26230 raft_consensus.cc:359] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.644985 26230 raft_consensus.cc:385] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:54.645289 26230 raft_consensus.cc:740] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.646083 26230 consensus_queue.cc:260] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.648730 26230 ts_tablet_manager.cc:1434] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 13:22:54.649827 26230 tablet_bootstrap.cc:492] T d9971b1d28824061bed7951411f342d1 P 6b39791309b04490b6c527fdb0df2046: Bootstrap starting.
I20260709 13:22:54.640233 26229 tablet_bootstrap.cc:492] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599: Bootstrap starting.
I20260709 13:22:54.656967 26230 tablet_bootstrap.cc:654] T d9971b1d28824061bed7951411f342d1 P 6b39791309b04490b6c527fdb0df2046: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:54.660761 26229 tablet_bootstrap.cc:654] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:54.674594 25932 tablet_service.cc:1511] Processing CreateTablet for tablet 3b3dec675c28487c858ee78013a85c76 (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:54.676075 25932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b3dec675c28487c858ee78013a85c76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.690124 26230 tablet_bootstrap.cc:492] T d9971b1d28824061bed7951411f342d1 P 6b39791309b04490b6c527fdb0df2046: No bootstrap required, opened a new log
I20260709 13:22:54.691676 26230 ts_tablet_manager.cc:1403] T d9971b1d28824061bed7951411f342d1 P 6b39791309b04490b6c527fdb0df2046: Time spent bootstrapping tablet: real 0.042s	user 0.008s	sys 0.011s
I20260709 13:22:54.692938 26229 tablet_bootstrap.cc:492] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599: No bootstrap required, opened a new log
I20260709 13:22:54.693580 26229 ts_tablet_manager.cc:1403] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599: Time spent bootstrapping tablet: real 0.054s	user 0.006s	sys 0.017s
I20260709 13:22:54.693689 26000 tablet_service.cc:1511] Processing CreateTablet for tablet 3b3dec675c28487c858ee78013a85c76 (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:54.695497 26231 tablet_bootstrap.cc:492] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151: No bootstrap required, opened a new log
I20260709 13:22:54.696062 26231 ts_tablet_manager.cc:1403] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151: Time spent bootstrapping tablet: real 0.097s	user 0.018s	sys 0.028s
I20260709 13:22:54.696645 26229 raft_consensus.cc:359] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.697461 26229 raft_consensus.cc:385] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:54.697791 26229 raft_consensus.cc:740] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5efb07a61464511be72667a6cab7599, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.698581 26229 consensus_queue.cc:260] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.699085 26231 raft_consensus.cc:359] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.699913 26231 raft_consensus.cc:385] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:54.700241 26231 raft_consensus.cc:740] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.700865 26076 tablet_service.cc:1511] Processing CreateTablet for tablet 3b3dec675c28487c858ee78013a85c76 (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:54.701093 26231 consensus_queue.cc:260] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.702251 26076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b3dec675c28487c858ee78013a85c76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.703848 26231 ts_tablet_manager.cc:1434] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:22:54.705072 26231 tablet_bootstrap.cc:492] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151: Bootstrap starting.
I20260709 13:22:54.694804 26230 raft_consensus.cc:359] T d9971b1d28824061bed7951411f342d1 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.709470 26230 raft_consensus.cc:385] T d9971b1d28824061bed7951411f342d1 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:54.710851 26230 raft_consensus.cc:740] T d9971b1d28824061bed7951411f342d1 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.710537 26000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b3dec675c28487c858ee78013a85c76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.711728 26231 tablet_bootstrap.cc:654] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:54.713354 26229 ts_tablet_manager.cc:1434] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.000s
I20260709 13:22:54.718215 26230 consensus_queue.cc:260] T d9971b1d28824061bed7951411f342d1 P 6b39791309b04490b6c527fdb0df2046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.720810 26230 ts_tablet_manager.cc:1434] T d9971b1d28824061bed7951411f342d1 P 6b39791309b04490b6c527fdb0df2046: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.000s
I20260709 13:22:54.723981 26229 tablet_bootstrap.cc:492] T f5867e87b6214e5595f804343a724e00 P f5efb07a61464511be72667a6cab7599: Bootstrap starting.
I20260709 13:22:54.724331 25925 tablet_service.cc:1511] Processing CreateTablet for tablet 2e528a28b0d34e54887b528172f326eb (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:54.725757 26230 tablet_bootstrap.cc:492] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046: Bootstrap starting.
I20260709 13:22:54.727653 25925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e528a28b0d34e54887b528172f326eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.727661 26001 tablet_service.cc:1511] Processing CreateTablet for tablet 2e528a28b0d34e54887b528172f326eb (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:54.730346 26081 tablet_service.cc:1511] Processing CreateTablet for tablet 2e528a28b0d34e54887b528172f326eb (DEFAULT_TABLE table=soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:54.731278 26217 raft_consensus.cc:493] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:54.731668 26081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e528a28b0d34e54887b528172f326eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.731822 26217 raft_consensus.cc:515] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.735169 25939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2629764551584920aa90bb1e5af028a4" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599" is_pre_election: true
I20260709 13:22:54.735064 26217 leader_election.cc:290] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
W20260709 13:22:54.737339 25983 leader_election.cc:343] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:54.737782 26230 tablet_bootstrap.cc:654] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:54.738822 26091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2629764551584920aa90bb1e5af028a4" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151" is_pre_election: true
W20260709 13:22:54.740686 25981 leader_election.cc:343] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:54.741139 25981 leader_election.cc:304] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046; no voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599
I20260709 13:22:54.745327 26001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e528a28b0d34e54887b528172f326eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:54.745820 26217 raft_consensus.cc:2749] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:54.752377 26230 tablet_bootstrap.cc:492] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046: No bootstrap required, opened a new log
I20260709 13:22:54.752995 26230 ts_tablet_manager.cc:1403] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.001s
I20260709 13:22:54.756102 26230 raft_consensus.cc:359] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.756809 26230 raft_consensus.cc:385] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:54.757092 26230 raft_consensus.cc:740] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.759056 26231 tablet_bootstrap.cc:492] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151: No bootstrap required, opened a new log
I20260709 13:22:54.759625 26231 ts_tablet_manager.cc:1403] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent bootstrapping tablet: real 0.055s	user 0.010s	sys 0.018s
I20260709 13:22:54.757766 26230 consensus_queue.cc:260] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.762671 26230 ts_tablet_manager.cc:1434] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.001s
I20260709 13:22:54.762724 26231 raft_consensus.cc:359] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.763612 26231 raft_consensus.cc:385] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:54.763918 26231 raft_consensus.cc:740] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.764817 26231 consensus_queue.cc:260] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.767012 26229 tablet_bootstrap.cc:654] T f5867e87b6214e5595f804343a724e00 P f5efb07a61464511be72667a6cab7599: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:54.770076 26231 ts_tablet_manager.cc:1434] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:22:54.771275 26231 tablet_bootstrap.cc:492] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151: Bootstrap starting.
I20260709 13:22:54.773634 26230 tablet_bootstrap.cc:492] T c650179e86e34300b488920b6251f0ef P 6b39791309b04490b6c527fdb0df2046: Bootstrap starting.
I20260709 13:22:54.773886 26229 tablet_bootstrap.cc:492] T f5867e87b6214e5595f804343a724e00 P f5efb07a61464511be72667a6cab7599: No bootstrap required, opened a new log
I20260709 13:22:54.774503 26229 ts_tablet_manager.cc:1403] T f5867e87b6214e5595f804343a724e00 P f5efb07a61464511be72667a6cab7599: Time spent bootstrapping tablet: real 0.051s	user 0.013s	sys 0.000s
I20260709 13:22:54.777757 26229 raft_consensus.cc:359] T f5867e87b6214e5595f804343a724e00 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.778654 26229 raft_consensus.cc:385] T f5867e87b6214e5595f804343a724e00 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:54.779038 26229 raft_consensus.cc:740] T f5867e87b6214e5595f804343a724e00 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5efb07a61464511be72667a6cab7599, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.779877 26229 consensus_queue.cc:260] T f5867e87b6214e5595f804343a724e00 P f5efb07a61464511be72667a6cab7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.780968 26230 tablet_bootstrap.cc:654] T c650179e86e34300b488920b6251f0ef P 6b39791309b04490b6c527fdb0df2046: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:54.785082 26229 ts_tablet_manager.cc:1434] T f5867e87b6214e5595f804343a724e00 P f5efb07a61464511be72667a6cab7599: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:22:54.777792 26231 tablet_bootstrap.cc:654] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:54.786468 26229 tablet_bootstrap.cc:492] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599: Bootstrap starting.
I20260709 13:22:54.795018 26230 tablet_bootstrap.cc:492] T c650179e86e34300b488920b6251f0ef P 6b39791309b04490b6c527fdb0df2046: No bootstrap required, opened a new log
I20260709 13:22:54.795691 26230 ts_tablet_manager.cc:1403] T c650179e86e34300b488920b6251f0ef P 6b39791309b04490b6c527fdb0df2046: Time spent bootstrapping tablet: real 0.022s	user 0.004s	sys 0.015s
I20260709 13:22:54.798718 26230 raft_consensus.cc:359] T c650179e86e34300b488920b6251f0ef P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.798971 26181 raft_consensus.cc:493] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:54.799182 26229 tablet_bootstrap.cc:654] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:54.800139 26181 raft_consensus.cc:515] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.799688 26230 raft_consensus.cc:385] T c650179e86e34300b488920b6251f0ef P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:54.801664 26230 raft_consensus.cc:740] T c650179e86e34300b488920b6251f0ef P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.802343 26181 leader_election.cc:290] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:54.802546 26230 consensus_queue.cc:260] T c650179e86e34300b488920b6251f0ef P 6b39791309b04490b6c527fdb0df2046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.804116 25939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4833eb35531e43efb9b9ed29308ea39c" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599" is_pre_election: true
I20260709 13:22:54.804962 26231 tablet_bootstrap.cc:492] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151: No bootstrap required, opened a new log
I20260709 13:22:54.805377 26230 ts_tablet_manager.cc:1434] T c650179e86e34300b488920b6251f0ef P 6b39791309b04490b6c527fdb0df2046: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:22:54.805540 26231 ts_tablet_manager.cc:1403] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.004s
W20260709 13:22:54.806267 26057 leader_election.cc:343] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:54.806604 26230 tablet_bootstrap.cc:492] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046: Bootstrap starting.
I20260709 13:22:54.808027 26011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4833eb35531e43efb9b9ed29308ea39c" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046" is_pre_election: true
I20260709 13:22:54.808732 26231 raft_consensus.cc:359] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.809514 26231 raft_consensus.cc:385] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:22:54.809645 26056 leader_election.cc:343] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:54.810384 26056 leader_election.cc:304] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151; no voters: 6b39791309b04490b6c527fdb0df2046, f5efb07a61464511be72667a6cab7599
I20260709 13:22:54.811575 26181 raft_consensus.cc:2749] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:54.809849 26231 raft_consensus.cc:740] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.813488 26230 tablet_bootstrap.cc:654] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:54.813704 26231 consensus_queue.cc:260] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.814980 26229 tablet_bootstrap.cc:492] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599: No bootstrap required, opened a new log
I20260709 13:22:54.815539 26229 ts_tablet_manager.cc:1403] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.000s
I20260709 13:22:54.816854 26231 ts_tablet_manager.cc:1434] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.003s
I20260709 13:22:54.818267 26231 tablet_bootstrap.cc:492] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151: Bootstrap starting.
I20260709 13:22:54.818642 26229 raft_consensus.cc:359] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.819456 26229 raft_consensus.cc:385] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:54.819774 26229 raft_consensus.cc:740] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5efb07a61464511be72667a6cab7599, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.820533 26229 consensus_queue.cc:260] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.823321 26229 ts_tablet_manager.cc:1434] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:22:54.824694 26229 tablet_bootstrap.cc:492] T 982a987942324746b31628f39ac33ee4 P f5efb07a61464511be72667a6cab7599: Bootstrap starting.
I20260709 13:22:54.831377 26229 tablet_bootstrap.cc:654] T 982a987942324746b31628f39ac33ee4 P f5efb07a61464511be72667a6cab7599: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:54.837061 26230 tablet_bootstrap.cc:492] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046: No bootstrap required, opened a new log
I20260709 13:22:54.837814 26230 ts_tablet_manager.cc:1403] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046: Time spent bootstrapping tablet: real 0.031s	user 0.019s	sys 0.011s
I20260709 13:22:54.839484 26231 tablet_bootstrap.cc:654] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:54.840456 26229 tablet_bootstrap.cc:492] T 982a987942324746b31628f39ac33ee4 P f5efb07a61464511be72667a6cab7599: No bootstrap required, opened a new log
I20260709 13:22:54.840543 26230 raft_consensus.cc:359] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.841168 26229 ts_tablet_manager.cc:1403] T 982a987942324746b31628f39ac33ee4 P f5efb07a61464511be72667a6cab7599: Time spent bootstrapping tablet: real 0.017s	user 0.007s	sys 0.004s
I20260709 13:22:54.841348 26230 raft_consensus.cc:385] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:54.841665 26230 raft_consensus.cc:740] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.842336 26230 consensus_queue.cc:260] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.844434 26230 ts_tablet_manager.cc:1434] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.002s
I20260709 13:22:54.844575 26229 raft_consensus.cc:359] T 982a987942324746b31628f39ac33ee4 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.845566 26229 raft_consensus.cc:385] T 982a987942324746b31628f39ac33ee4 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:54.846024 26229 raft_consensus.cc:740] T 982a987942324746b31628f39ac33ee4 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5efb07a61464511be72667a6cab7599, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.845567 26230 tablet_bootstrap.cc:492] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046: Bootstrap starting.
I20260709 13:22:54.847687 26229 consensus_queue.cc:260] T 982a987942324746b31628f39ac33ee4 P f5efb07a61464511be72667a6cab7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.848624 26231 tablet_bootstrap.cc:492] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151: No bootstrap required, opened a new log
I20260709 13:22:54.849198 26231 ts_tablet_manager.cc:1403] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent bootstrapping tablet: real 0.031s	user 0.009s	sys 0.006s
I20260709 13:22:54.850355 26229 ts_tablet_manager.cc:1434] T 982a987942324746b31628f39ac33ee4 P f5efb07a61464511be72667a6cab7599: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.005s
I20260709 13:22:54.851581 26229 tablet_bootstrap.cc:492] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599: Bootstrap starting.
I20260709 13:22:54.852383 26231 raft_consensus.cc:359] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.853217 26231 raft_consensus.cc:385] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:54.853519 26231 raft_consensus.cc:740] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.853948 26230 tablet_bootstrap.cc:654] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:54.854465 26231 consensus_queue.cc:260] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.856951 26231 ts_tablet_manager.cc:1434] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:22:54.859414 26231 tablet_bootstrap.cc:492] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151: Bootstrap starting.
I20260709 13:22:54.865759 26229 tablet_bootstrap.cc:654] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:54.866460 26231 tablet_bootstrap.cc:654] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:54.879040 26229 tablet_bootstrap.cc:492] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599: No bootstrap required, opened a new log
I20260709 13:22:54.879761 26229 ts_tablet_manager.cc:1403] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.004s
I20260709 13:22:54.883181 26229 raft_consensus.cc:359] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.884217 26229 raft_consensus.cc:385] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:54.884593 26229 raft_consensus.cc:740] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5efb07a61464511be72667a6cab7599, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.885596 26229 consensus_queue.cc:260] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.889899 26230 tablet_bootstrap.cc:492] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046: No bootstrap required, opened a new log
I20260709 13:22:54.890592 26230 ts_tablet_manager.cc:1403] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046: Time spent bootstrapping tablet: real 0.045s	user 0.024s	sys 0.013s
I20260709 13:22:54.893647 26230 raft_consensus.cc:359] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.894320 26230 raft_consensus.cc:385] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:54.894622 26230 raft_consensus.cc:740] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.895447 26230 consensus_queue.cc:260] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.897234 26231 tablet_bootstrap.cc:492] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151: No bootstrap required, opened a new log
I20260709 13:22:54.897876 26231 ts_tablet_manager.cc:1403] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent bootstrapping tablet: real 0.039s	user 0.008s	sys 0.011s
I20260709 13:22:54.901062 26231 raft_consensus.cc:359] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.901803 26231 raft_consensus.cc:385] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:54.902107 26231 raft_consensus.cc:740] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.902839 26231 consensus_queue.cc:260] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.905138 26230 ts_tablet_manager.cc:1434] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:22:54.906136 26181 raft_consensus.cc:493] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:54.906478 26230 tablet_bootstrap.cc:492] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046: Bootstrap starting.
I20260709 13:22:54.906771 26181 raft_consensus.cc:515] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.909339 26181 leader_election.cc:290] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:54.911700 26011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9971b1d28824061bed7951411f342d1" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046" is_pre_election: true
I20260709 13:22:54.912495 26011 raft_consensus.cc:2468] T d9971b1d28824061bed7951411f342d1 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 0.
I20260709 13:22:54.914305 25939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9971b1d28824061bed7951411f342d1" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599" is_pre_election: true
I20260709 13:22:54.915038 25939 raft_consensus.cc:2468] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 0.
I20260709 13:22:54.915333 26230 tablet_bootstrap.cc:654] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:54.916072 26056 leader_election.cc:304] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, 86ab90de6ed14e13b5ee21a100ee8151; no voters: 
I20260709 13:22:54.917286 26181 raft_consensus.cc:2804] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:54.917663 26181 raft_consensus.cc:493] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:54.918004 26181 raft_consensus.cc:3060] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:54.918792 26231 ts_tablet_manager.cc:1434] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:22:54.922297 26231 tablet_bootstrap.cc:492] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151: Bootstrap starting.
I20260709 13:22:54.924789 26229 ts_tablet_manager.cc:1434] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599: Time spent starting tablet: real 0.045s	user 0.019s	sys 0.003s
I20260709 13:22:54.926050 26229 tablet_bootstrap.cc:492] T b0cf16ae335440edba437fd87306c982 P f5efb07a61464511be72667a6cab7599: Bootstrap starting.
I20260709 13:22:54.925590 26181 raft_consensus.cc:515] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.929065 25939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9971b1d28824061bed7951411f342d1" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599"
I20260709 13:22:54.929667 25939 raft_consensus.cc:3060] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:54.932088 26229 tablet_bootstrap.cc:654] T b0cf16ae335440edba437fd87306c982 P f5efb07a61464511be72667a6cab7599: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:54.932950 26231 tablet_bootstrap.cc:654] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:54.937422 25939 raft_consensus.cc:2468] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 1.
I20260709 13:22:54.937678 26181 leader_election.cc:290] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 election: Requested vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:54.939177 26057 leader_election.cc:304] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:54.939920 26011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9971b1d28824061bed7951411f342d1" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:54.940553 26011 raft_consensus.cc:3060] T d9971b1d28824061bed7951411f342d1 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:54.941725 26231 tablet_bootstrap.cc:492] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151: No bootstrap required, opened a new log
I20260709 13:22:54.942313 26231 ts_tablet_manager.cc:1403] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.000s
I20260709 13:22:54.944861 26229 tablet_bootstrap.cc:492] T b0cf16ae335440edba437fd87306c982 P f5efb07a61464511be72667a6cab7599: No bootstrap required, opened a new log
I20260709 13:22:54.945531 26229 ts_tablet_manager.cc:1403] T b0cf16ae335440edba437fd87306c982 P f5efb07a61464511be72667a6cab7599: Time spent bootstrapping tablet: real 0.020s	user 0.006s	sys 0.003s
I20260709 13:22:54.945782 26181 raft_consensus.cc:2804] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:54.945408 26231 raft_consensus.cc:359] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.948001 26231 raft_consensus.cc:385] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:54.948483 26231 raft_consensus.cc:740] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.949272 26229 raft_consensus.cc:359] T b0cf16ae335440edba437fd87306c982 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.949353 26231 consensus_queue.cc:260] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.950448 26229 raft_consensus.cc:385] T b0cf16ae335440edba437fd87306c982 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:54.950834 26229 raft_consensus.cc:740] T b0cf16ae335440edba437fd87306c982 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5efb07a61464511be72667a6cab7599, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.952450 26251 raft_consensus.cc:493] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:54.953055 26251 raft_consensus.cc:515] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.957192 26017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2629764551584920aa90bb1e5af028a4" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046" is_pre_election: true
I20260709 13:22:54.957990 26251 leader_election.cc:290] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:54.958001 26017 raft_consensus.cc:2468] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 0.
I20260709 13:22:54.951771 26229 consensus_queue.cc:260] T b0cf16ae335440edba437fd87306c982 P f5efb07a61464511be72667a6cab7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.960374 26011 raft_consensus.cc:2468] T d9971b1d28824061bed7951411f342d1 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 1.
I20260709 13:22:54.960964 26229 ts_tablet_manager.cc:1434] T b0cf16ae335440edba437fd87306c982 P f5efb07a61464511be72667a6cab7599: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:22:54.962131 26229 tablet_bootstrap.cc:492] T 3b3dec675c28487c858ee78013a85c76 P f5efb07a61464511be72667a6cab7599: Bootstrap starting.
I20260709 13:22:54.963963 26230 tablet_bootstrap.cc:492] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046: No bootstrap required, opened a new log
I20260709 13:22:54.966832 26230 ts_tablet_manager.cc:1403] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046: Time spent bootstrapping tablet: real 0.061s	user 0.018s	sys 0.018s
I20260709 13:22:54.967095 25907 leader_election.cc:304] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:54.968370 26229 tablet_bootstrap.cc:654] T 3b3dec675c28487c858ee78013a85c76 P f5efb07a61464511be72667a6cab7599: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:54.968340 26251 raft_consensus.cc:2804] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:54.973143 26251 raft_consensus.cc:493] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:54.973606 26257 raft_consensus.cc:493] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:54.975386 26258 raft_consensus.cc:493] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:54.975929 26258 raft_consensus.cc:515] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
W20260709 13:22:54.977075 25888 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:54.978010 26091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2629764551584920aa90bb1e5af028a4" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151" is_pre_election: true
I20260709 13:22:54.974277 26257 raft_consensus.cc:515] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.978825 26091 raft_consensus.cc:2468] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 0.
I20260709 13:22:54.980525 26230 raft_consensus.cc:359] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.981159 25941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c650179e86e34300b488920b6251f0ef" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599" is_pre_election: true
I20260709 13:22:54.981292 25939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2629764551584920aa90bb1e5af028a4" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599" is_pre_election: true
I20260709 13:22:54.981890 25941 raft_consensus.cc:2468] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 0.
I20260709 13:22:54.983053 26257 leader_election.cc:290] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:54.983433 26057 leader_election.cc:304] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:54.984210 26258 leader_election.cc:290] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:54.984794 26258 raft_consensus.cc:2804] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:54.985176 26258 raft_consensus.cc:493] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:54.985507 26258 raft_consensus.cc:3060] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:54.973603 26251 raft_consensus.cc:3060] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:54.986373 26231 ts_tablet_manager.cc:1434] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent starting tablet: real 0.044s	user 0.013s	sys 0.001s
I20260709 13:22:54.987768 26230 raft_consensus.cc:385] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:54.988129 26230 raft_consensus.cc:740] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.989044 26230 consensus_queue.cc:260] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.991637 26230 ts_tablet_manager.cc:1434] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:22:54.992879 26230 tablet_bootstrap.cc:492] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046: Bootstrap starting.
I20260709 13:22:54.994656 26258 raft_consensus.cc:515] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.988026 26257 raft_consensus.cc:493] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:54.995980 26257 raft_consensus.cc:515] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:54.997792 26217 raft_consensus.cc:493] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:55.000319 26217 raft_consensus.cc:515] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.001246 26230 tablet_bootstrap.cc:654] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:55.002619 26217 leader_election.cc:290] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:55.003374 25942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5867e87b6214e5595f804343a724e00" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599" is_pre_election: true
I20260709 13:22:54.998694 25941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c650179e86e34300b488920b6251f0ef" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599"
I20260709 13:22:55.004101 25942 raft_consensus.cc:2468] T f5867e87b6214e5595f804343a724e00 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 0.
I20260709 13:22:55.004297 25941 raft_consensus.cc:3060] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:55.004292 26086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5867e87b6214e5595f804343a724e00" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151" is_pre_election: true
I20260709 13:22:55.005532 25983 leader_election.cc:304] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:55.006426 26217 raft_consensus.cc:2804] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:55.006808 26217 raft_consensus.cc:493] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:55.007136 26217 raft_consensus.cc:3060] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:55.011204 26011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c650179e86e34300b488920b6251f0ef" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046" is_pre_election: true
I20260709 13:22:55.011976 26011 raft_consensus.cc:2468] T c650179e86e34300b488920b6251f0ef P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 0.
I20260709 13:22:55.013172 25941 raft_consensus.cc:2468] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 1.
I20260709 13:22:54.987807 26181 raft_consensus.cc:697] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Becoming Leader. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Running, Role: LEADER
I20260709 13:22:55.014781 26057 leader_election.cc:304] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:54.999660 26258 leader_election.cc:290] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 election: Requested vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:55.021226 26017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c650179e86e34300b488920b6251f0ef" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:55.017041 26217 raft_consensus.cc:515] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.033886 25941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5867e87b6214e5595f804343a724e00" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599" is_pre_election: true
I20260709 13:22:55.034432 26015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5867e87b6214e5595f804343a724e00" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046" is_pre_election: true
I20260709 13:22:55.035126 25941 raft_consensus.cc:2468] T f5867e87b6214e5595f804343a724e00 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 0.
I20260709 13:22:55.034461 26017 raft_consensus.cc:3060] T c650179e86e34300b488920b6251f0ef P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:55.036175 26217 leader_election.cc:290] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 election: Requested vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:55.015708 26256 raft_consensus.cc:2804] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:55.036458 26057 leader_election.cc:304] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:55.013662 26251 raft_consensus.cc:515] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.037320 26091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5867e87b6214e5595f804343a724e00" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
I20260709 13:22:55.015350 26011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2629764551584920aa90bb1e5af028a4" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046" is_pre_election: true
I20260709 13:22:55.038892 26011 raft_consensus.cc:2468] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 0.
I20260709 13:22:55.037314 26256 raft_consensus.cc:697] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Becoming Leader. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Running, Role: LEADER
I20260709 13:22:55.035456 26015 raft_consensus.cc:2393] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 86ab90de6ed14e13b5ee21a100ee8151 in current term 1: Already voted for candidate 6b39791309b04490b6c527fdb0df2046 in this term.
I20260709 13:22:55.039783 25941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5867e87b6214e5595f804343a724e00" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599"
I20260709 13:22:55.039165 26251 leader_election.cc:290] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 election: Requested vote from peers 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:55.040160 26256 consensus_queue.cc:237] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.041985 26015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2629764551584920aa90bb1e5af028a4" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:55.042644 26015 raft_consensus.cc:3060] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:55.039950 25939 raft_consensus.cc:2393] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 86ab90de6ed14e13b5ee21a100ee8151 in current term 1: Already voted for candidate f5efb07a61464511be72667a6cab7599 in this term.
I20260709 13:22:55.044909 26229 tablet_bootstrap.cc:492] T 3b3dec675c28487c858ee78013a85c76 P f5efb07a61464511be72667a6cab7599: No bootstrap required, opened a new log
I20260709 13:22:55.045439 26229 ts_tablet_manager.cc:1403] T 3b3dec675c28487c858ee78013a85c76 P f5efb07a61464511be72667a6cab7599: Time spent bootstrapping tablet: real 0.084s	user 0.019s	sys 0.025s
I20260709 13:22:55.029300 26086 raft_consensus.cc:2468] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 0.
I20260709 13:22:55.040597 25941 raft_consensus.cc:3060] T f5867e87b6214e5595f804343a724e00 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:55.047833 26056 leader_election.cc:304] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, 86ab90de6ed14e13b5ee21a100ee8151; no voters: 
I20260709 13:22:55.048873 26258 raft_consensus.cc:2804] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:55.049371 26258 raft_consensus.cc:493] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:55.049810 26258 raft_consensus.cc:3060] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:55.047338 26255 raft_consensus.cc:2804] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:55.054381 26255 raft_consensus.cc:493] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:55.054791 26255 raft_consensus.cc:3060] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:55.056375 26086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2629764551584920aa90bb1e5af028a4" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
I20260709 13:22:55.057395 26229 raft_consensus.cc:359] T 3b3dec675c28487c858ee78013a85c76 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.058200 26229 raft_consensus.cc:385] T 3b3dec675c28487c858ee78013a85c76 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:55.058475 26229 raft_consensus.cc:740] T 3b3dec675c28487c858ee78013a85c76 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5efb07a61464511be72667a6cab7599, State: Initialized, Role: FOLLOWER
I20260709 13:22:55.059173 26229 consensus_queue.cc:260] T 3b3dec675c28487c858ee78013a85c76 P f5efb07a61464511be72667a6cab7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.061753 26258 raft_consensus.cc:515] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.062335 25941 raft_consensus.cc:2468] T f5867e87b6214e5595f804343a724e00 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 1.
I20260709 13:22:55.063678 25983 leader_election.cc:304] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, f5efb07a61464511be72667a6cab7599; no voters: 86ab90de6ed14e13b5ee21a100ee8151
I20260709 13:22:55.064236 26258 leader_election.cc:290] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 election: Requested vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:55.064908 26217 raft_consensus.cc:2804] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:55.065425 26217 raft_consensus.cc:697] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046 [term 1 LEADER]: Becoming Leader. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Running, Role: LEADER
I20260709 13:22:55.063980 26229 ts_tablet_manager.cc:1434] T 3b3dec675c28487c858ee78013a85c76 P f5efb07a61464511be72667a6cab7599: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.003s
I20260709 13:22:55.066483 26011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2629764551584920aa90bb1e5af028a4" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:55.067270 26229 tablet_bootstrap.cc:492] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599: Bootstrap starting.
I20260709 13:22:55.066277 26217 consensus_queue.cc:237] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.077319 26086 raft_consensus.cc:2393] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f5efb07a61464511be72667a6cab7599 in current term 1: Already voted for candidate 86ab90de6ed14e13b5ee21a100ee8151 in this term.
I20260709 13:22:55.076726 25839 catalog_manager.cc:5697] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b39791309b04490b6c527fdb0df2046 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "6b39791309b04490b6c527fdb0df2046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:55.073729 26229 tablet_bootstrap.cc:654] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:55.067799 25939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2629764551584920aa90bb1e5af028a4" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599"
I20260709 13:22:55.082350 25939 raft_consensus.cc:2393] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 86ab90de6ed14e13b5ee21a100ee8151 in current term 1: Already voted for candidate f5efb07a61464511be72667a6cab7599 in this term.
I20260709 13:22:55.083580 26057 leader_election.cc:304] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151; no voters: 6b39791309b04490b6c527fdb0df2046, f5efb07a61464511be72667a6cab7599
I20260709 13:22:55.084759 26258 raft_consensus.cc:2749] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:22:55.088431 26017 raft_consensus.cc:2468] T c650179e86e34300b488920b6251f0ef P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 1.
I20260709 13:22:55.093412 26255 raft_consensus.cc:515] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.094728 26229 tablet_bootstrap.cc:492] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599: No bootstrap required, opened a new log
I20260709 13:22:55.095300 26229 ts_tablet_manager.cc:1403] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599: Time spent bootstrapping tablet: real 0.028s	user 0.016s	sys 0.001s
I20260709 13:22:55.096562 25939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5867e87b6214e5595f804343a724e00" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599"
I20260709 13:22:55.015260 26181 consensus_queue.cc:237] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.033133 26257 leader_election.cc:290] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:54.987627 26231 tablet_bootstrap.cc:492] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151: Bootstrap starting.
I20260709 13:22:55.033855 26230 tablet_bootstrap.cc:492] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046: No bootstrap required, opened a new log
I20260709 13:22:55.098135 26229 raft_consensus.cc:359] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.098641 26230 ts_tablet_manager.cc:1403] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046: Time spent bootstrapping tablet: real 0.106s	user 0.015s	sys 0.023s
I20260709 13:22:55.097420 25939 raft_consensus.cc:2393] T f5867e87b6214e5595f804343a724e00 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 86ab90de6ed14e13b5ee21a100ee8151 in current term 1: Already voted for candidate 6b39791309b04490b6c527fdb0df2046 in this term.
I20260709 13:22:55.100658 26017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5867e87b6214e5595f804343a724e00" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:55.101392 26230 raft_consensus.cc:359] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.102156 26230 raft_consensus.cc:385] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:55.102229 26056 leader_election.cc:304] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151; no voters: 6b39791309b04490b6c527fdb0df2046, f5efb07a61464511be72667a6cab7599
I20260709 13:22:55.103156 26257 raft_consensus.cc:2749] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:22:55.103626 26230 raft_consensus.cc:740] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Initialized, Role: FOLLOWER
I20260709 13:22:55.103912 26255 leader_election.cc:290] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 election: Requested vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:55.104401 26230 consensus_queue.cc:260] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.105726 26231 tablet_bootstrap.cc:654] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:55.106523 26230 ts_tablet_manager.cc:1434] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:22:55.106949 26229 raft_consensus.cc:385] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:55.107224 26229 raft_consensus.cc:740] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5efb07a61464511be72667a6cab7599, State: Initialized, Role: FOLLOWER
I20260709 13:22:55.108492 26230 tablet_bootstrap.cc:492] T 2e528a28b0d34e54887b528172f326eb P 6b39791309b04490b6c527fdb0df2046: Bootstrap starting.
I20260709 13:22:55.097291 26015 raft_consensus.cc:2468] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 1.
I20260709 13:22:55.111459 26229 consensus_queue.cc:260] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.112469 25907 leader_election.cc:304] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, f5efb07a61464511be72667a6cab7599; no voters: 86ab90de6ed14e13b5ee21a100ee8151
I20260709 13:22:55.113592 26231 tablet_bootstrap.cc:492] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151: No bootstrap required, opened a new log
I20260709 13:22:55.114167 26231 ts_tablet_manager.cc:1403] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent bootstrapping tablet: real 0.127s	user 0.006s	sys 0.007s
I20260709 13:22:55.117229 26231 raft_consensus.cc:359] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.118330 26231 raft_consensus.cc:385] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:55.118649 26231 raft_consensus.cc:740] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Initialized, Role: FOLLOWER
I20260709 13:22:55.123172 26251 raft_consensus.cc:2804] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:55.123790 26251 raft_consensus.cc:697] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Becoming Leader. State: Replica: f5efb07a61464511be72667a6cab7599, State: Running, Role: LEADER
I20260709 13:22:55.123981 26231 consensus_queue.cc:260] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.124630 26251 consensus_queue.cc:237] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.126964 26231 ts_tablet_manager.cc:1434] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent starting tablet: real 0.012s	user 0.011s	sys 0.000s
I20260709 13:22:55.128003 26231 tablet_bootstrap.cc:492] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151: Bootstrap starting.
I20260709 13:22:55.129616 26229 ts_tablet_manager.cc:1434] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.002s
I20260709 13:22:55.131384 26230 tablet_bootstrap.cc:654] T 2e528a28b0d34e54887b528172f326eb P 6b39791309b04490b6c527fdb0df2046: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:55.132018 25839 catalog_manager.cc:5697] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "86ab90de6ed14e13b5ee21a100ee8151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:55.134707 26231 tablet_bootstrap.cc:654] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:55.146162 26230 tablet_bootstrap.cc:492] T 2e528a28b0d34e54887b528172f326eb P 6b39791309b04490b6c527fdb0df2046: No bootstrap required, opened a new log
I20260709 13:22:55.146737 26230 ts_tablet_manager.cc:1403] T 2e528a28b0d34e54887b528172f326eb P 6b39791309b04490b6c527fdb0df2046: Time spent bootstrapping tablet: real 0.038s	user 0.018s	sys 0.003s
I20260709 13:22:55.146905 26231 tablet_bootstrap.cc:492] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151: No bootstrap required, opened a new log
I20260709 13:22:55.147521 26231 ts_tablet_manager.cc:1403] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.004s
I20260709 13:22:55.149902 26230 raft_consensus.cc:359] T 2e528a28b0d34e54887b528172f326eb P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.150856 26230 raft_consensus.cc:385] T 2e528a28b0d34e54887b528172f326eb P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:55.150568 26231 raft_consensus.cc:359] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.151240 26230 raft_consensus.cc:740] T 2e528a28b0d34e54887b528172f326eb P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Initialized, Role: FOLLOWER
I20260709 13:22:55.151402 26231 raft_consensus.cc:385] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:55.151750 26231 raft_consensus.cc:740] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Initialized, Role: FOLLOWER
I20260709 13:22:55.152313 26230 consensus_queue.cc:260] T 2e528a28b0d34e54887b528172f326eb P 6b39791309b04490b6c527fdb0df2046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.152474 26231 consensus_queue.cc:260] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.154640 26230 ts_tablet_manager.cc:1434] T 2e528a28b0d34e54887b528172f326eb P 6b39791309b04490b6c527fdb0df2046: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:22:55.155088 26231 ts_tablet_manager.cc:1434] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:22:55.155906 26231 tablet_bootstrap.cc:492] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151: Bootstrap starting.
I20260709 13:22:55.153023 25839 catalog_manager.cc:5697] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "86ab90de6ed14e13b5ee21a100ee8151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:55.161317 26231 tablet_bootstrap.cc:654] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:55.168282 25836 catalog_manager.cc:5697] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5efb07a61464511be72667a6cab7599 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "f5efb07a61464511be72667a6cab7599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:55.170971 26231 tablet_bootstrap.cc:492] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151: No bootstrap required, opened a new log
I20260709 13:22:55.171553 26231 ts_tablet_manager.cc:1403] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.004s
I20260709 13:22:55.174002 26231 raft_consensus.cc:359] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.174698 26231 raft_consensus.cc:385] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:55.174957 26231 raft_consensus.cc:740] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Initialized, Role: FOLLOWER
I20260709 13:22:55.175570 26231 consensus_queue.cc:260] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.177615 26231 ts_tablet_manager.cc:1434] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:22:55.181841 26181 raft_consensus.cc:493] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:55.182502 26181 raft_consensus.cc:515] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.184705 26181 leader_election.cc:290] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:55.186638 25939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "982a987942324746b31628f39ac33ee4" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599" is_pre_election: true
I20260709 13:22:55.187422 25939 raft_consensus.cc:2468] T 982a987942324746b31628f39ac33ee4 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 0.
I20260709 13:22:55.188869 26015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "982a987942324746b31628f39ac33ee4" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046" is_pre_election: true
I20260709 13:22:55.189009 26057 leader_election.cc:304] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:55.189638 26015 raft_consensus.cc:2468] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 0.
I20260709 13:22:55.190344 26181 raft_consensus.cc:2804] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:55.190780 26181 raft_consensus.cc:493] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:55.191061 26181 raft_consensus.cc:3060] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:55.197075 26181 raft_consensus.cc:515] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.198979 26181 leader_election.cc:290] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 election: Requested vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:55.199736 25939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "982a987942324746b31628f39ac33ee4" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599"
I20260709 13:22:55.200052 26015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "982a987942324746b31628f39ac33ee4" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:55.200456 25939 raft_consensus.cc:3060] T 982a987942324746b31628f39ac33ee4 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:55.200618 26015 raft_consensus.cc:3060] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:55.207921 25939 raft_consensus.cc:2468] T 982a987942324746b31628f39ac33ee4 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 1.
I20260709 13:22:55.207921 26015 raft_consensus.cc:2468] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 1.
I20260709 13:22:55.209319 26057 leader_election.cc:304] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:55.210033 26181 raft_consensus.cc:2804] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:55.210449 26181 raft_consensus.cc:697] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Becoming Leader. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Running, Role: LEADER
I20260709 13:22:55.211166 26181 consensus_queue.cc:237] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.255306 26217 raft_consensus.cc:493] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:55.256958 26217 raft_consensus.cc:515] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.259966 25839 catalog_manager.cc:5697] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "86ab90de6ed14e13b5ee21a100ee8151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:55.261530 26217 leader_election.cc:290] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:55.262745 25939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4833eb35531e43efb9b9ed29308ea39c" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599" is_pre_election: true
I20260709 13:22:55.263293 26086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4833eb35531e43efb9b9ed29308ea39c" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151" is_pre_election: true
I20260709 13:22:55.264084 25939 raft_consensus.cc:2468] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 0.
I20260709 13:22:55.264477 26086 raft_consensus.cc:2468] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 0.
I20260709 13:22:55.266683 25983 leader_election.cc:304] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:55.268931 26217 raft_consensus.cc:2804] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:55.270327 26217 raft_consensus.cc:493] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:55.270881 26217 raft_consensus.cc:3060] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:55.278432 26181 raft_consensus.cc:493] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:55.279610 26181 raft_consensus.cc:515] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.284001 26181 leader_election.cc:290] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:55.284912 25939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0cf16ae335440edba437fd87306c982" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599" is_pre_election: true
I20260709 13:22:55.286077 25939 raft_consensus.cc:2468] T b0cf16ae335440edba437fd87306c982 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 0.
I20260709 13:22:55.286087 26015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0cf16ae335440edba437fd87306c982" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046" is_pre_election: true
I20260709 13:22:55.287233 26015 raft_consensus.cc:2468] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 0.
I20260709 13:22:55.287070 26217 raft_consensus.cc:515] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.287847 26057 leader_election.cc:304] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:55.289646 26181 raft_consensus.cc:2804] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:55.289763 26217 leader_election.cc:290] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 election: Requested vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:55.290272 26181 raft_consensus.cc:493] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:55.290738 26181 raft_consensus.cc:3060] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:55.291029 26086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4833eb35531e43efb9b9ed29308ea39c" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
I20260709 13:22:55.291132 25939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4833eb35531e43efb9b9ed29308ea39c" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599"
I20260709 13:22:55.291761 26086 raft_consensus.cc:3060] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:55.292011 25939 raft_consensus.cc:3060] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:55.303293 26086 raft_consensus.cc:2468] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 1.
I20260709 13:22:55.304721 25981 leader_election.cc:304] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, 86ab90de6ed14e13b5ee21a100ee8151; no voters: 
I20260709 13:22:55.305161 25939 raft_consensus.cc:2468] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 1.
I20260709 13:22:55.305805 26217 raft_consensus.cc:2804] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:55.306496 26217 raft_consensus.cc:697] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [term 1 LEADER]: Becoming Leader. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Running, Role: LEADER
I20260709 13:22:55.306313 26181 raft_consensus.cc:515] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.308020 26217 consensus_queue.cc:237] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.309609 26181 leader_election.cc:290] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 election: Requested vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:55.309624 25939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0cf16ae335440edba437fd87306c982" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599"
I20260709 13:22:55.309852 26015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0cf16ae335440edba437fd87306c982" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:55.310602 25939 raft_consensus.cc:3060] T b0cf16ae335440edba437fd87306c982 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:55.310698 26015 raft_consensus.cc:3060] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:55.313905 26251 raft_consensus.cc:493] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:55.314709 26251 raft_consensus.cc:515] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.318439 26017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e528a28b0d34e54887b528172f326eb" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046" is_pre_election: true
I20260709 13:22:55.319223 26017 raft_consensus.cc:2468] T 2e528a28b0d34e54887b528172f326eb P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 0.
I20260709 13:22:55.320797 25907 leader_election.cc:304] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:55.323029 26250 raft_consensus.cc:2804] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:55.323161 26251 leader_election.cc:290] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:55.323868 26086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e528a28b0d34e54887b528172f326eb" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151" is_pre_election: true
I20260709 13:22:55.324823 26250 raft_consensus.cc:493] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:55.325423 26250 raft_consensus.cc:3060] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:55.324741 26086 raft_consensus.cc:2468] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 0.
I20260709 13:22:55.324582 26015 raft_consensus.cc:2468] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 1.
I20260709 13:22:55.324729 25939 raft_consensus.cc:2468] T b0cf16ae335440edba437fd87306c982 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 1.
I20260709 13:22:55.327302 26056 leader_election.cc:304] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, 86ab90de6ed14e13b5ee21a100ee8151; no voters: 
I20260709 13:22:55.328509 26181 raft_consensus.cc:2804] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:55.329411 26181 raft_consensus.cc:697] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Becoming Leader. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Running, Role: LEADER
I20260709 13:22:55.331024 26181 consensus_queue.cc:237] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.338317 26250 raft_consensus.cc:515] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.342092 26015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e528a28b0d34e54887b528172f326eb" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:55.342866 26015 raft_consensus.cc:3060] T 2e528a28b0d34e54887b528172f326eb P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:55.343173 26086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e528a28b0d34e54887b528172f326eb" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
I20260709 13:22:55.344029 26086 raft_consensus.cc:3060] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:55.345803 25839 catalog_manager.cc:5697] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b39791309b04490b6c527fdb0df2046 (127.14.23.66). New cstate: current_term: 1 leader_uuid: "6b39791309b04490b6c527fdb0df2046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:55.348238 26250 leader_election.cc:290] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 election: Requested vote from peers 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:55.356557 26086 raft_consensus.cc:2468] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 1.
I20260709 13:22:55.356557 26015 raft_consensus.cc:2468] T 2e528a28b0d34e54887b528172f326eb P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 1.
I20260709 13:22:55.358213 25906 leader_election.cc:304] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:55.360776 26250 raft_consensus.cc:2804] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:55.361406 26250 raft_consensus.cc:697] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Becoming Leader. State: Replica: f5efb07a61464511be72667a6cab7599, State: Running, Role: LEADER
I20260709 13:22:55.362661 26250 consensus_queue.cc:237] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.370172 25836 catalog_manager.cc:5697] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "86ab90de6ed14e13b5ee21a100ee8151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:55.375509 25838 catalog_manager.cc:5697] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5efb07a61464511be72667a6cab7599 (127.14.23.65). New cstate: current_term: 1 leader_uuid: "f5efb07a61464511be72667a6cab7599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:55.414119 26181 raft_consensus.cc:493] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:55.414701 26181 raft_consensus.cc:515] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.416306 26181 leader_election.cc:290] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:55.417275 25939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3dec675c28487c858ee78013a85c76" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599" is_pre_election: true
I20260709 13:22:55.417389 26015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3dec675c28487c858ee78013a85c76" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046" is_pre_election: true
I20260709 13:22:55.418141 25939 raft_consensus.cc:2468] T 3b3dec675c28487c858ee78013a85c76 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 0.
I20260709 13:22:55.418155 26015 raft_consensus.cc:2468] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 0.
I20260709 13:22:55.419322 26057 leader_election.cc:304] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:55.419970 26181 raft_consensus.cc:2804] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:55.420277 26181 raft_consensus.cc:493] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:55.420540 26181 raft_consensus.cc:3060] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:55.426566 26181 raft_consensus.cc:515] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.428210 26181 leader_election.cc:290] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 election: Requested vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:55.428975 25939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3dec675c28487c858ee78013a85c76" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5efb07a61464511be72667a6cab7599"
I20260709 13:22:55.429358 26015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3dec675c28487c858ee78013a85c76" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:55.429700 25939 raft_consensus.cc:3060] T 3b3dec675c28487c858ee78013a85c76 P f5efb07a61464511be72667a6cab7599 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:55.430066 26015 raft_consensus.cc:3060] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:55.438793 25939 raft_consensus.cc:2468] T 3b3dec675c28487c858ee78013a85c76 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 1.
I20260709 13:22:55.438840 26015 raft_consensus.cc:2468] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 1.
I20260709 13:22:55.440527 26056 leader_election.cc:304] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, 86ab90de6ed14e13b5ee21a100ee8151; no voters: 
I20260709 13:22:55.441574 26181 raft_consensus.cc:2804] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:55.442308 26181 raft_consensus.cc:697] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Becoming Leader. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Running, Role: LEADER
I20260709 13:22:55.443440 26181 consensus_queue.cc:237] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.453118 25838 catalog_manager.cc:5697] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67). New cstate: current_term: 1 leader_uuid: "86ab90de6ed14e13b5ee21a100ee8151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:55.479250 26086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c650179e86e34300b488920b6251f0ef"
dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
mode: GRACEFUL
new_leader_uuid: "f5efb07a61464511be72667a6cab7599"
 from {username='slave'} at 127.0.0.1:51018
I20260709 13:22:55.479892 26086 raft_consensus.cc:606] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Received request to transfer leadership to TS f5efb07a61464511be72667a6cab7599
I20260709 13:22:55.487890 26086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d9971b1d28824061bed7951411f342d1"
dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
mode: GRACEFUL
new_leader_uuid: "f5efb07a61464511be72667a6cab7599"
 from {username='slave'} at 127.0.0.1:51018
I20260709 13:22:55.488430 26086 raft_consensus.cc:606] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Received request to transfer leadership to TS f5efb07a61464511be72667a6cab7599
I20260709 13:22:55.508903 26015 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4833eb35531e43efb9b9ed29308ea39c"
dest_uuid: "6b39791309b04490b6c527fdb0df2046"
mode: GRACEFUL
new_leader_uuid: "f5efb07a61464511be72667a6cab7599"
 from {username='slave'} at 127.0.0.1:46330
I20260709 13:22:55.509413 26015 raft_consensus.cc:606] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [term 1 LEADER]: Received request to transfer leadership to TS f5efb07a61464511be72667a6cab7599
I20260709 13:22:55.517684 26181 consensus_queue.cc:1048] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:55.518296 26086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "982a987942324746b31628f39ac33ee4"
dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
mode: GRACEFUL
new_leader_uuid: "6b39791309b04490b6c527fdb0df2046"
 from {username='slave'} at 127.0.0.1:51018
I20260709 13:22:55.519016 26086 raft_consensus.cc:606] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Received request to transfer leadership to TS 6b39791309b04490b6c527fdb0df2046
I20260709 13:22:55.524530 26086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b0cf16ae335440edba437fd87306c982"
dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
mode: GRACEFUL
new_leader_uuid: "6b39791309b04490b6c527fdb0df2046"
 from {username='slave'} at 127.0.0.1:51018
I20260709 13:22:55.525163 26086 raft_consensus.cc:606] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Received request to transfer leadership to TS 6b39791309b04490b6c527fdb0df2046
I20260709 13:22:55.526291 26256 raft_consensus.cc:993] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151: : Instructing follower f5efb07a61464511be72667a6cab7599 to start an election
I20260709 13:22:55.526961 26255 raft_consensus.cc:1081] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Signalling peer f5efb07a61464511be72667a6cab7599 to start an election
I20260709 13:22:55.529089 25941 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d9971b1d28824061bed7951411f342d1"
dest_uuid: "f5efb07a61464511be72667a6cab7599"
 from {username='slave'} at 127.0.0.1:33210
I20260709 13:22:55.537199 26091 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3b3dec675c28487c858ee78013a85c76"
dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
mode: GRACEFUL
new_leader_uuid: "6b39791309b04490b6c527fdb0df2046"
 from {username='slave'} at 127.0.0.1:51018
I20260709 13:22:55.537950 26091 raft_consensus.cc:606] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Received request to transfer leadership to TS 6b39791309b04490b6c527fdb0df2046
I20260709 13:22:55.543181 25941 raft_consensus.cc:493] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:55.543874 25941 raft_consensus.cc:3060] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:55.548185 26181 consensus_queue.cc:1048] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:55.559062 25941 raft_consensus.cc:515] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.564049 25942 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e528a28b0d34e54887b528172f326eb"
dest_uuid: "f5efb07a61464511be72667a6cab7599"
mode: GRACEFUL
new_leader_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
 from {username='slave'} at 127.0.0.1:33226
I20260709 13:22:55.564703 25942 raft_consensus.cc:606] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Received request to transfer leadership to TS 86ab90de6ed14e13b5ee21a100ee8151
I20260709 13:22:55.566629 26217 consensus_queue.cc:1048] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:55.573107 26015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9971b1d28824061bed7951411f342d1" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:55.573246 26091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9971b1d28824061bed7951411f342d1" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
I20260709 13:22:55.573833 26015 raft_consensus.cc:3060] T d9971b1d28824061bed7951411f342d1 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:55.573951 26091 raft_consensus.cc:3055] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:55.574450 26091 raft_consensus.cc:740] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Running, Role: LEADER
I20260709 13:22:55.575474 26091 consensus_queue.cc:260] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.576776 26091 raft_consensus.cc:3060] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:55.582214 26250 consensus_queue.cc:1048] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:55.583874 26015 raft_consensus.cc:2468] T d9971b1d28824061bed7951411f342d1 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 2.
I20260709 13:22:55.585484 25941 leader_election.cc:290] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 2 election: Requested vote from peers 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:55.585377 25907 leader_election.cc:304] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:55.586836 26250 raft_consensus.cc:2804] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:55.587263 26250 raft_consensus.cc:697] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599 [term 2 LEADER]: Becoming Leader. State: Replica: f5efb07a61464511be72667a6cab7599, State: Running, Role: LEADER
I20260709 13:22:55.585389 26091 raft_consensus.cc:2468] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 2.
I20260709 13:22:55.588598 26250 consensus_queue.cc:237] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.592070 26217 consensus_queue.cc:1048] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:55.606197 25836 catalog_manager.cc:5697] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599 reported cstate change: term changed from 1 to 2, leader changed from 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67) to f5efb07a61464511be72667a6cab7599 (127.14.23.65). New cstate: current_term: 2 leader_uuid: "f5efb07a61464511be72667a6cab7599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:55.609319 26250 consensus_queue.cc:1048] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:55.626019 26256 consensus_queue.cc:1048] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:55.645633 26256 consensus_queue.cc:1048] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:55.662676 26181 raft_consensus.cc:993] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151: : Instructing follower f5efb07a61464511be72667a6cab7599 to start an election
I20260709 13:22:55.663156 26256 raft_consensus.cc:1081] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Signalling peer f5efb07a61464511be72667a6cab7599 to start an election
I20260709 13:22:55.664710 25942 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c650179e86e34300b488920b6251f0ef"
dest_uuid: "f5efb07a61464511be72667a6cab7599"
 from {username='slave'} at 127.0.0.1:33210
I20260709 13:22:55.665293 25942 raft_consensus.cc:493] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:55.665730 25942 raft_consensus.cc:3060] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:55.672506 25942 raft_consensus.cc:515] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.674256 25942 leader_election.cc:290] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 2 election: Requested vote from peers 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:55.675012 26015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c650179e86e34300b488920b6251f0ef" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:55.675307 26086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c650179e86e34300b488920b6251f0ef" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
I20260709 13:22:55.675612 26015 raft_consensus.cc:3060] T c650179e86e34300b488920b6251f0ef P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:55.675876 26086 raft_consensus.cc:3055] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:55.676191 26086 raft_consensus.cc:740] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Running, Role: LEADER
I20260709 13:22:55.676936 26086 consensus_queue.cc:260] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.677975 26086 raft_consensus.cc:3060] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:55.681471 26015 raft_consensus.cc:2468] T c650179e86e34300b488920b6251f0ef P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 2.
I20260709 13:22:55.682694 25907 leader_election.cc:304] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:55.683501 26250 raft_consensus.cc:2804] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:55.683709 26086 raft_consensus.cc:2468] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 2.
I20260709 13:22:55.683879 26250 raft_consensus.cc:697] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599 [term 2 LEADER]: Becoming Leader. State: Replica: f5efb07a61464511be72667a6cab7599, State: Running, Role: LEADER
I20260709 13:22:55.684897 26250 consensus_queue.cc:237] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.691298 25840 catalog_manager.cc:5697] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599 reported cstate change: term changed from 1 to 2, leader changed from 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67) to f5efb07a61464511be72667a6cab7599 (127.14.23.65). New cstate: current_term: 2 leader_uuid: "f5efb07a61464511be72667a6cab7599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:55.709026 26256 consensus_queue.cc:1048] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:55.720965 26181 consensus_queue.cc:1048] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:55.729971 26255 raft_consensus.cc:993] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151: : Instructing follower 6b39791309b04490b6c527fdb0df2046 to start an election
I20260709 13:22:55.730476 26255 raft_consensus.cc:1081] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Signalling peer 6b39791309b04490b6c527fdb0df2046 to start an election
I20260709 13:22:55.732456 26011 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "982a987942324746b31628f39ac33ee4"
dest_uuid: "6b39791309b04490b6c527fdb0df2046"
 from {username='slave'} at 127.0.0.1:48450
I20260709 13:22:55.732997 26011 raft_consensus.cc:493] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:55.733439 26011 raft_consensus.cc:3060] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:55.740046 26011 raft_consensus.cc:515] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.742013 26011 leader_election.cc:290] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 2 election: Requested vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:55.742769 25942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "982a987942324746b31628f39ac33ee4" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f5efb07a61464511be72667a6cab7599"
I20260709 13:22:55.742990 26086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "982a987942324746b31628f39ac33ee4" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
I20260709 13:22:55.743567 25942 raft_consensus.cc:3060] T 982a987942324746b31628f39ac33ee4 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:55.743621 26086 raft_consensus.cc:3055] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:55.744117 26086 raft_consensus.cc:740] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Running, Role: LEADER
I20260709 13:22:55.744932 26086 consensus_queue.cc:260] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.746004 26086 raft_consensus.cc:3060] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:55.750077 25942 raft_consensus.cc:2468] T 982a987942324746b31628f39ac33ee4 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 2.
I20260709 13:22:55.751456 25983 leader_election.cc:304] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:55.751837 26086 raft_consensus.cc:2468] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 2.
I20260709 13:22:55.752234 26217 raft_consensus.cc:2804] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:55.752810 26217 raft_consensus.cc:697] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046 [term 2 LEADER]: Becoming Leader. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Running, Role: LEADER
I20260709 13:22:55.753592 26217 consensus_queue.cc:237] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.754580 26255 consensus_queue.cc:1048] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:55.761140 25840 catalog_manager.cc:5697] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046 reported cstate change: term changed from 1 to 2, leader changed from 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67) to 6b39791309b04490b6c527fdb0df2046 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "6b39791309b04490b6c527fdb0df2046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:55.770360 26255 consensus_queue.cc:1048] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:55.778980 26255 raft_consensus.cc:993] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151: : Instructing follower 6b39791309b04490b6c527fdb0df2046 to start an election
I20260709 13:22:55.779150 26217 consensus_queue.cc:1048] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:55.779476 26181 raft_consensus.cc:1081] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Signalling peer 6b39791309b04490b6c527fdb0df2046 to start an election
I20260709 13:22:55.781427 26015 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b0cf16ae335440edba437fd87306c982"
dest_uuid: "6b39791309b04490b6c527fdb0df2046"
 from {username='slave'} at 127.0.0.1:48450
I20260709 13:22:55.781937 26015 raft_consensus.cc:493] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:55.782560 26015 raft_consensus.cc:3060] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:55.789403 26015 raft_consensus.cc:515] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.790761 26289 raft_consensus.cc:993] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046: : Instructing follower f5efb07a61464511be72667a6cab7599 to start an election
I20260709 13:22:55.791393 26137 raft_consensus.cc:1081] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [term 1 LEADER]: Signalling peer f5efb07a61464511be72667a6cab7599 to start an election
I20260709 13:22:55.793673 26086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0cf16ae335440edba437fd87306c982" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
I20260709 13:22:55.793903 25941 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4833eb35531e43efb9b9ed29308ea39c"
dest_uuid: "f5efb07a61464511be72667a6cab7599"
 from {username='slave'} at 127.0.0.1:33224
I20260709 13:22:55.793973 25939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0cf16ae335440edba437fd87306c982" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f5efb07a61464511be72667a6cab7599"
I20260709 13:22:55.794386 26086 raft_consensus.cc:3055] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:55.794684 25941 raft_consensus.cc:493] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:55.794867 26015 leader_election.cc:290] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 2 election: Requested vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:55.794763 25939 raft_consensus.cc:3060] T b0cf16ae335440edba437fd87306c982 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:55.795190 25941 raft_consensus.cc:3060] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:55.794772 26086 raft_consensus.cc:740] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Running, Role: LEADER
I20260709 13:22:55.797127 26086 consensus_queue.cc:260] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.798087 26137 consensus_queue.cc:1048] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:55.798426 26086 raft_consensus.cc:3060] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:55.801394 25941 raft_consensus.cc:515] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.804737 26091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4833eb35531e43efb9b9ed29308ea39c" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
I20260709 13:22:55.804118 25941 leader_election.cc:290] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 2 election: Requested vote from peers 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:55.807227 26015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4833eb35531e43efb9b9ed29308ea39c" candidate_uuid: "f5efb07a61464511be72667a6cab7599" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:55.807838 26015 raft_consensus.cc:3055] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:55.808161 26015 raft_consensus.cc:740] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Running, Role: LEADER
I20260709 13:22:55.808979 26015 consensus_queue.cc:260] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
W20260709 13:22:55.810057 26137 consensus_queue.cc:1175] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "86ab90de6ed14e13b5ee21a100ee8151" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 13:22:55.811120 26015 raft_consensus.cc:3060] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:55.813392 25939 raft_consensus.cc:2468] T b0cf16ae335440edba437fd87306c982 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 2.
I20260709 13:22:55.814626 25983 leader_election.cc:304] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:55.815527 26137 raft_consensus.cc:2804] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:55.816013 26137 raft_consensus.cc:697] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046 [term 2 LEADER]: Becoming Leader. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Running, Role: LEADER
I20260709 13:22:55.816934 25942 raft_consensus.cc:1240] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Rejecting Update request from peer 6b39791309b04490b6c527fdb0df2046 for earlier term 1. Current term is 2. Ops: []
I20260709 13:22:55.817028 26137 consensus_queue.cc:237] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.818778 26015 raft_consensus.cc:2468] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5efb07a61464511be72667a6cab7599 in term 2.
I20260709 13:22:55.819732 26086 raft_consensus.cc:2468] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 2.
I20260709 13:22:55.820031 25907 leader_election.cc:304] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, f5efb07a61464511be72667a6cab7599; no voters: 86ab90de6ed14e13b5ee21a100ee8151
I20260709 13:22:55.822149 26250 raft_consensus.cc:2804] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:55.822676 26250 raft_consensus.cc:697] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599 [term 2 LEADER]: Becoming Leader. State: Replica: f5efb07a61464511be72667a6cab7599, State: Running, Role: LEADER
I20260709 13:22:55.823626 26250 consensus_queue.cc:237] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.831053 25838 catalog_manager.cc:5697] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599 reported cstate change: term changed from 1 to 2, leader changed from 6b39791309b04490b6c527fdb0df2046 (127.14.23.66) to f5efb07a61464511be72667a6cab7599 (127.14.23.65). New cstate: current_term: 2 leader_uuid: "f5efb07a61464511be72667a6cab7599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:55.838344 25840 catalog_manager.cc:5697] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046 reported cstate change: term changed from 1 to 2, leader changed from 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67) to 6b39791309b04490b6c527fdb0df2046 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "6b39791309b04490b6c527fdb0df2046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:55.848230 26249 consensus_queue.cc:1048] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:55.859597 26251 consensus_queue.cc:1048] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:55.867508 26250 raft_consensus.cc:993] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599: : Instructing follower 86ab90de6ed14e13b5ee21a100ee8151 to start an election
I20260709 13:22:55.868019 26251 raft_consensus.cc:1081] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Signalling peer 86ab90de6ed14e13b5ee21a100ee8151 to start an election
I20260709 13:22:55.869663 26087 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2e528a28b0d34e54887b528172f326eb"
dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
 from {username='slave'} at 127.0.0.1:51008
I20260709 13:22:55.870128 26087 raft_consensus.cc:493] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:55.870405 26087 raft_consensus.cc:3060] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:55.877009 26087 raft_consensus.cc:515] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.878686 26087 leader_election.cc:290] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 2 election: Requested vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:55.879698 25942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e528a28b0d34e54887b528172f326eb" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f5efb07a61464511be72667a6cab7599"
I20260709 13:22:55.879870 26015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e528a28b0d34e54887b528172f326eb" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:55.880427 25942 raft_consensus.cc:3055] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:55.880501 26015 raft_consensus.cc:3060] T 2e528a28b0d34e54887b528172f326eb P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:55.880808 25942 raft_consensus.cc:740] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5efb07a61464511be72667a6cab7599, State: Running, Role: LEADER
I20260709 13:22:55.881721 25942 consensus_queue.cc:260] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.883092 25942 raft_consensus.cc:3060] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:55.886694 26015 raft_consensus.cc:2468] T 2e528a28b0d34e54887b528172f326eb P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 2.
I20260709 13:22:55.888006 26056 leader_election.cc:304] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, 86ab90de6ed14e13b5ee21a100ee8151; no voters: 
I20260709 13:22:55.888785 25942 raft_consensus.cc:2468] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 2.
I20260709 13:22:55.888841 26181 raft_consensus.cc:2804] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:55.889442 26181 raft_consensus.cc:697] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 LEADER]: Becoming Leader. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Running, Role: LEADER
I20260709 13:22:55.890435 26181 consensus_queue.cc:237] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.897488 25840 catalog_manager.cc:5697] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151 reported cstate change: term changed from 1 to 2, leader changed from f5efb07a61464511be72667a6cab7599 (127.14.23.65) to 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "86ab90de6ed14e13b5ee21a100ee8151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:55.906800 26181 consensus_queue.cc:1048] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:55.919099 26255 consensus_queue.cc:1048] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:55.926699 26255 raft_consensus.cc:993] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151: : Instructing follower 6b39791309b04490b6c527fdb0df2046 to start an election
I20260709 13:22:55.927191 26181 raft_consensus.cc:1081] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Signalling peer 6b39791309b04490b6c527fdb0df2046 to start an election
I20260709 13:22:55.928694 26015 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3b3dec675c28487c858ee78013a85c76"
dest_uuid: "6b39791309b04490b6c527fdb0df2046"
 from {username='slave'} at 127.0.0.1:48450
I20260709 13:22:55.929185 26015 raft_consensus.cc:493] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:55.929502 26015 raft_consensus.cc:3060] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:55.935273 26015 raft_consensus.cc:515] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.937646 26015 leader_election.cc:290] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 2 election: Requested vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67:38207)
I20260709 13:22:55.938203 26087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3dec675c28487c858ee78013a85c76" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
I20260709 13:22:55.938946 26087 raft_consensus.cc:3055] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:55.938210 25942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3dec675c28487c858ee78013a85c76" candidate_uuid: "6b39791309b04490b6c527fdb0df2046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f5efb07a61464511be72667a6cab7599"
I20260709 13:22:55.939416 26087 raft_consensus.cc:740] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Running, Role: LEADER
I20260709 13:22:55.939690 25942 raft_consensus.cc:3060] T 3b3dec675c28487c858ee78013a85c76 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:55.940436 26087 consensus_queue.cc:260] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.941835 26087 raft_consensus.cc:3060] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:55.947304 25942 raft_consensus.cc:2468] T 3b3dec675c28487c858ee78013a85c76 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 2.
I20260709 13:22:55.948746 25983 leader_election.cc:304] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:55.949595 26137 raft_consensus.cc:2804] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:55.949434 26087 raft_consensus.cc:2468] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b39791309b04490b6c527fdb0df2046 in term 2.
I20260709 13:22:55.950317 26137 raft_consensus.cc:697] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046 [term 2 LEADER]: Becoming Leader. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Running, Role: LEADER
I20260709 13:22:55.951176 26137 consensus_queue.cc:237] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:55.958997 25838 catalog_manager.cc:5697] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046 reported cstate change: term changed from 1 to 2, leader changed from 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67) to 6b39791309b04490b6c527fdb0df2046 (127.14.23.66). New cstate: current_term: 2 leader_uuid: "6b39791309b04490b6c527fdb0df2046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:56.183693 26087 raft_consensus.cc:1275] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Refusing update from remote peer f5efb07a61464511be72667a6cab7599: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:56.185087 26251 consensus_queue.cc:1048] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:56.191128 26015 raft_consensus.cc:1275] T d9971b1d28824061bed7951411f342d1 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Refusing update from remote peer f5efb07a61464511be72667a6cab7599: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:56.195066 26250 consensus_queue.cc:1048] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:56.203915 26087 raft_consensus.cc:1275] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Refusing update from remote peer f5efb07a61464511be72667a6cab7599: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:56.205343 26250 consensus_queue.cc:1048] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:56.206633 25942 raft_consensus.cc:1275] T b0cf16ae335440edba437fd87306c982 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Refusing update from remote peer 6b39791309b04490b6c527fdb0df2046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:56.211711 26289 consensus_queue.cc:1048] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:56.221066 26015 raft_consensus.cc:1275] T c650179e86e34300b488920b6251f0ef P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Refusing update from remote peer f5efb07a61464511be72667a6cab7599: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:56.224071 26250 consensus_queue.cc:1048] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:56.229039 26087 raft_consensus.cc:1275] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Refusing update from remote peer 6b39791309b04490b6c527fdb0df2046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:56.231294 26289 consensus_queue.cc:1048] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:56.251025 26087 raft_consensus.cc:3060] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:56.256554 26087 raft_consensus.cc:1275] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Refusing update from remote peer f5efb07a61464511be72667a6cab7599: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:56.257791 26250 consensus_queue.cc:1048] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:56.265954 26015 raft_consensus.cc:1275] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Refusing update from remote peer f5efb07a61464511be72667a6cab7599: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:56.268457 26250 consensus_queue.cc:1048] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:56.268610 26087 raft_consensus.cc:1275] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Refusing update from remote peer 6b39791309b04490b6c527fdb0df2046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:56.270354 26289 consensus_queue.cc:1048] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:56.280858 25939 raft_consensus.cc:1275] T 982a987942324746b31628f39ac33ee4 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Refusing update from remote peer 6b39791309b04490b6c527fdb0df2046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:56.282452 26137 consensus_queue.cc:1048] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:56.407264 26015 raft_consensus.cc:1275] T 2e528a28b0d34e54887b528172f326eb P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Refusing update from remote peer 86ab90de6ed14e13b5ee21a100ee8151: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:56.408466 26181 consensus_queue.cc:1048] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:56.419788 25939 raft_consensus.cc:1275] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Refusing update from remote peer 86ab90de6ed14e13b5ee21a100ee8151: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:56.421445 26305 consensus_queue.cc:1048] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:56.539281 26087 raft_consensus.cc:1275] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Refusing update from remote peer 6b39791309b04490b6c527fdb0df2046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:56.540728 26217 consensus_queue.cc:1048] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:56.549665 25942 raft_consensus.cc:1275] T 3b3dec675c28487c858ee78013a85c76 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Refusing update from remote peer 6b39791309b04490b6c527fdb0df2046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:56.550997 26137 consensus_queue.cc:1048] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:58.568619 25836 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38910:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:22:58.589664 25836 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38910:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:22:58.650314 14429 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:58.698161 26343 tablet.cc:1724] T c650179e86e34300b488920b6251f0ef P 6b39791309b04490b6c527fdb0df2046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:58.727610 26011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c5cf4443b384aee96a6383ab797603c"
dest_uuid: "6b39791309b04490b6c527fdb0df2046"
mode: GRACEFUL
new_leader_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
 from {username='slave'} at 127.0.0.1:46346
I20260709 13:22:58.728400 26011 raft_consensus.cc:606] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [term 2 LEADER]: Received request to transfer leadership to TS 86ab90de6ed14e13b5ee21a100ee8151
W20260709 13:22:58.751685 25906 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:38207, user_credentials={real_user=slave}} blocked reactor thread for 71379.9us
W20260709 13:22:58.769666 25907 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.66:40197, user_credentials={real_user=slave}} blocked reactor thread for 93772us
I20260709 13:22:58.762007 26341 tablet.cc:1724] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:58.760047 26343 tablet.cc:1724] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:58.804329 26358 tablet.cc:1724] T 982a987942324746b31628f39ac33ee4 P f5efb07a61464511be72667a6cab7599: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:58.803457 26364 tablet.cc:1724] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:58.863652 26341 tablet.cc:1724] T b0cf16ae335440edba437fd87306c982 P f5efb07a61464511be72667a6cab7599: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:58.859838 26357 log.cc:927] Time spent T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151: Append to log took a long time: real 0.068s	user 0.002s	sys 0.000s
I20260709 13:22:58.871826 26366 tablet.cc:1724] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:58.906641 26376 tablet.cc:1724] T 2e528a28b0d34e54887b528172f326eb P 6b39791309b04490b6c527fdb0df2046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:58.941131 26375 tablet.cc:1724] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:58.979492 26372 tablet.cc:1724] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:58.982491 26379 tablet.cc:1724] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:58.922901 26373 tablet.cc:1724] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:58.943984 26343 tablet.cc:1724] T d9971b1d28824061bed7951411f342d1 P 6b39791309b04490b6c527fdb0df2046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:59.005084 26376 tablet.cc:1724] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:59.011282 26358 tablet.cc:1724] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:59.013753 25939 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6e06dcb93a347bfb244d15cc49cc104"
dest_uuid: "f5efb07a61464511be72667a6cab7599"
mode: GRACEFUL
new_leader_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
 from {username='slave'} at 127.0.0.1:52412
I20260709 13:22:59.014451 25939 raft_consensus.cc:606] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Received request to transfer leadership to TS 86ab90de6ed14e13b5ee21a100ee8151
I20260709 13:22:59.015962 14429 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:22:59.016592 14429 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:59.022504 26382 tablet.cc:1724] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:59.026065 26364 tablet.cc:1724] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:59.032652 26380 tablet.cc:1724] T f5867e87b6214e5595f804343a724e00 P f5efb07a61464511be72667a6cab7599: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:59.040737 26376 tablet.cc:1724] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:59.021582 26341 tablet.cc:1724] T 3b3dec675c28487c858ee78013a85c76 P f5efb07a61464511be72667a6cab7599: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:59.056346 26375 tablet.cc:1724] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:59.070232 26379 tablet.cc:1724] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:59.078763 26341 tablet.cc:1724] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:59.083516 26366 tablet.cc:1724] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:59.054685 25906 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.23.67:38207, user_credentials={real_user=slave}} blocked reactor thread for 54346.2us
I20260709 13:22:59.088563 26379 tablet.cc:1724] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:59.106222 26366 tablet.cc:1724] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:59.065143 26372 tablet.cc:1724] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:59.116887 26315 raft_consensus.cc:993] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046: : Instructing follower 86ab90de6ed14e13b5ee21a100ee8151 to start an election
I20260709 13:22:59.117373 26315 raft_consensus.cc:1081] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [term 2 LEADER]: Signalling peer 86ab90de6ed14e13b5ee21a100ee8151 to start an election
I20260709 13:22:59.119632 26085 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c5cf4443b384aee96a6383ab797603c"
dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
 from {username='slave'} at 127.0.0.1:51012
I20260709 13:22:59.120239 26085 raft_consensus.cc:493] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:59.120616 26085 raft_consensus.cc:3060] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:59.134241 26085 raft_consensus.cc:515] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:59.136539 26085 leader_election.cc:290] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 3 election: Requested vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:59.137543 26015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5cf4443b384aee96a6383ab797603c" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:59.137543 25939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5cf4443b384aee96a6383ab797603c" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f5efb07a61464511be72667a6cab7599"
I20260709 13:22:59.138370 26015 raft_consensus.cc:3055] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:59.138463 25939 raft_consensus.cc:3060] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:59.138903 26015 raft_consensus.cc:740] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6b39791309b04490b6c527fdb0df2046, State: Running, Role: LEADER
I20260709 13:22:59.139788 26015 consensus_queue.cc:260] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:59.141146 26015 raft_consensus.cc:3060] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:59.146437 25939 raft_consensus.cc:2468] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 3.
I20260709 13:22:59.147258 26015 raft_consensus.cc:2468] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 3.
I20260709 13:22:59.147758 26057 leader_election.cc:304] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86ab90de6ed14e13b5ee21a100ee8151, f5efb07a61464511be72667a6cab7599; no voters: 
I20260709 13:22:59.148703 26354 raft_consensus.cc:2804] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:59.149197 26354 raft_consensus.cc:697] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [term 3 LEADER]: Becoming Leader. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Running, Role: LEADER
I20260709 13:22:59.149132 25840 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9: soft_deleted_table [id=c84d067fbd2845da902d5c7f9ae81ed2] alter complete (version 1)
I20260709 13:22:59.150265 26354 consensus_queue.cc:237] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:59.164556 25838 catalog_manager.cc:5697] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 reported cstate change: term changed from 2 to 3, leader changed from 6b39791309b04490b6c527fdb0df2046 (127.14.23.66) to 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67). New cstate: current_term: 3 leader_uuid: "86ab90de6ed14e13b5ee21a100ee8151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:59.341566 26346 raft_consensus.cc:993] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599: : Instructing follower 86ab90de6ed14e13b5ee21a100ee8151 to start an election
I20260709 13:22:59.342077 26250 raft_consensus.cc:1081] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Signalling peer 86ab90de6ed14e13b5ee21a100ee8151 to start an election
I20260709 13:22:59.343461 26085 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f6e06dcb93a347bfb244d15cc49cc104"
dest_uuid: "86ab90de6ed14e13b5ee21a100ee8151"
 from {username='slave'} at 127.0.0.1:51008
I20260709 13:22:59.343943 26085 raft_consensus.cc:493] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:59.344218 26085 raft_consensus.cc:3060] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:59.349604 26085 raft_consensus.cc:515] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:59.351433 26085 leader_election.cc:290] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 2 election: Requested vote from peers f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725), 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197)
I20260709 13:22:59.352370 25939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e06dcb93a347bfb244d15cc49cc104" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f5efb07a61464511be72667a6cab7599"
I20260709 13:22:59.352438 26015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e06dcb93a347bfb244d15cc49cc104" candidate_uuid: "86ab90de6ed14e13b5ee21a100ee8151" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6b39791309b04490b6c527fdb0df2046"
I20260709 13:22:59.352952 25939 raft_consensus.cc:3055] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:59.353048 26015 raft_consensus.cc:3060] T f6e06dcb93a347bfb244d15cc49cc104 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:59.353219 25939 raft_consensus.cc:740] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5efb07a61464511be72667a6cab7599, State: Running, Role: LEADER
I20260709 13:22:59.353977 25939 consensus_queue.cc:260] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:59.354921 25939 raft_consensus.cc:3060] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:59.357944 26015 raft_consensus.cc:2468] T f6e06dcb93a347bfb244d15cc49cc104 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 2.
I20260709 13:22:59.359078 26056 leader_election.cc:304] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b39791309b04490b6c527fdb0df2046, 86ab90de6ed14e13b5ee21a100ee8151; no voters: 
I20260709 13:22:59.359745 25939 raft_consensus.cc:2468] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86ab90de6ed14e13b5ee21a100ee8151 in term 2.
I20260709 13:22:59.359881 26354 raft_consensus.cc:2804] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:59.360387 26354 raft_consensus.cc:697] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 LEADER]: Becoming Leader. State: Replica: 86ab90de6ed14e13b5ee21a100ee8151, State: Running, Role: LEADER
I20260709 13:22:59.361213 26354 consensus_queue.cc:237] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } }
I20260709 13:22:59.370491 25838 catalog_manager.cc:5697] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151 reported cstate change: term changed from 1 to 2, leader changed from f5efb07a61464511be72667a6cab7599 (127.14.23.65) to 86ab90de6ed14e13b5ee21a100ee8151 (127.14.23.67). New cstate: current_term: 2 leader_uuid: "86ab90de6ed14e13b5ee21a100ee8151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86ab90de6ed14e13b5ee21a100ee8151" member_type: VOTER last_known_addr { host: "127.14.23.67" port: 38207 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:59.661595 25939 raft_consensus.cc:1275] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [term 3 FOLLOWER]: Refusing update from remote peer 86ab90de6ed14e13b5ee21a100ee8151: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:59.662891 26354 consensus_queue.cc:1048] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:59.673445 26015 raft_consensus.cc:1275] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [term 3 FOLLOWER]: Refusing update from remote peer 86ab90de6ed14e13b5ee21a100ee8151: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:59.677085 26393 consensus_queue.cc:1048] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:59.896173 25939 raft_consensus.cc:1275] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Refusing update from remote peer 86ab90de6ed14e13b5ee21a100ee8151: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:59.897272 26393 consensus_queue.cc:1048] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5efb07a61464511be72667a6cab7599" member_type: VOTER last_known_addr { host: "127.14.23.65" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:59.905952 26015 raft_consensus.cc:1275] T f6e06dcb93a347bfb244d15cc49cc104 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Refusing update from remote peer 86ab90de6ed14e13b5ee21a100ee8151: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:59.907811 26354 consensus_queue.cc:1048] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b39791309b04490b6c527fdb0df2046" member_type: VOTER last_known_addr { host: "127.14.23.66" port: 40197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:00.076241 14429 tablet_server.cc:179] TabletServer@127.14.23.65:0 shutting down...
W20260709 13:23:00.080662 26057 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 18 similar messages]
W20260709 13:23:00.090593 26057 consensus_peers.cc:597] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 -> Peer f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725): Couldn't send request to peer f5efb07a61464511be72667a6cab7599. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:00.116286 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:23:00.117161 14429 tablet_replica.cc:333] T 3b3dec675c28487c858ee78013a85c76 P f5efb07a61464511be72667a6cab7599: stopping tablet replica
I20260709 13:23:00.118180 14429 raft_consensus.cc:2243] T 3b3dec675c28487c858ee78013a85c76 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.118665 14429 raft_consensus.cc:2272] T 3b3dec675c28487c858ee78013a85c76 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.120931 14429 tablet_replica.cc:333] T b0cf16ae335440edba437fd87306c982 P f5efb07a61464511be72667a6cab7599: stopping tablet replica
I20260709 13:23:00.121728 14429 raft_consensus.cc:2243] T b0cf16ae335440edba437fd87306c982 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.122295 14429 raft_consensus.cc:2272] T b0cf16ae335440edba437fd87306c982 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.124782 14429 tablet_replica.cc:333] T 982a987942324746b31628f39ac33ee4 P f5efb07a61464511be72667a6cab7599: stopping tablet replica
I20260709 13:23:00.125759 14429 raft_consensus.cc:2243] T 982a987942324746b31628f39ac33ee4 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.126299 14429 raft_consensus.cc:2272] T 982a987942324746b31628f39ac33ee4 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.128418 14429 tablet_replica.cc:333] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599: stopping tablet replica
I20260709 13:23:00.129213 14429 raft_consensus.cc:2243] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:00.130131 14429 raft_consensus.cc:2272] T 2629764551584920aa90bb1e5af028a4 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.132836 14429 tablet_replica.cc:333] T f5867e87b6214e5595f804343a724e00 P f5efb07a61464511be72667a6cab7599: stopping tablet replica
I20260709 13:23:00.133445 14429 raft_consensus.cc:2243] T f5867e87b6214e5595f804343a724e00 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.133873 14429 raft_consensus.cc:2272] T f5867e87b6214e5595f804343a724e00 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.135946 14429 tablet_replica.cc:333] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599: stopping tablet replica
I20260709 13:23:00.136480 14429 raft_consensus.cc:2243] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.136860 14429 raft_consensus.cc:2272] T edbb1098626c47439e5e3b7d2edca307 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.139041 14429 tablet_replica.cc:333] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599: stopping tablet replica
I20260709 13:23:00.139760 14429 raft_consensus.cc:2243] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:00.140497 14429 raft_consensus.cc:2272] T d9971b1d28824061bed7951411f342d1 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.142539 14429 tablet_replica.cc:333] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599: stopping tablet replica
I20260709 13:23:00.143069 14429 raft_consensus.cc:2243] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:00.143795 14429 raft_consensus.cc:2272] T c650179e86e34300b488920b6251f0ef P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.145722 14429 tablet_replica.cc:333] T 2a1e227945bc461f8c4226b73d895b63 P f5efb07a61464511be72667a6cab7599: stopping tablet replica
I20260709 13:23:00.146317 14429 raft_consensus.cc:2243] T 2a1e227945bc461f8c4226b73d895b63 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.146736 14429 raft_consensus.cc:2272] T 2a1e227945bc461f8c4226b73d895b63 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.148775 14429 tablet_replica.cc:333] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599: stopping tablet replica
I20260709 13:23:00.149279 14429 raft_consensus.cc:2243] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.149691 14429 raft_consensus.cc:2272] T 2e528a28b0d34e54887b528172f326eb P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.151564 14429 tablet_replica.cc:333] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599: stopping tablet replica
I20260709 13:23:00.152123 14429 raft_consensus.cc:2243] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.152539 14429 raft_consensus.cc:2272] T 7be8d8fb42cd4874a094e408852cfa43 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.154557 14429 tablet_replica.cc:333] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599: stopping tablet replica
I20260709 13:23:00.155090 14429 raft_consensus.cc:2243] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:00.155898 14429 raft_consensus.cc:2272] T 4a610fca006245cbac0d5f66538993e4 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.158169 14429 tablet_replica.cc:333] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599: stopping tablet replica
I20260709 13:23:00.158767 14429 raft_consensus.cc:2243] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.159202 14429 raft_consensus.cc:2272] T 6c5cf4443b384aee96a6383ab797603c P f5efb07a61464511be72667a6cab7599 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.162217 14429 tablet_replica.cc:333] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599: stopping tablet replica
I20260709 13:23:00.162829 14429 raft_consensus.cc:2243] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:00.163654 14429 raft_consensus.cc:2272] T 64e40e1547c14815b57cd70e9bc884b2 P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.166206 14429 tablet_replica.cc:333] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599: stopping tablet replica
I20260709 13:23:00.166960 14429 raft_consensus.cc:2243] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:00.167742 14429 raft_consensus.cc:2272] T 4833eb35531e43efb9b9ed29308ea39c P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.170013 14429 tablet_replica.cc:333] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599: stopping tablet replica
I20260709 13:23:00.170651 14429 raft_consensus.cc:2243] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.171144 14429 raft_consensus.cc:2272] T f6e06dcb93a347bfb244d15cc49cc104 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.173712 14429 tablet_replica.cc:333] T cd842edf9c67436fb5d5e3172a02908f P f5efb07a61464511be72667a6cab7599: stopping tablet replica
I20260709 13:23:00.174294 14429 raft_consensus.cc:2243] T cd842edf9c67436fb5d5e3172a02908f P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.174760 14429 raft_consensus.cc:2272] T cd842edf9c67436fb5d5e3172a02908f P f5efb07a61464511be72667a6cab7599 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.176860 14429 tablet_replica.cc:333] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599: stopping tablet replica
I20260709 13:23:00.177333 14429 raft_consensus.cc:2243] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:00.178162 14429 raft_consensus.cc:2272] T db9a2bfd8849495da7c36e5712750012 P f5efb07a61464511be72667a6cab7599 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:00.195784 25983 consensus_peers.cc:597] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046 -> Peer f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725): Couldn't send request to peer f5efb07a61464511be72667a6cab7599. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:00.200327 25983 consensus_peers.cc:597] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 -> Peer f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725): Couldn't send request to peer f5efb07a61464511be72667a6cab7599. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:00.221980 14429 tablet_server.cc:196] TabletServer@127.14.23.65:0 shutdown complete.
I20260709 13:23:00.259536 14429 tablet_server.cc:179] TabletServer@127.14.23.66:0 shutting down...
W20260709 13:23:00.270609 26056 consensus_peers.cc:597] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151 -> Peer 6b39791309b04490b6c527fdb0df2046 (127.14.23.66:40197): Couldn't send request to peer 6b39791309b04490b6c527fdb0df2046. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:00.282619 26057 consensus_peers.cc:597] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 -> Peer f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725): Couldn't send request to peer f5efb07a61464511be72667a6cab7599. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:45725: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:00.285988 26057 consensus_peers.cc:597] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151 -> Peer f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725): Couldn't send request to peer f5efb07a61464511be72667a6cab7599. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:45725: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:00.301510 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:23:00.302296 14429 tablet_replica.cc:333] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046: stopping tablet replica
I20260709 13:23:00.302994 14429 raft_consensus.cc:2243] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:00.303824 14429 raft_consensus.cc:2272] T 3b3dec675c28487c858ee78013a85c76 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.305976 14429 tablet_replica.cc:333] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046: stopping tablet replica
I20260709 13:23:00.306639 14429 raft_consensus.cc:2243] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:00.307394 14429 raft_consensus.cc:2272] T b0cf16ae335440edba437fd87306c982 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.309406 14429 tablet_replica.cc:333] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046: stopping tablet replica
I20260709 13:23:00.309979 14429 raft_consensus.cc:2243] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:00.310720 14429 raft_consensus.cc:2272] T 982a987942324746b31628f39ac33ee4 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.313055 14429 tablet_replica.cc:333] T c650179e86e34300b488920b6251f0ef P 6b39791309b04490b6c527fdb0df2046: stopping tablet replica
I20260709 13:23:00.313730 14429 raft_consensus.cc:2243] T c650179e86e34300b488920b6251f0ef P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.314277 14429 raft_consensus.cc:2272] T c650179e86e34300b488920b6251f0ef P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.316336 14429 tablet_replica.cc:333] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046: stopping tablet replica
I20260709 13:23:00.316908 14429 raft_consensus.cc:2243] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:23:00.317241 25983 consensus_peers.cc:597] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 -> Peer f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725): Couldn't send request to peer f5efb07a61464511be72667a6cab7599. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:45725: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:00.317693 14429 raft_consensus.cc:2272] T f5867e87b6214e5595f804343a724e00 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.319844 14429 tablet_replica.cc:333] T 2e528a28b0d34e54887b528172f326eb P 6b39791309b04490b6c527fdb0df2046: stopping tablet replica
I20260709 13:23:00.320636 14429 raft_consensus.cc:2243] T 2e528a28b0d34e54887b528172f326eb P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.321098 14429 raft_consensus.cc:2272] T 2e528a28b0d34e54887b528172f326eb P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.323114 14429 tablet_replica.cc:333] T 7be8d8fb42cd4874a094e408852cfa43 P 6b39791309b04490b6c527fdb0df2046: stopping tablet replica
I20260709 13:23:00.323679 14429 raft_consensus.cc:2243] T 7be8d8fb42cd4874a094e408852cfa43 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.324120 14429 raft_consensus.cc:2272] T 7be8d8fb42cd4874a094e408852cfa43 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.326328 14429 tablet_replica.cc:333] T db9a2bfd8849495da7c36e5712750012 P 6b39791309b04490b6c527fdb0df2046: stopping tablet replica
I20260709 13:23:00.326880 14429 raft_consensus.cc:2243] T db9a2bfd8849495da7c36e5712750012 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.327324 14429 raft_consensus.cc:2272] T db9a2bfd8849495da7c36e5712750012 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.329358 14429 tablet_replica.cc:333] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046: stopping tablet replica
I20260709 13:23:00.330060 14429 raft_consensus.cc:2243] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:00.330891 14429 raft_consensus.cc:2272] T cd842edf9c67436fb5d5e3172a02908f P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.332921 14429 tablet_replica.cc:333] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046: stopping tablet replica
I20260709 13:23:00.333420 14429 raft_consensus.cc:2243] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.333799 14429 raft_consensus.cc:2272] T 2629764551584920aa90bb1e5af028a4 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.335757 14429 tablet_replica.cc:333] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046: stopping tablet replica
I20260709 13:23:00.336270 14429 raft_consensus.cc:2243] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:00.337007 14429 raft_consensus.cc:2272] T edbb1098626c47439e5e3b7d2edca307 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.339067 14429 tablet_replica.cc:333] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046: stopping tablet replica
I20260709 13:23:00.339648 14429 raft_consensus.cc:2243] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.340086 14429 raft_consensus.cc:2272] T 6c5cf4443b384aee96a6383ab797603c P 6b39791309b04490b6c527fdb0df2046 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.342537 14429 tablet_replica.cc:333] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046: stopping tablet replica
I20260709 13:23:00.343089 14429 raft_consensus.cc:2243] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.343525 14429 raft_consensus.cc:2272] T 4a610fca006245cbac0d5f66538993e4 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.345638 14429 tablet_replica.cc:333] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046: stopping tablet replica
I20260709 13:23:00.346318 14429 raft_consensus.cc:2243] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.346763 14429 raft_consensus.cc:2272] T 4833eb35531e43efb9b9ed29308ea39c P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.348814 14429 tablet_replica.cc:333] T f6e06dcb93a347bfb244d15cc49cc104 P 6b39791309b04490b6c527fdb0df2046: stopping tablet replica
I20260709 13:23:00.349489 14429 raft_consensus.cc:2243] T f6e06dcb93a347bfb244d15cc49cc104 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.349997 14429 raft_consensus.cc:2272] T f6e06dcb93a347bfb244d15cc49cc104 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.352610 14429 tablet_replica.cc:333] T 64e40e1547c14815b57cd70e9bc884b2 P 6b39791309b04490b6c527fdb0df2046: stopping tablet replica
I20260709 13:23:00.353176 14429 raft_consensus.cc:2243] T 64e40e1547c14815b57cd70e9bc884b2 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.353628 14429 raft_consensus.cc:2272] T 64e40e1547c14815b57cd70e9bc884b2 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.355665 14429 tablet_replica.cc:333] T d9971b1d28824061bed7951411f342d1 P 6b39791309b04490b6c527fdb0df2046: stopping tablet replica
I20260709 13:23:00.356181 14429 raft_consensus.cc:2243] T d9971b1d28824061bed7951411f342d1 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.356631 14429 raft_consensus.cc:2272] T d9971b1d28824061bed7951411f342d1 P 6b39791309b04490b6c527fdb0df2046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.358590 14429 tablet_replica.cc:333] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046: stopping tablet replica
I20260709 13:23:00.359058 14429 raft_consensus.cc:2243] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:00.359838 14429 raft_consensus.cc:2272] T 2a1e227945bc461f8c4226b73d895b63 P 6b39791309b04490b6c527fdb0df2046 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.398543 14429 tablet_server.cc:196] TabletServer@127.14.23.66:0 shutdown complete.
I20260709 13:23:00.431752 14429 tablet_server.cc:179] TabletServer@127.14.23.67:0 shutting down...
I20260709 13:23:00.458985 14429 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:23:00.459676 14429 tablet_replica.cc:333] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151: stopping tablet replica
I20260709 13:23:00.460336 14429 raft_consensus.cc:2243] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.460850 14429 raft_consensus.cc:2272] T 3b3dec675c28487c858ee78013a85c76 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.463006 14429 tablet_replica.cc:333] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151: stopping tablet replica
I20260709 13:23:00.463621 14429 raft_consensus.cc:2243] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.464068 14429 raft_consensus.cc:2272] T b0cf16ae335440edba437fd87306c982 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.466143 14429 tablet_replica.cc:333] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151: stopping tablet replica
I20260709 13:23:00.466733 14429 raft_consensus.cc:2243] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.467171 14429 raft_consensus.cc:2272] T 982a987942324746b31628f39ac33ee4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.469111 14429 tablet_replica.cc:333] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151: stopping tablet replica
I20260709 13:23:00.469681 14429 raft_consensus.cc:2243] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.470149 14429 raft_consensus.cc:2272] T f5867e87b6214e5595f804343a724e00 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.472064 14429 tablet_replica.cc:333] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151: stopping tablet replica
I20260709 13:23:00.472616 14429 raft_consensus.cc:2243] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.473040 14429 raft_consensus.cc:2272] T 2629764551584920aa90bb1e5af028a4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.474921 14429 tablet_replica.cc:333] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151: stopping tablet replica
I20260709 13:23:00.475474 14429 raft_consensus.cc:2243] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.475903 14429 raft_consensus.cc:2272] T c650179e86e34300b488920b6251f0ef P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.478027 14429 tablet_replica.cc:333] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151: stopping tablet replica
I20260709 13:23:00.478771 14429 raft_consensus.cc:2243] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:00.479708 14429 raft_consensus.cc:2272] T 2e528a28b0d34e54887b528172f326eb P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:00.480990 26057 consensus_peers.cc:597] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151 -> Peer f5efb07a61464511be72667a6cab7599 (127.14.23.65:45725): Couldn't send request to peer f5efb07a61464511be72667a6cab7599. Status: Network error: Client connection negotiation failed: client connection to 127.14.23.65:45725: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:00.482064 14429 tablet_replica.cc:333] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151: stopping tablet replica
I20260709 13:23:00.482705 14429 raft_consensus.cc:2243] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:00.483510 14429 raft_consensus.cc:2272] T 7be8d8fb42cd4874a094e408852cfa43 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.485630 14429 tablet_replica.cc:333] T edbb1098626c47439e5e3b7d2edca307 P 86ab90de6ed14e13b5ee21a100ee8151: stopping tablet replica
I20260709 13:23:00.486236 14429 raft_consensus.cc:2243] T edbb1098626c47439e5e3b7d2edca307 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.486676 14429 raft_consensus.cc:2272] T edbb1098626c47439e5e3b7d2edca307 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.488667 14429 tablet_replica.cc:333] T 4a610fca006245cbac0d5f66538993e4 P 86ab90de6ed14e13b5ee21a100ee8151: stopping tablet replica
I20260709 13:23:00.489162 14429 raft_consensus.cc:2243] T 4a610fca006245cbac0d5f66538993e4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.489571 14429 raft_consensus.cc:2272] T 4a610fca006245cbac0d5f66538993e4 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.491612 14429 tablet_replica.cc:333] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151: stopping tablet replica
I20260709 13:23:00.492091 14429 raft_consensus.cc:2243] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:23:00.492842 14429 raft_consensus.cc:2272] T 6c5cf4443b384aee96a6383ab797603c P 86ab90de6ed14e13b5ee21a100ee8151 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.495072 14429 tablet_replica.cc:333] T 64e40e1547c14815b57cd70e9bc884b2 P 86ab90de6ed14e13b5ee21a100ee8151: stopping tablet replica
I20260709 13:23:00.495599 14429 raft_consensus.cc:2243] T 64e40e1547c14815b57cd70e9bc884b2 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.496006 14429 raft_consensus.cc:2272] T 64e40e1547c14815b57cd70e9bc884b2 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.498005 14429 tablet_replica.cc:333] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151: stopping tablet replica
I20260709 13:23:00.498487 14429 raft_consensus.cc:2243] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.498900 14429 raft_consensus.cc:2272] T 4833eb35531e43efb9b9ed29308ea39c P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.500697 14429 tablet_replica.cc:333] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151: stopping tablet replica
I20260709 13:23:00.501214 14429 raft_consensus.cc:2243] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:00.502034 14429 raft_consensus.cc:2272] T f6e06dcb93a347bfb244d15cc49cc104 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.504516 14429 tablet_replica.cc:333] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151: stopping tablet replica
I20260709 13:23:00.505056 14429 raft_consensus.cc:2243] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.505470 14429 raft_consensus.cc:2272] T d9971b1d28824061bed7951411f342d1 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.507344 14429 tablet_replica.cc:333] T 2a1e227945bc461f8c4226b73d895b63 P 86ab90de6ed14e13b5ee21a100ee8151: stopping tablet replica
I20260709 13:23:00.507839 14429 raft_consensus.cc:2243] T 2a1e227945bc461f8c4226b73d895b63 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.508237 14429 raft_consensus.cc:2272] T 2a1e227945bc461f8c4226b73d895b63 P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.510282 14429 tablet_replica.cc:333] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151: stopping tablet replica
I20260709 13:23:00.510751 14429 raft_consensus.cc:2243] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.511189 14429 raft_consensus.cc:2272] T db9a2bfd8849495da7c36e5712750012 P 86ab90de6ed14e13b5ee21a100ee8151 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.513110 14429 tablet_replica.cc:333] T cd842edf9c67436fb5d5e3172a02908f P 86ab90de6ed14e13b5ee21a100ee8151: stopping tablet replica
I20260709 13:23:00.513561 14429 raft_consensus.cc:2243] T cd842edf9c67436fb5d5e3172a02908f P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:00.513998 14429 raft_consensus.cc:2272] T cd842edf9c67436fb5d5e3172a02908f P 86ab90de6ed14e13b5ee21a100ee8151 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.554148 14429 tablet_server.cc:196] TabletServer@127.14.23.67:0 shutdown complete.
I20260709 13:23:00.585208 14429 master.cc:562] Master@127.14.23.126:41765 shutting down...
I20260709 13:23:00.613667 14429 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:00.614221 14429 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.614548 14429 tablet_replica.cc:333] T 00000000000000000000000000000000 P 5c581d2486bd457b9c3575d973c5bee9: stopping tablet replica
I20260709 13:23:00.634270 14429 master.cc:584] Master@127.14.23.126:41765 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (10881 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (26860 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (324957 ms total)
[  PASSED  ] 8 tests.
I20260709 13:23:00.687111 14429 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 392 messages since previous log ~17 seconds ago
I20260709 13:23:00.687343 14429 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 389 messages since previous log ~17 seconds ago
I20260709 13:23:00.687505 14429 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 380 messages since previous log ~17 seconds ago
I20260709 13:23:00.687687 14429 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~35 seconds ago
I20260709 13:23:00.687872 14429 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~35 seconds ago
I20260709 13:23:00.688060 14429 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/master/catalog_manager.cc:5227: suppressed but not reported on 5 messages since previous log ~106 seconds ago
I20260709 13:23:00.688241 14429 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 11 messages since previous log ~161 seconds ago
I20260709 13:23:00.688462 14429 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 7 messages since previous log ~0 seconds ago
I20260709 13:23:00.688736 14429 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~9 seconds ago
